Copy the Requisite Files into a Subset folder

All the files used to set up a Subset job are shipped during the installation of the VIP Server Controller.

You will not need to edit most of these files. However, it is useful to understand which files are used during the automated Subsetting Actions. This is valuable when debugging failed Subsetting Actions as a parameter might be pointing to the wrong location. The location of all files is therefore summarised below.

Understand and locate the files used during Subsetting

If you followed the standard installation, these folders will be stored on your C: Drive.

You will find seven folders. Each folder’s name is prefaced with “VIP”:

C:\VIPTDM contains the files used to configure Test Data Automation utilities, including data subsetting.

The files used during subsetting are separated into three folders. One contains the files you will edit and work with. The other contains files that will remain static.

It is strongly recommended that you leave these files in the default location if possible, or at least that they remain in one location. This will let you use many of the same parameters throughout the automated Subset Actions. If you move these files between Actions, you will need to update several additional parameters.

1. C:\VIPTDM\SubsetCloneAndMask – Contains files that you will not edit

This folder contains the generic automated workflows used to perform the actions. Do not edit files in this folder. If possible, the location of this folder will should also remain the same.

The files contained in this folder are:

1. The automated workflows used to perform the Subset Actions:
   1. SQLSubset.enc.vip
   2. SQLActions.enc.vip
2. The .DLL Files associated with the automated Workflows.
3. The .XML files associated with the automation.
4. TXT Files associated with the automation. 

Note: The VIP flows are encrypted and you need a valid subsetting license to run them. If you do not have licensed copies of the above files, please email your Curiosity account manager or email info@Curiosity.Software.

2. C:\VIPTDM\SubsetCloneAndMaskRun – Contains files that you will not edit

This folder contains additional static files used when performing Subsetting, Cloning and Masking. The file relevant to Subsetting is:

1. A FlowCache.txt – this is used to tell the VIP Server Controller the location of VIP workflows to put “in memory”. This improves the performance of the automation.

The folder also contains subfolders for running a “Mask”, “Subset” or “Clone”. These folders contain the files you will need to edit.

3. The “C:\VIPTDM\SubsetCloneAndMaskRun\Subset” – This is your “Run Directory” It contains files that you should Copy and edit

This folder contains the files you will edit when running a Subset (see below).

Create a new Run Directory

C:\VIPTDM\SubsetCloneAndMaskRun\Subset” contains the files you will edit when running a Subset (see below).

This folder is grouped by different data sources. Select the type of data which you wish to Subset:

This folder contains the configuration files that you should edit. These files are:

1. A blank Basic Control Spreadsheet that points the VIP workflow to the database that will be modelled and subsetted. This is called “VIPSubset” followed by the name of the data type.
2. cfg – A “Configuration File” that will load a set of default parameters into The VIP Server Controller. This allows you to quickly edit a set of pre-set parameters and also provides the default location of the static, unedited files used during Subsetting.
3. The standard scripts used to execute the Subset Actions. These scripts are used to parameterize and execution the Subset Actions if executing Actions from the Command Line instead of the VIP Server Controller. The scripts are:
   1. Subset1_MetaData.cmd
   2. Subset2_Prep.cmd
   3. Subset3_Build.cmd
   4. Subset4_Subset.cmd

Note: the file name of each script reflects the action it performs. The numeration reflects the typical order in which you would run the actions when iterating Subsets.

It is recommended that you create a new “Run Directory” folder for each individual Subset. This enables you to easily return to edit individual subsetting jobs. It also avoids unnecessary rework, helping to avoid previously configured Subsets being edited or lost. You might additionally incorporate a file structure where each Subset folder is stored under individual folders for individual Databases or Schemas.

Start by copying all files from the relevant data type folder to a new folder. Below, you can see an example copied from C:\VIPTDM\SubsetCloneAndMaskRun\Subset\SQLServer: