Actions used in subsetting

May 20, 2019
|By Thomas Pryce

Search Knowledge Base by Keyword

Created OnMay 20, 2019
byThomas Pryce
< Back

The Subsetting is performed using an automated VIP workflow. This high-speed workflow performs “Actions”. Each Action is informed either by sheets in the Basic/Advanced Control Spreadsheet, or by information read from the Source Database. Some of this information can be user defined; other information is generated by the Subset actions.

The actions to run the Basic Subset are:

  1. The TABLES and GETKEYS Actions retrieve metadata from the Source Database.
  2. The PREPENV Action create tables and indexes in the Staging Database.
  3. The BUILDMODEL creates the rules to drive the Subset.
  4. The SUBSET Action writes data to the Staging Database.

A Subset will run until one of following completion criteria is fulfilled:

  1. A maximum specified number of rows is reached;
  2. A maximum number of recursions is reached;
  3. The “Found Criteria” are fulfilled;
  4. There are no more rows in the Source Database that match the Subset criteria. The Subset will stop recurring when no rows were added in the last recursion.

This Basic Subset will generate the Advanced Control Spreadsheet, containing additional sheets. These sheets contain additional parameters and the automatically formulated Subset rules. The information in these sheets can then be used to perform Advanced Subsets. You can subset iteratively by toggling the Subset Rules, Tables and Relationships that will be used in the next Subset.

Actions used to perform iterative Subsets after the Basic Subset include:

  1. DROP: Drops the tables registered by the TABLES action. You will only run this if something has gone wrong in a Subset, and you want to create a wholly new Data Subset.
    1. PREPENV: If you drop the tables, you will need to re-register tables before performing the next Subset.
  2. TRUNCATE: Deletes data from the Target Database or Schema.
  3. BUILDMODEL: Creates the rules to drive the Advanced Subset, based on the Control Spreadsheet.
  4. SUBSET: Writes the new Data Subset to the Staging Database, based on the updated Control Spreadsheet.