Extract data rule

Description

Extract data rules are used to export column data (e.g. keys) of selected rows into a file, a report, or a table. This is useful when selected column data should be logged with each execution. Such a list might be used to document which rows are used for testing. The list of selected column data can also be used in a start condition for other XDM task templates or other applications.

There are three possible target objects for the extracted data: flat files, tables in a database, or batch reports. For every table that matches the given schema and table pattern, XDM writes the selected keys into a file, table, or report. The target objects need not exist before task execution, they are created during the execution.

It is possible to define more than one extract data rule in a task template. One rule can match multiple tables, then the data of all matching columns of all tables are written into the target object. Also, if multiple rules match on table, the data is written into the different target objects.

Concepts and more information

Rules in XDM

Properties

The table below documents the available properties for extract data rules. The 'name' column displays the property name as it can be used in Groovy and Java Scripts.

Name	Type	Default	Description
Active active	Boolean	true	Specifies whether the rule is active and therefore used, or inactive and therefore ignored, when a task is executed.
Add Meta Data addMetaData	Boolean	false	When Add Meta Data is selected, XDM will append two columns with task execution name and source table name to the target table. Otherwise, only the extracted data will be written.
Append append	Boolean	false	This property is only relevant when the output type is either Table or File. When Append is chosen, XDM will append the new data to the existing stored data. Otherwise, the existing data will be overwritten.
Column pattern columnPattern	String	n/a	Specifies a selection pattern which is used to match the names of columns. The rule will be applied to the matching columns. If you use patterns for the table schema or table name, then the rule will be applied to all columns which match the column pattern in all matching tables.
Output Columns columns	String	n/a	Specifies a comma separated list of column names. For each listed column the respective data value is extracted into the the output resource defined by the rule. If a column does not exist on a table matching the pattern, a null value is written to the resource.
Create Table createTable	Boolean	false	This option is only relevant when the output type is Table. When Create Table is chosen, XDM will create the table during task execution if it does not already exist. If not selected, a key insertion error will occur if the table does not exist.
Description description	String	n/a	An optional description for this object. The description can contain multiple lines to give more context on the configured object. The description is not used in a technical context.
File Encoding fileEncoding	String	n/a	This property is only relevant when the output type is File. Specifies the encoding for the output file. The value is selected from a drop down list of available encodings.
File Name fileName	String	n/a	This property is only relevant when the output type is File. Specifies the name of the file in which the extracted data is stored. A relative path entry (with no leading slash) writes the data into the current tasks execution directory . With an absolute path (with a leading slash), the data is written to the specified directory within docker. This directory must exist. If the specified directory is also defined as a volume mount point of the dataflow container, such as `./xdm-data:/xdm/data`, then the file can be accessed from outside the container.
Name pattern namePattern	String	%	Specifies a selection pattern which is used to match the name of a database table.
Output Connection outputConnection	Connection	n/a	This property is only relevant when the output type is Table. Specifies the database connection for the output table.
Output Schema outputSchemaName	String	n/a	This property is only relevant when the output type is Table. Specifies the schema for the output table.
Output Table outputTableName	String	n/a	This property is only relevant when the output type is Table. Specifies the name of output table in which the extracted data is stored.
Output Type outputType	String	BATCHREPORT	Specifies the form in which the extracted data is returned. The available options are: Batch Report The output is presented as a report in the task execution. File Extracted data is stored as a file. Table Extracted data is stored as a table in the target environment.
Report Name reportName	String	n/a	This property is only relevant when the output type is Batch Report. Specifies the name of the report in which the extracted data will be presented.
Schema pattern schemaPattern	String	%	Specifies a selection pattern which is used to match the schema of a database table.
Scope scope	Scope	SOURCE	Can be set to either Source or Target. If set to Source, the patterns specified in the fields schema pattern and name pattern refer to the names of source objects. If set to Target, they refer to the names of target objects and the extracted data contains, if modification is used, the modified values.
Tags tags	Tag	n/a	Contains the tags that apply to this object. These tags can be used in the search to find objects quickly and effortlessly.

Name

Type

Default

Description

Active

active

Boolean

true

Specifies whether the rule is active and therefore used, or inactive and therefore ignored, when a task is executed.

Add Meta Data

addMetaData

Boolean

false

When Add Meta Data is selected, XDM will append two columns with task execution name and source table name to the target table. Otherwise, only the extracted data will be written.

Append

append

Boolean

false

This property is only relevant when the output type is either Table or File.

When Append is chosen, XDM will append the new data to the existing stored data. Otherwise, the existing data will be overwritten.

Column pattern

columnPattern

String

n/a

Specifies a selection pattern which is used to match the names of columns. The rule will be applied to the matching columns. If you use patterns for the table schema or table name, then the rule will be applied to all columns which match the column pattern in all matching tables.

Output Columns

columns

String

n/a

Specifies a comma separated list of column names. For each listed column the respective data value is extracted into the the output resource defined by the rule. If a column does not exist on a table matching the pattern, a null value is written to the resource.

Create Table

createTable

Boolean

false

This option is only relevant when the output type is Table.

When Create Table is chosen, XDM will create the table during task execution if it does not already exist. If not selected, a key insertion error will occur if the table does not exist.

Description

description

String

n/a

An optional description for this object. The description can contain multiple lines to give more context on the configured object. The description is not used in a technical context.

File Encoding

fileEncoding

String

n/a

This property is only relevant when the output type is File.

Specifies the encoding for the output file. The value is selected from a drop down list of available encodings.

File Name

fileName

String

n/a

This property is only relevant when the output type is File.

Specifies the name of the file in which the extracted data is stored. A relative path entry (with no leading slash) writes the data into the current tasks execution directory . With an absolute path (with a leading slash), the data is written to the specified directory within docker. This directory must exist. If the specified directory is also defined as a volume mount point of the dataflow container, such as ./xdm-data:/xdm/data, then the file can be accessed from outside the container.

Name pattern

namePattern

String

Specifies a selection pattern which is used to match the name of a database table.

Output Connection

outputConnection

Connection

n/a

This property is only relevant when the output type is Table.

Specifies the database connection for the output table.

Output Schema

outputSchemaName

String

n/a

This property is only relevant when the output type is Table.

Specifies the schema for the output table.

Output Table

outputTableName

String

n/a

This property is only relevant when the output type is Table.

Specifies the name of output table in which the extracted data is stored.

Output Type

outputType

String

BATCHREPORT

Specifies the form in which the extracted data is returned. The available options are:

Batch Report: The output is presented as a report in the task execution.
File: Extracted data is stored as a file.
Table: Extracted data is stored as a table in the target environment.

Report Name

reportName

String

n/a

This property is only relevant when the output type is Batch Report.

Specifies the name of the report in which the extracted data will be presented.

Schema pattern

schemaPattern

String

Specifies a selection pattern which is used to match the schema of a database table.

Scope

scope

Scope

SOURCE

Can be set to either Source or Target. If set to Source, the patterns specified in the fields schema pattern and name pattern refer to the names of source objects. If set to Target, they refer to the names of target objects and the extracted data contains, if modification is used, the modified values.

Tags

Actions

The available actions are described below. Some actions apply to the list, while others are specific to selected extract data rules.

List Actions

The following actions are available on the extract data rules list. If the action is disabled a tooltip will provide the exact reason for the deactivation. The required permissions are described in detail for each action.

Bulk Delete
Create
Export CSV
Import CSV
List History

Delete the selected objects.

The following options are available:

Cascade: Recursively delete depending objects.

When using cascade, dependent objects are deleted first also with cascade enabled. Thus, a cascade deletion is a recursive function that deeply searches for dependent objects and deletes them first. There is only a confirmation for the first object. The dependent objects are deleted without confirmation but only when the user has the DELETE permission.

This feature is only available in development mode. More information about development mode can be found in the chapter User Settings. It should be used with caution.

An object in the result list can have two different states, these are:

DELETED: The object could be deleted.
NOT_DELETED: The object could be not deleted. This may be because the executing person does not have a delete permission on the object or the object is still referenced by others. A detailed reason can be determined with the help of the error message. If the object is still in use, these objects are also displayed.