Column property rule

Description

Column property rules allow to set specific operation settings on single columns or tables during the task execution.

The rules can set one or more properties to one or more columns or tables. When the column pattern is non-empty, the rules are applied to the columns that match the pattern. If the pattern is empty, the properties are applied to the table.

Concepts and more information

  • This rule can be configured through the XDM User Interface (recommended) or using a task stage hook. A configuration examples for using a task stage hook can be found in the corresponding section of the configuration examples.

Properties

The table below documents the available properties for column property rules. The 'name' column displays the property name as it can be used in Groovy and Java Scripts.

Name

Type

Default

Description

Active

active

Boolean

true

Specifies whether the rule is active and therefore used, or inactive and therefore ignored, when a task is executed.

Column pattern

columnPattern

String

%

Specifies a selection pattern which is used to match the names of columns. The rule will be applied to the matching columns. If you use patterns for the table schema or table name, then the rule will be applied to all columns which match the column pattern in all matching tables. If the column pattern is left empty, the properties will be set to the table matching the schema and table pattern.

Description

description

String

n/a

An optional description for this object. The description can contain multiple lines to give more context on the configured object. The description is not used in a technical context.

Name pattern

namePattern

String

%

Specifies a selection pattern which is used to match the name of database objects of the specified object type.

Property Map

propertyMap

List

n/a

Contains the properties that are set by this rule. The key and value are strings. The possible keys, values and their definition are listed below this table.

Schema pattern

schemaPattern

String

%

Specifies a selection pattern which is used to match the schema of database objects of the specified object type.

Tags

tags

Tag

n/a

Contains the tags that apply to this object. These tags can be used in the search to find objects quickly and effortlessly.

Actions

The available actions are described below. Some actions apply to the list, while others are specific to selected column property rules.

List Actions

The following actions are available on the column property rules list. If the action is disabled a tooltip will provide the exact reason for the deactivation. The required permissions are described in detail for each action.

  • Bulk Delete

  • Create

  • Export CSV

  • Import CSV

  • List History

Delete the selected objects.

The following options are available:

Cascade

Recursively delete depending objects.

When using cascade, dependent objects are deleted first also with cascade enabled. Thus, a cascade deletion is a recursive function that deeply searches for dependent objects and deletes them first. There is only a confirmation for the first object. The dependent objects are deleted without confirmation but only when the user has the DELETE permission.

This feature is only available in development mode. More information about development mode can be found in the chapter User Settings. It should be used with caution.

An object in the result list can have two different states, these are:

DELETED

The object could be deleted.

NOT_DELETED

The object could be not deleted. This may be because the executing person does not have a delete permission on the object or the object is still referenced by others. A detailed reason can be determined with the help of the error message. If the object is still in use, these objects are also displayed.

The following permissions are required on the list:

  • DELETE

  • READ

Creates a new object in the current list. Depending on the object type either a popup dialog is shown for the most important settings, or the complete object is shown in edit mode. The dialog provides the option to create the object and remain in the current list or to switch to the newly created object in edit mode to perform further changes.

The following permissions are required on the list:

  • CREATE

Exports the current list in CSV format. This will start a download operation for your browser.

The following permissions are required on the list:

  • READ

Creates new objects in the list from a CSV file. The format must comply with the format produced by the export. All imported objects will be added to the list. The import terminates with an error message if an object with the same name already exists and Replace rules is set to false.

Replace rules

The Replace rules option determines whether a rule is appended or replaced. If set to true, all current rules will be replaced with the new rules, otherwise the new rules are appended to the existing rules.

The following permissions are required on the list:

  • WRITE

The history list tracks all modifications made to objects within it. A new record is added each time an object is created, edited, or deleted. A record indicates who made the change, which object was affected, and when the change was made.

For more information about the concept of the history refer to the history concepts.

The following permissions are required on the list:

  • READ

Object Actions

The following actions are available on specific column property rules. In order to execute the action, the user must possess the necessary permissions for the object. The permissions required for each action are described individually. If the user does not have these permissions, the action will be disabled and the tooltip will provide the exact reason for the deactivation.

  • Check

  • Copy

  • Delete

  • Edit

  • Object History

This action validates the object and its dependencies, reporting configuration errors that could cause issues during task or workflow execution. The validation will cascade through the child objects of the checked objects and objects referenced by them.

For instance, if an installed application of an environment is checked, the check will process the application model, the specified version, the connection, modification sets, and involved modification methods. If an object has rules, all active rules will be checked. The modeling connection and version, including their modification sets and methods, will also be checked. Deactivated objects will not be included in recursive checks, but can be checked individually if the check is executed on the object itself.

Checks often require additional information from the context of the objects being checked, such as necessary connections or custom parameter values. The check will gather information from the objects being checked and use it to perform checks on child objects. Any required additional information must be provided before the check begins. The check queries the user to provide these missing information.

Database object checks

For all rules which reference database objects such as tables, columns, etc, the check verifies that the those objects exist in the database system. If a connection can be inferred from the context, then this connection is used. If no connection is available in the context, it must be specified before the check is executed.

Connection checks

For objects which configure access to external systems, such as connections or storage locations, the configuration check verifies that access can be established using the given credentials. Furthermore, additional operations on database connections are performed to check whether the credential user has the necessary authorization to access relevant database objects. In particular, the credential user’s permission to read source tables and write to target tables is verified. Similarly, for storage locations the check verifies that the credential user has permission to write to the working directory.

Code checks

For all entities containing code segments, such as modification methods or condition scripts, the syntax for the code is checked. This does not check, however, whether at run time all necessary variables are likely to be available.

The following permissions are required:

  • READ

The copy action creates an identical copy of the object. A new entry is created in the object list and all properties in the new object are set identical to the copied object.

The following permissions are required:

  • READ

  • WRITE

Delete the object. If the object is still used by another entity, an error message is displayed, and the object is not deleted. The delete operation must be confirmed in a separate popup.

The following options are available:

Cascade

Recursively delete depending objects.

When using cascade, dependent objects are deleted first also with cascade enabled. Thus, a cascade deletion is a recursive function that deeply searches for dependent objects and deletes them first. There is only a confirmation for the first object. The dependent objects are deleted without confirmation but only when the user has the DELETE permission.

This feature is only available in development mode. More information about development mode can be found in the chapter User Settings. It should be used with caution.

The following permissions are required:

  • DELETE

  • READ

Opens the current entity in edit mode.

The following permissions are required:

  • READ

  • WRITE

The history displays all changes made to the respective XDM object, including any changes made to its rules.

Each change record includes information about the operation performed (e.g. CREATE, UPDATE, DELETE), the timestamp, and the user responsible for the change.

For more information about the concept of the history refer to the history concepts.

The following permissions are required:

  • READ

Column properties

Transport Character Data As Binary

Key: transportCharacterDataAsBinary

Type: Boolean (Possible values: true, false)

Default value: No value set

Only Db2 z/OS. Configurable for character columns. Indicates that the data is transported as binary to prevent any automatic conversion by the database JDBC driver. If used in an RLP or CTC task, in the target database, the following user defined function must exist for the user connecting from XDM to the database.

CREATE FUNCTION BININSERT (HEXTEXT VARCHAR(1024))
RETURNS VARCHAR(512)
LANGUAGE SQL
DETERMINISTIC
CONTAINS SQL
RETURN VARCHAR_BIT_FORMAT(HEXTEXT)

If the function needs to named differently or the function needs to be created in a different schema, the function name and schema can be configured in the task stage hook. For example, if the function needs to be created within a schema, modify the query as follows:

CREATE FUNCTION <MYSCHEMA>.<MYBININSERT> (
  HEXTEXT VARCHAR(20000)
)
RETURNS VARCHAR(10000)
LANGUAGE SQL
DETERMINISTIC
CONTAINS SQL
RETURN VARCHAR_BIT_FORMAT(HEXTEXT);
Replace <MYSCHEMA> with the actual schema name and <MYBININSERT> with the desired function name.

When using a different schema or an own function name, this must be configured using the BININSERT_FUNCTION_NAME task runtime property. The property can be set using the following task stage hook to define the BININSERT function name.

version: 35
data:
- type: "TaskStageHook"
  variation: null
  lastChangedDate: "2024-10-08T12:08:35"
  lastChangedBy: "admin"
  replacePermissions: false
  replaceObject: true
  attributes:
    code: "properties.put('BININSERT_FUNCTION_NAME', '<MYSCHEMA>.<MYBININSERT>');"
    codeType: "GROOVY"
    description: null
    displayName: "Set BININSERT"
    libraries: []
    mappingTableAccessMode: "READ"
    mappingTableContainers: []
    parameters: []
    tags: []
The stage hook should be executed in the PRE Stage 1 phase of the task. When using icebox tasks, the task should be executed in the PRE Stage 1 phase of the from icebox task.

Table properties

Table properties need to be set by using an empty column pattern.

Enable SubsetApply Batch mode

Key: enableApplyTableBatchMode

Type: Boolean (Possible values: true, false)

Default value: No value set

Forces the SubsetApply to enable the batch processing mode for the table. This can be useful for large tables to improve performance. This even enables the batch mode if the activated flag for the connection is not set. Also, if the evaluation of columns would disable the batch mode (e.g. because the table contains LOBs), the batch mode is still enabled. This may lead to an error, depending on the database and the data in the table.

Disable SubsetApply Batch mode

Key: disableApplyTableBatchMode

Type: Boolean (Possible values: true, false)

Default value: No value set

Forces the SubsetApply to disable the batch processing mode for the table. This can be useful in situation, where the batch mode causes problems. Disabling the batch mode leads to decreased performance, but may increase the stability of the task execution.

Manually adjust the fill mode for a table

Key: fillModeForTable

Type: String (Possible values: APPEND, DELETE, INSERT, MERGE, REPLACE, TABLE_REPLACE, UPDATE, NONE)

Default value: No value set

Manually adjusts the fill mode for a table. This can be useful in situations where the fill mode for one table should be different from the others. For example, you might have a start table in a row level processor task that is only used to refer to other tables, but should not be updated itself. In this case, you cannot use an exclude rule or a reduction rule, because then no data will be selected from the start table. Instead, you can use a column property rule to set the fill mode of the start table to "NONE".