Dynamic Environment-Based Schema and Table Naming in XDM Copy Tasks

Overview

In some installations, schema and table names contain an identifier that reflects the specific environment (e.g., A, B, or C environments). When using Compatibility Table Copy Tasks (CTC) or Native Table Copy Tasks in XDM, these environment identifiers must often be dynamically set based on the source and target connections used.

This guide explains how to dynamically assign schema and table name letters for environments (A, B, C) in Compatibility Copy Tasks or Native Copy Tasks by detecting the environment from the connection used, and then passing it into selection and mapping rules through custom parameters.

Prerequisites

To apply the solution described in this manual, the user should:

Be familiar with XDM platform configuration.
Understand the concept of Custom Parameters in XDM.
Have basic knowledge of Selection Rules and Mapping Rules in Copy Tasks.
Be capable of configuring and executing XDM Workflows with Task Templates.
Understand Groovy-like scripting used in workflow configuration.

Multiple Steps

Step 1: Create Custom Parameters

Define two custom parameters of type STRING to hold the environment identifier (letter) for source and target schemas/tables.

- type: "CustomParameter"
  attributes:
    displayName: "Source Environment Letter"
    variableName: "sourceLetter"
    parameterType: "STRING"
    options:
      - displayName: "A-Environment"
        value: "A"
      - displayName: "B-Environment"
        value: "B"
      - displayName: "C-Environment"
        value: "C"

- type: "CustomParameter"
  attributes:
    displayName: "Target Environment Letter"
    variableName: "targetLetter"
    parameterType: "STRING"
    options:
      - displayName: "A-Environment"
        value: "A"
      - displayName: "B-Environment"
        value: "B"
      - displayName: "C-Environment"
        value: "C"

These parameters will be dynamically filled in the workflow depending on the source/target connections.

Step 2: Use Parameters in Mapping and Selection Rules

Reference the parameters sourceLetter and targetLetter in your rules.

Example: Selection Rule

This rule matches schemas based on the source environment letter:

Selection Type

Pattern

Object Type

TABLE

Schema pattern

DB2${sourceLetter}

Name pattern

TEST${sourceLetter}

Example: Mapping Rule

This rules maps schema names or table names dynamically:

Field Selection Mode

Simple

Object Type

TABLE

Field Type

NAME

Source selection pattern

TEST${sourceLetter}

Target value mode

Simple Pattern

Target Value

%${targetLetter}

Field Selection Mode

Simple

Object Type

ANY

Field Type

SCHEMA

Source selection pattern

DB2${sourceLetter}

Target value mode

Simple Pattern

Target Value

%${targetLetter}

Step 3: Dynamically Assign Parameters in Workflow

Use the workflow to extract the environment letter based on the connection display name. Assign these letters to the corresponding custom parameters. The custom parameters must be added to the workflow.

def sourceConnectionName = workflow.mySource.displayName
def targetConnectionName = workflow.myTarget.displayName

sourceConnectionLetter = ""
targetConnectionLetter = ""

if (sourceConnectionName.startsWith("A")) {
  sourceConnectionLetter = "A"
}
if (sourceConnectionName.startsWith("B")) {
  sourceConnectionLetter = "B"
}
if (sourceConnectionName.startsWith("C")) {
  sourceConnectionLetter = "C"
}

if (targetConnectionName.startsWith("A")) {
  targetConnectionLetter = "A"
}
if (targetConnectionName.startsWith("B")) {
  targetConnectionLetter = "B"
}
if (targetConnectionName.startsWith("C")) {
  targetConnectionLetter = "C"
}

TaskTemplateRunner()
   .taskTemplate('DynamicalCTC')
   .taskNamePattern('Workflow of ${taskTemplate.displayName} at ${.now?iso_local}')
   .parameter('sourceConnection', workflow.mySource)
   .parameter('targetConnection', workflow.myTarget)
   .parameter('sourceLetter', sourceConnectionLetter)
   .parameter('targetLetter', targetConnectionLetter)
   .run()

Note: This logic can be improved using a switch statement or regex matching for more flexible identification of the connection environment.

The complete Workflow can be found here.

Execution Example

If data should be copied from the A environment to the C environment using an XDM task, it is only necessary to create a workflow and select connections A and C for the parameters mySource and myTarget. Then, when the workflow is executed, the A environment is used as the source and the C environment as the target, and the data is copied accordingly.

Benefits

Dynamic configuration: No need to hard-code schema/table names for each environment.
Reusability: The same task/template can be reused across different environments without duplication.
Flexibility: Adding a new environment (e.g., D-Environment) is as simple as extending the parameter options.
Automation: The environment identification is fully automated using connection display names.
Integration-ready: Ideal for usage in DataShops and enhance the workflow integration pipelines.

Conclusion

This approach allows Copy Tasks in XDM to dynamically adapt to different technical environments using flexible schema and table naming. By combining custom parameters with workflow logic, you can ensure scalable and environment-agnostic configuration of your data pipelines. The use of environment-specific letters improves clarity, automation, and maintainability in multi-environment setups.

For further optimization, consider abstracting the environment identification logic into a reusable function or extending the environment mapping to support additional metadata.