XDM3 configuration examples

Matcher

Read the chapter matcher for more information about matcher.

Germany

  • Bank code - DICTIONARY

  • Bank name - REGEX

  • City - DICTIONARY

  • Company - REGEX

  • First name - DICTIONARY

  • Last name - DICTIONARY

  • Street - REGEX

  • Title - DICTIONARY

  • Zip code - REGEX

International

  • Bank identifier code - DICTIONARY

  • International bank account number - REGEX

  • Phone - REGEX

  • eMail - REGEX

Modification methods

Read the chapter modification method for more information about modification methods. Information about the concept of modification methods can be found in the modify data chapter of Working with XDM.

Some modification methods need the lookup tables to be uploaded in your XDM environment as mapping table XDM Lookup Table. Read the chapter Installation of lookup tables for more information.

Allowance number

Description

Builds an allowance number (Zulagennummer) based on the following input parameters:

  • number of the insurer (place 1-2)

  • date of birth of the insured person (place 3-8)

  • family name of the insured person (place 9)

  • gender, 00-49 male, 50-99 female (place 10-11)

  • check digit (place 12)

The allowance number is equal to the German social security number and is e.g. important to determine the pension income of an employee, to manage health insurance, as well as other issues concerning social security. The number is normally assigned to an employee via the first paycheck. Some employees like public officials, judges and soldiers have to request their allowance number manually. The allowance number is used to determine the increment of the German Riester pension (German subsidized private pension).

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Area Number

Number of the insurer

areaNumber

STRING

-

-

Birth Date

Birth date in the format ddMMYY or YYYY-MM-dd or as Date

birthDate

STRING

-

-

Last Name

Last name at the time of the application

lastName

STRING

-

-

Gender

00-49 for male, 50-99 for female

gender

STRING

-

-

Auto increment

Description

This method generates a new value each time it is called. Internally, it uses a database sequence that provides incrementing values.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Sequence Name

Specifies the name of the sequence.

sequenceName

STRING

AUTO_INCREMENT

-

IncrementBy

Specifies the step size of the sequence. Negative values will lead to a down counting sequence.

incrementBy

STRING

-

-

Start Value

Specifies the value the sequence is starting with.

startValue

STRING

-

-

Min Value

Specifies the lower limit of the sequence. This will be used as start value or at cycling.

minValue

STRING

-

-

Max Value

Specifies the maximum value for the sequence. This will be used as start value for down counting sequences.

maxValue

STRING

-

-

Cycle

Specifies whether the sequence cycles at the borders or causing an error.

cycle

BOOLEAN

FALSE

-

Drop Sequence

Specifies if the sequence is dropped at the end of the modification method finishes.

dropSequence

BOOLEAN

FALSE

-

Bank identifier code

Description

Generates a bank identifier code (also known as ISO 9362 or SWIFT code) based on the specified seed value. The country parameter specifies the country/region of origin ('AT', 'CH', 'DE', 'NL'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the bank identifier code.

country

STRING

-

-

Bank name

Description

Generates a bank name based on the specified seed value. The country parameter specifies the country/region of origin ('AT', 'CH', 'DE', 'NL'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the bank name.

country

STRING

-

-

Canadian social insurance number

Description

Generates a Canadian social insurance number (SIN) based on the specified seed value. The generated SIN is returned as a string. Leading zeroes are preserved.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Specify a value that is used to calculate the hash.

seed

STRING

-

-

Delimiter

The delimiter character used to separate the groups of digits. May be empty.

delimiter

STRING

-

-

City

Description

Generates a city name based on the specified seed value. The country parameter specifies the country/region of origin ('US', 'DE', 'FR', 'IT', 'NL', 'AT', 'BE', 'CA', 'CH', 'UK'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the city.

country

STRING

-

-

Credit card number

Description

Generates a credit card number. The first parameter should be an existing credit card number. The second parameter specifies how many digits are copied unchanged from the original number.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

The original credit card number.

seed

STRING

-

-

Retain

The number of digits to retain from the original number.

retain

NUMBER

-

-

Czech birthday number

Description

Generates a Czech birthday number based on the birthday and sex parameters. The generated number is returned as a string. If the value is a date it will be used as is. All other values will be hashed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Birthday

Specify a value that is used to calculate the hash.

birthday

STRING

-

-

Sex

Specifies the gender of the first name.

sex

STRING

-

-

Delimiter

The delimiter character used to separate the groups of digits. May be empty.

delimiter

STRING

/

-

E Mail

Description

Generates an email address based on the specified seed value. The country parameter specifies the country/region of origin for the first name ('EN', 'DE', 'FR', 'IT', 'NL'). The sex parameter specifies the sex ('M' for male, 'F' for female). For both parameters, only a single value can be specified (e.g. 'DE' and 'F'). The domain parameter specifies the domain name of the email address. If the domain parameter is omitted, it defaults to 'example.com'. This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Specifies the value that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the email.

country

STRING

-

-

Sex

Specifies the gender of the first name.

sex

STRING

-

-

Domain

Domain that is used to generate e-mail addresses.

domain

STRING

example.com

-

First name

Description

Generates a first name based on the specified seed value. The country parameter specifies the country/region of origin for the first name ('EN', 'DE', 'FR', 'IT', 'NL'). The sex parameter specifies the sex ('M' for male, 'F' for female). For both parameters, only a single value can be specified (e.g. 'DE' and 'F'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Specifies the value that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the first name.

country

STRING

-

-

Sex

Specifies the gender of the first name.

sex

STRING

-

-

German tax identification number

Description

This modification method works column based. The input is the value of the current column, which is used to generate a number looking like the German tax identification number, which has to fulfill the following requirements:

  • The first number must not be 0

  • In the first 10 digits there is one number double or triple

  • If there are 3 same numbers at the position 1 to 10 those double numbers could never be consecutive

Information about how the check digit is generated and the criteria of a valid number: Prüfziffernberechnung (PDF): https://www.zfa.deutsche-rentenversicherung-bund.de/de/Inhalt/public/4_ID/47_Pruefziffernberechnung/001_Pruefziffernberechnung.pdf?__blob=publicationFile&v=2

Background information about the number: Tax identification number – BZSt: https://www.bzst.de/EN/Private_individuals/Tax_identification_number/tax_identification_number_node.html

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

The original German tax identification number

seed

STRING

-

-

IMEI

Description

Generates an IMEI based on an existing IMEI and the specified seed value.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Specifies the value that is used to calculate the hash.

seed

STRING

-

-

International bank account number

Description

Generates an international bank account number (IBAN) based on the specified seed value. The country parameter specifies the country/region of origin ('AT', 'CH', 'DE', 'NL'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the international bank account number.

country

STRING

-

-

Last name

Description

Generates a last name based on the specified seed value. The country parameter specifies the country/region of origin for the last name ('EN', 'DE', 'FR', 'IT', 'NL'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Specifies the value that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the last name.

country

STRING

-

-

Latitude

Description

Generates a latitude component of geographical coordinates based on the specified seed value. The country parameter specifies the country/region of origin ('US', 'DE', 'FR', 'IT', 'NL', 'AT', 'BE', 'CA', 'CH', 'UK'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the latitude.

country

STRING

-

-

Longitude

Description

Generates the longitude component of geographical coordinates based on the specified seed value. The country parameter specifies the country/region of origin ('US', 'DE', 'FR', 'IT', 'NL', 'AT', 'BE', 'CA', 'CH', 'UK'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the longitude.

country

STRING

-

-

Modify time data

Description

Change a time data value by modifying it or generate or set a new value. Input can be a date, time, timestamp or string column. This method internally parses each of the input values to a timestamp and then allows to modify each part of this timestamp separately. For each of the parts (year, month, day, hour, minute, second, fractions) one of the following values can be specified: * <value> → Set the specified value

  • +<value> → to add the specified amount to this part

  • -<value> → to subtract a specified amount from this part

  • NOW → take the value of the current timestamp

  • NOW+<value> → to add a given amount to the current timestamps part and use it

  • NOW-<value> → to subtract a given amount from the current timestamps part and use it

  • SEED(<min>:<max>) → Generate a new value for this part based on a given seed. The seed can be a column name of the table to use the columns value. Otherwise the seed value will be used as it is. The part in brackets is optional and can limit the range of the generated value. <min> → specifies the low limit for the generated value. <max> → specifies the high limit for the generated value. Both <min> and <max> are optional. If not specified the default value for the limit will be used. The seed value has to be specified in parameter seed.

  • +SEED(<min>:<max>) → Generate a new value like SEED(<min>:<max>) but this value will be added to the current value of the part.

  • -SEED(<min>:<max>) → Generate a new value like SEED(<min>:<max>) but this value will be subtracted from the current value of the part.

  • RAND(<min>:<max>) → Generate a new random value. The part in brackets is optional and can limit the range of the generated value. <min> → specifies the low limit for the generated value. <max> → specifies the high limit for the generated value. Both <min> and <max> are optional. If not specified the default value for the limit will be used. The default limits depend on the part of the timestamp.

  • +RAND(<min>:<max>) → Generate a new value like RAND(<min>:<max>) but this value will be added to the current value of the part.

  • -RAND(<min>:<max>) → Generate a new value like RAND(<min>:<max>) but this value will be subtracted from the current value of the part.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Year

The rule or value that will be used to build a new year value for the given time data. The possible rules are described in the methods descriptions.

year

STRING

-

-

Month

The rule or value that will be used to build a new month value for the given time data. The possible rules are described in the methods descriptions.

month

STRING

-

-

Day

The rule or value that will be used to build a new day value for the given time data. The possible rules are described in the methods descriptions.

day

STRING

-

-

Hour

The rule or value that will be used to build a new hour value for the given time data. The possible rules are described in the methods descriptions.

hour

STRING

-

-

Minute

The rule or value that will be used to build a new minute value for the given time data. The possible rules are described in the methods descriptions.

minute

STRING

-

-

Second

The rule or value that will be used to build a new second value for the given time data. The possible rules are described in the methods descriptions.

second

STRING

-

-

Fraction

The rule or value that will be used to build a new fraction value for the given time data. The possible rules are described in the methods descriptions.

fraction

STRING

-

-

Seed

Specify the seed which can be used to generate new values for the time data parts. If a column name is specified the value of this column will be used as seed.

seed

STRING

-

-

FormatPattern

When the input value is a string this pattern will be used to parse the value of this string into a timestamp and to format the output into the same format.

formatPattern

STRING

yyyy-MM-dd-HH.mm.ss

-

FractionPattern

When the input value is a string this pattern will be used to parse the value of this string into the fractions part of the timestamp and to format the output into the same format.

fractionPattern

STRING

.SSSSSSSSSSSSZ

-

IgnoreNullInputs

Specify if null value inputs should be ignored. If set to false new values are generated for null inputs.

ignoreNull

BOOLEAN

true

-

National bank identifier code

Description

Generates a national bank identifier code (also known as Bankleitzahl in Austria and Germany, and as Instituts-Identifikation or Bankclearing-Nummer in Switzerland) based on the specified seed value. The country parameter specifies the country/region of origin ('AT', 'CH', 'DE', 'NL'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the national bank identifier code.

country

STRING

-

-

Pattern

Description

Generates a sequence of letters and digits according to a pattern.The letter 'A' is replaced by a letter from A to Z.The digit '9' is replaced by a digit from 0 to 9.The letter 'N' is replaced by either a letter or a digit.The letter 'X' is replaced by a hexadecimal digit (0 to F).All other characters are preserved as they are.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Specifies the value that is used to calculate the hash.

seed

STRING

-

-

Pattern

The pattern string based upon which the resulting sequence of letters and digits will be calculated.

pattern

STRING

-

-

Postal Code

Description

Generates a postal code based on the specified seed value. The country parameter specifies the country/region of origin ('US', 'DE', 'FR', 'IT', 'NL', 'AT', 'BE', 'CA', 'CH', 'UK'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the postal code.

country

STRING

-

-

Restart sequence

Description

This modification method reads the highest value of the selected column of the table and adds 1 to this value. "Then the selected column of the target table restarts with this value. On this way it is avoided that "the current sequence number will ever be reached what would cause loss of data. "You have to select the primary key-column in the linked modification set for the correct usage.

Input type

COLUMN

RiskClasses

Description

Masks address data while preserving similar hazard zone or risk class classifications. This method requires the lookup table RiskByAddresses to be installed and populated with address and risk class data.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

City

Address component: City

city

STRING

-

-

Zip

Address component: ZipCode

zip

STRING

-

-

Street

Address component: Street

street

STRING

-

-

House number

Address component: House number

housenumber

STRING

-

-

GK

Flood Hazard Classes (1 -4)

gk

STRING

-

-

SGK

Heavy rain class (1 - 3)

sgk

STRING

-

-

Zone

Zone of stream (B), coastal (K), island (I)

zone

STRING

-

-

EMS

Seismic hazard estimation (1 - 12).

ems

STRING

-

-

SLZ

Snow load zones (1, 1a, 2, 2a, 3, 3a).

slz

STRING

-

-

LDV

Lightning density (0 - 4).

ldv

STRING

-

-

Set Value

Description

Sets the specified value to the column.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

New Value

The new value to be set.

newValue

STRING

-

-

State

Description

Generates an administrative country subdivision name based on the specified seed value. The country parameter specifies the country/region of origin ('US', 'DE', 'FR', 'IT', 'NL', 'AT', 'BE', 'CA', 'CH', 'UK'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the state name.

country

STRING

-

-

Street

Description

Generates a street name based on the specified seed value. The country parameter specifies the country/region of origin ('US', 'DE', 'FR', 'IT', 'NL', 'AT', 'BE', 'CA', 'CH', 'UK'). Only a single country value can be specified (e.g. 'DE'). This method requires the lookup tables to be installed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Column that is used to calculate the hash.

seed

STRING

-

-

Country

Specifies the origin of the street name.

country

STRING

-

-

Swedish id

Description

Generates a Swedish id based on the birthday parameter. The generated ID is returned as a string. If the value is a date it will be used as is. All other values will be hashed.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Birthday

Specify a value that is used to calculate the hash.

birthday

STRING

-

-

Delimiter

The delimiter character used to separate the groups of digits. May be empty.

delimiter

STRING

-

-

U.S. social security number

Description

Generates a U.S. Social Security Number (SSN) based on the specified seed value.The generated SSN is returned as a string. Leading zeroes are preserved.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Specify a value that is used to calculate the hash.

seed

STRING

-

-

Delimiter

The delimiter character used to separate the groups of digits. May be empty.

delimiter

STRING

-

-

UUID

Description

Generates a version 4 universally unique identifier (UUID) based on the specified seed value.

Input type

COLUMN

Parameters
Display name Description Variable name Data type Default value Possible values

Seed

Specifies the value that is used to calculate the hash.

seed

STRING

-

-

Task execution reports

Read the chapter task execution report for more information about task execution reports.

Classification term report

Description

The above example shows what data was copied during the execution of a task based on classification terms. This report shows the name and values of the copied classification terms.

Summary Report

Description

The above example shows how much data was extracted, applied, deleted, discarded, or modified when copying from source to target.

Classification Term Detail Report

Description

This report shows which data was copied during a task execution based on classification terms. In comparison to the classification term report two columns have been added in this report. Through this you can also see the name of the task template and the task that used the classification term. With these names the task type can be identified. Through the task type it can be determined whether the classification term value has been written, read or deleted.

So for example it can be determined that row level delete task A has deleted a classification term my term which has the value 5 and a row level processor task B has written a classification term your term which has the value 6.

Attribute generators

Read the chapter attribute generator for more information about attribute generators.

Basic attribute generators

Download all generators of this group: Download example file

UbsBooleanGenerator

Description

Generates true or false based on the configured probability.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

True Probability

Probability for true (0..1 or 0..100).

trueProbability

NUMBER

-

UbsCategoryGenerator

Description

Generates a value from a predefined value list with optional weights.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Values

Comma-separated list of possible values.

values

STRING

-

Weighting

Optional comma-separated list of positive numeric weights.

weighting

STRING

-

UbsDateGenerator

Description

Generates a date or timestamp between start and end date. The output format is configurable via the format parameter.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Start Date

Inclusive lower bound (ISO date, ISO date-time or epoch millis).

startDate

STRING

-

End Date

Inclusive upper bound (ISO date, ISO date-time or epoch millis).

endDate

STRING

-

Format

DateTimeFormatter output pattern, e.g. yyyy-MM-dd or yyyy-MM-dd’T’HH:mm:ss.

format

STRING

-

UbsNumberGenerator

Description

Generates a random integer value within the configured range. The lower bound is configured via minValue and the upper bound via maxValue.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Minimum Value

Inclusive lower bound of the generated random number.

minValue

NUMBER

-

Maximum Value

Inclusive upper bound of the generated random number.

maxValue

NUMBER

-

Seed

Optional seed for deterministic random number generation.

seed

NUMBER

-

UbsPhoneNumberGenerator

Description

Generates a synthetic phone number for a country and output format.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Country

Country code (e.g. DE, CH, AT, US).

country

STRING

-

Format

Output format: E164 or NATIONAL.

format

STRING

-

UbsTextGenerator

Description

Generates a random text value. The generated text can be configured via length, character set, prefix and suffix.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Length

Number of random characters to generate.

length

NUMBER

-

Character Set

Optional set of characters used for random text generation.

characterSet

STRING

-

Prefix

Optional prefix added before the generated text.

prefix

STRING

-

Suffix

Optional suffix added after the generated text.

suffix

STRING

-

Seed

Optional seed for deterministic text generation.

seed

NUMBER

-

UbsUuidGenerator

Description

Generates a unique UUID. Optional version currently supports v4.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Version

Optional UUID version. Supported value: 4.

version

NUMBER

-

Address attribute generators

Download all generators of this group: Download example file

UbsCityGenerator

Description

Generates a city name based on the specified seed value. The country parameter specifies the country/region of origin ('DE', 'CH', 'AT', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the city.

country

STRING

-

UbsLatitudeGenerator

Description

Generates the latitude of geographic coordinates based on the specified seed value. The country parameter specifies the country/region of origin ('DE', 'CH', 'AT', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the geographic coordinates.

country

STRING

-

UbsLongitudeGenerator

Description

Generates the longitude of geographic coordinates based on the specified seed value. The country parameter specifies the country/region of origin ('DE', 'CH', 'AT', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the geographic coordinates.

country

STRING

-

UbsPostalCodeGenerator

Description

Generates a postal code based on the specified seed value. The country parameter specifies the country/region of origin ('DE', 'CH', 'AT', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the postal code.

country

STRING

-

UbsStateGenerator

Description

Generates an administrative region (e.g. state/canton) based on the specified seed value. The country parameter specifies the country/region of origin ('DE', 'CH', 'AT', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the state/region.

country

STRING

-

UbsStreetGenerator

Description

Generates a street name based on the specified seed value. The country parameter specifies the country/region of origin ('DE', 'CH', 'AT', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the street name.

country

STRING

-

Banking attribute generators

Download all generators of this group: Download example file

UbsBankIdentifierCodeGenerator

Description

Generates a BIC/SWIFT code (ISO 9362) based on the specified seed value. The country parameter specifies the country/region of origin ('AT', 'CH', 'DE', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the BIC.

country

STRING

-

UbsBankNameGenerator

Description

Generates a bank name based on the specified seed value. The country parameter specifies the country/region of origin ('AT', 'CH', 'DE', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the bank name.

country

STRING

-

UbsCreditCardNumberGenerator

Description

Generates a credit card number while retaining a configurable number of leading digits. The retain parameter specifies how many leading digits of the seed to keep.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Retain

Number of leading digits to retain from the seed (0..15).

retain

NUMBER

-

UbsInternationalBankAccountNumberGenerator

Description

Generates an IBAN based on the specified seed value. The optional country parameter specifies the country/region of origin ('AT', 'CH', 'DE', 'NL'). If no country is provided, one supported country with available bank data is chosen automatically. Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Optional: specifies the country prefix of the IBAN. If omitted, a supported country is selected automatically.

country

STRING

-

UbsNationalBankIdentifierCodeGenerator

Description

Generates a national bank identifier code (AT/DE: Bankleitzahl, CH: Clearing number) based on the specified seed value. The country parameter specifies the country/region of origin ('AT', 'CH', 'DE', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Column that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the national bank code.

country

STRING

-

Identity attribute generators

Download all generators of this group: Download example file

UbsEmailGenerator

Description

Generates an email address based on a seed, country and sex. Optionally a custom domain can be specified. The country parameter specifies the country/region of origin ('EN', 'DE', 'FR', 'IT', 'NL'). The sex parameter specifies the gender ('M' for male, 'F' for female). Default domain is 'example.com'. This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Specifies the value that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the name parts.

country

STRING

-

Sex

Specifies the gender of the first name.

sex

STRING

-

Domain

Target domain of the generated email address (default: example.com).

domain

STRING

-

UbsFirstNameGenerator

Description

Generates a first name based on the specified seed value. The country parameter specifies the country/region of origin ('EN', 'DE', 'FR', 'IT', 'NL'). The sex parameter specifies the gender ('M' for male, 'F' for female). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Specifies the value that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the first name.

country

STRING

-

Sex

Specifies the gender of the first name.

sex

STRING

-

UbsLastNameGenerator

Description

Generates a last name based on the specified seed value. The country parameter specifies the country/region of origin ('EN', 'DE', 'FR', 'IT', 'NL'). Only a single country value can be specified (e.g. 'DE'). This generator requires the lookup tables to be installed.

Script Language

GROOVY

Parameters
Display name Description Variable name Data type Default value

Seed

Specifies the value that is used to calculate the hash.

seed

STRING

-

Country

Specifies the origin of the last name.

country

STRING

-

Task stage hooks

Read the chapter task stage hook for more information about task stage hooks. Information about the concept of a task stage hook can be found in the task stage hooks chapter of Working with XDM.

Delete expired tasks

Description

This task stage hook deletes tasks of the same task template that exceed the specified retention count or retention date. By default, the hook only deletes these tasks that have been implicitly created either by the execution of a data shop, or while executing a task template. Both actions implicitly create a new task in the background. To avoid that the list of created tasks grows and grows you can automatically delete tasks that have not been executed for some time. This hook uses an internal API. It is maintained by UBS Hainer

Code Type

GROOVY

Parameters
Display name Description Data type Default value

taskRetentionPeriod

Specifies a period of time after tasks are deleted.

STRING

-

taskRetentionCount

The number of tasks that should be kept. If more than the specified amount of tasks exists the oldest executed tasks will be deleted. A value of zero defines that no tasks will be deleted.

NUMBER

-

implicitOnly

Specifies that only implicit tasks should be dropped, or even manually created tasks. Implicit tasks are created by a data shop execution, or by the Jenkins plugin.

BOOLEAN

true

Delete expired workflows

Description

This task stage hook deletes workflows of the same workflow template that exceed the specified retention count or retention date. By default the hook only deletes these workflows that have been implicitly created either by the execution of a data shop, or while execution a workflow template. Both actions implicitly create a new workflow in the background. To avoid that the list of created workflows grows you can automatically delete workflows that have been executed for some time. This hook uses an internal API. It is maintained by UBS Hainer

Code Type

GROOVY

Parameters
Display name Description Data type Default value

workflowRetentionPeriod

Specifies a period of time after workflows are deleted.

STRING

-

workflowRetentionCount

The number of workflows that should be kept. If more than the specified amount of workflows exists the oldest executed workflows will be deleted. A value of zero defines that no workflows will be deleted.

NUMBER

-

implicitOnly

Specifies that only implicit workflows should be dropped, or even manually created workflows. Implicit workflows are created by a data shop, or by the Jenkins plugin.

BOOLEAN

true

Delete icebox generation

Description

This can be called in a From IceBox task and deletes the generation that has been restored by the task. The generation is deleted by calling the REST API of the core server. It includes the deletion of the entry in the user interface and the data of the generation This hook uses an internal API. It is maintained by UBS Hainer

Code Type

GROOVY

Extract icebox generation data

Description

This example can be used to extract all icebox generation parameters and copy it into the task’s runtime properties. The parameter names have the prefix ig_. These parameters can be used with ctx.getTaskRuntimeProperty(<propertyName>), for example in modification methods.

Code Type

GROOVY

Fill MTC for Risk Classes

Description

This task stage hook creates and populates a risk lookup table in a Mapping Table Container with address and risk class data. The data can be provided from a database, a CSV file, or a combination of both.

Code Type

GROOVY

Parameters
Display name Description Data type Default value

addressRiskConnection

XDM db Connection to address and risk class data

CONNECTION

-

addressRiskFile

XDM csv File to address and risk class data

FILE

-

Print start keys

Description

Row-level processing tasks use start keys to identify the database records that will be copied during a task run. These keys can be viewed after task execution export under files/startValues.csv. It is possible that some specified keys do not exist in the database. This hook logs the keys that are actually found in the database at the start of the task run. The results can be viewed in the task execution details for the task stage where the hook is used. This is useful, for example, to analyze why no data may have been copied, or to pass the resolved start keys to another task stage for further processing.

Code Type

Ant

Create Mapping Rule

Description

Sometimes, in RLP tasks, the properties of the source and target must be adjusted manually. Although Mapping Rules are not supported for RLP, this stage hook provides a way to define custom mappings, for example if column names differ. Such a mapping rule can be created using this stage hook. Any mapping described in the Mapping Rules chapter can be used, except those with objectType = "ANY" and fieldType = "SCHEMA", because the schema is managed by the installed application.

Code Type

GROOVY

Parameters
Display name Description Data type Default value

fieldSelectionMode

Controls the types of objects which can be selected by Object type and Field type

STRING

Simple

objectType

Specifies the type of database object that the rule matches

STRING

TABLE

fieldType

Specifies the attribute of a database object that the rule affects.

STRING

NAME

sourceSelectionPattern

Specifies the selection pattern for the source field to be mapped.

STRING

%

targetValue

Specifies how the target value is derived from the source value.

STRING

-

Build File Bridge Table from CSV

Description

The File Bridge server is an interface between CSV files and JDBC. It enables SQL access to data stored in files outside of a relational database system. The column names and data types of the CSV file are defined using CREATE statements for the File Bridge server. The server interprets these definitions as tables and makes them accessible via JDBC. This stage hook generates the required CREATE statements from a CSV input file. To do this, the column names and data types must be specified in the input file. The first line of the input file contains the column names, and the second line contains the data type of each column. Supported data types are:

  • CHAR

  • VARCHAR

  • INTEGER

  • SMALLINT

  • BIGINT

  • FLOAT

  • DOUBLE

  • BINARY

  • DECIMAL

  • DATE

  • TIME

  • TIMESTAMP

If the data type is CHAR, VARCHAR, or DECIMAL, the length must be specified in parentheses immediately after the data type. All remaining lines of the input file contain the actual data.

The stage hook reads the column names and data types from the input file in order to build and execute the CREATE statements for the file bridge server, thereby defining the table space and table for the file. It then removes the data type row from the input file and writes all remaining lines to a new output file. This output file is used as the data source by the file bridge server.

For this to work, the output folder must be accessible from the dataflow server on which the hook runs. The hook can process multiple input files: it processes all CSV files located in the input folder as well as all CSV files contained in any subfolders of that folder.

The table name in the file bridge server is derived from the file name in the input folder. To avoid duplicate names, the table name for files in subfolders is composed of the subfolder name and the file name, joined with an underscore (_).

Example csv file:

ID;FIRSTNAME;LASTNAME;BIRTHDATE
INTEGER;VARCHAR;VARCHAR;DATE
123;"John";"Doe";01-01-1985
456;"Jane";"Doe";02-02-1988
789;"Max";"Mustermann";03-03-1990
Code Type

GROOVY

Parameters
Display name Description Data type Default value

outputFolderName

Defines the internal folder mounted in the dataflow container where the hook writes the output file. The corresponding host directory must be specified in the installation YAML file.

STRING

-

header

Specifies whether column headers are included in the output file.

BOOLEAN

false

quoteStrings

Indicates that values in string columns are enclosed by the character specified as charDel.

STRING

QUOTE STRINGS

charDel

The character used as a delimiter for CHAR and VARCHAR data types.

STRING

"

colDel

The column delimiter used to split columns in the input CSV file.

STRING

;

inputFolderName

Defines the internal folder mounted in the dataflow container where the input file is stored. The corresponding host directory must be specified in the installation YAML file.

STRING

-

jdbcServerConnection

Defines the File Bridge connection in XDM used to access the File Bridge server.

CONNECTION

-

schemaName

The name of the schema in which the table is created.

STRING

-

encoding

The character encoding of the CSV files.

STRING

UTF-8

Workflows

Read the chapter workflow for more information about workflows. Information about the concept of a task stage hook can be found in the workflows chapter of Working with XDM.

Copy with icebox and delete target

Description

This example workflow calls a to icebox task to save an environment. Then it starts a from icebox task to select the icebox generation to restore. The selection criteria from this generation is given to a delete task to clear the target environment. In the next step a from icebox task is used to clear remaining data in the target environment before the load from the first from icebox task restores the data to the target environment.

Delete object container cache directory

Description

This executable workflow executes a task stage hook which deletes the entire object container cache stored in the database. The cache is stored in the table batch_object_cache within the schema defined by ${taskStageHook.xdmSchema}.

Using this workflow may lead to inconsistencies if tasks using the object container retention period are currently running and possibly writing to the object container table while another task deletes entries from this table using this hook. Please be aware of such scenarios, especially when scheduling this workflow.

IceBox to cloud migrator

Description

This workflow migrates existing IceBox generations to an AWS S3 cloud storage.

The workflow requires that the following configuration in the XDM properties have been configured:

  • admin.db.url

  • admin.db.user

  • admin.db.password

  • admin.db.schema

  • xdm.blob.store.s3.bucket

  • xdm.blob.store.s3.credentials.accessKey

  • xdm.blob.store.s3.credentials.secretKey

  • xdm.blob.store.s3.region.static

  • xdm.blob.store.s3.endpoint

Mapping table container and H2 database migrators

Description

The workflow Migrate Mapping Table Container migrates existing Mapping Table Container to use an actual version of the H2 database.

The workflow just iterates over the Mapping Table Container list and calls for each container the migration hook. The hook will download the database from the Mapping Table Container, migrate the database and will upload it again. Please make sure that the executing user has the appropriate permissions to read and write the mapping table container.

Furthermore, there are two more workflows Migrate H2 files in /xdm/mapping and Migrate H2 databases in /xdm/data. These will migrate H2 databases found in the directories /xdm/mapping and /xdm/data. When the databases were created with a user-defined user and password, the user and password must be changed in the workflow.

Find invalid null values in Condition Scripts and Lookup Scripts

Description

This workflow locates any usage of the now deprecated form of null values ("<null-value>") in Lookup Scripts and Base Table Conditional Scripts. CAUTION: This workflow needs to be executed with a user that has the permission to read all Lookup Scripts and Base Table Conditional Scripts in the system. Otherwise, the workflow will not find all invalid null values.