Modification method

A modification method contains the script and its parameters used in data modification. The script defines how a single value or entire row of table data should be modified.

Concepts and more information

Modification Examples

There are scripts for typical use cases and best practices in the modification method examples.

Find ready to use examples for modification methods in the modification methods section of useful examples. These methods can be downloaded and directly imported into an XDM installation.

Permissions

Modification Methods have specific permissions to manage user access. The table below displays the available permissions and their purposes.

For more details about the concept of XDMs permission management refer to Permission Management.

Permission

Description

ADMINISTRATION

Specifies that the grantee can grant and revoke permissions to and from other users.

A user that creates an object automatically receives the ADMINISTRATION permission on that object.

DELETE

Specifies that the grantee can delete objects of the selected types.

READ

Specifies that the grantee has read permission on the object. The grantee is able to see the object in lists and can see all of the object’s details, such as rules or access permissions.

In addition, the grantee can reference this object. For example, a user who has READ permission on a credential object can refer to this credential object when creating a new database connection.

WRITE

Specifies that the grantee has the permission to change the settings and attributes of an object. This also includes modifying any rule lists that might be associated with the object (for example, the selection rules of a task template).

Properties

The table below documents the available properties for modification methods. The 'name' column displays the property name as it can be used in Groovy and Java Scripts.

Name

Type

Default

Description

Code

code

String

/* * * * The following variables are available in this script: * ===================================================== * ctx : A context object that offers a number…​

Modification methods need to use the functions init(), apply(), and close().

The init() function is executed once prior to the start of modification, when the modification context is created.

The actual modification code should be placed in the apply() function. It is executed each time the modification method is used.

The close() function is executed once after modification is complete, immediately before the modification context is closed.

You can, of course, add your own functions. These must then be called in the appropriate place in init(), apply(), or close(), depending on your requirements.

The detailed documentation of the modification method code can be found in the here.

More examples of different usages with modification methods can be found in the Modification Method examples.

Description

description

String

n/a

An optional description for this object. The description can contain multiple lines to give more context on the configured object. The description is not used in a technical context.

Name

displayName

String

n/a

Specifies the name of the object. The name is used to display and identify the object in lists. The name can contain any valid UTF-8 characters. This field is mandatory.

Input type

inputType

InputType

COLUMN

Specifies the type of modification that is provided when the modification method is called. This field has one of the following two possible values:

Column

This means the modification method modifies the same columns in each row. Only the corresponding fields in the data array will be stored in the target table. The columns on which the method is invoked is specified by a pattern in the modification rule which uses the modification method.

Row

This means the modification method modifies more than one column in each row. In this case the modification method can modify all data fields of the input array. The entire data array will be stored in the target table.

Legacy script

legacyScript

Boolean

false

Determines whether the method code should use the old legacy evaluation internally or the new one, where the method script references can be used.

Libraries

libraries

File

n/a

A list of files that can be accessed in the modification method code. When editing the modification method, all XDM files of the specified script language are made available for selection. The content of the selected XDM files is automatically added to the script context. All methods, variables, and classes in the selected files can be accessed in the method code.

Mapping table access mode

mappingTableAccessMode

AccessMode

READ

Specifies the mode in which the modification methods accesses the mapping table container.

READ

The modification method access the mapping table container in read mode. The method will only read data from the container. An exception is thrown, if the method tries to update the container.

READ_WRITE

The method will open the container for read/write access. The method can update the data of the mapping table container.

Mapping table containers

mappingTableContainers

MappingTableContainer

n/a

A list of mapping table containers that can be used in the code of the modification method. The mapping table container can be accessed by the variable name that is specified in the mapping table container declaration. The injected mapping table container is an established JDBC connection to drop and create new tables and to modify their contents.

Script language

scriptLanguage

Enum

GROOVY

This script language specifies in which language all scripts of the object are written. Currently the following languages are supported:

Groovy (GROOVY)

specifies that the code is written in Groovy.

JavaScript (JS)

specifies that the code is a JavaScript.

Tags

tags

Tag

n/a

Contains the tags that apply to this object. These tags can be used in the search to find objects quickly and effortlessly.

Embedded parameters: parameters

Modification parameters used in modification methods are variables in the Groovy or JavaScript code. Parameter values are supplied by the modification rule which uses the method.

Name

Type

Default

Description

dataType

String

n/a

Specifies the type of the parameter.

An example for using the connection data type can be found in the script examples.

The available values are:

Boolean

A boolean value that can have the values true or false.

Connection

A XDM connection.

Date

A date or timestamp.

Number

A numeric parameter.

The type Number can only be used for integer numbers. For floating-point numbers use the type String instead. When writing floating-point numbers as strings to a decimal column of a database table, XDM ensures that the data is written correctly.
String

A string parameter.

defaultValue

String

n/a

Allows the specification of a default value for the parameter.

description

String

n/a

An optional description for this parameter. The description can contain multiple lines to give more context on the configured object. The description is not used in a technical context.

displayName

String

n/a

This is the name that is used to identify the parameter when it is referenced in a modification rule. These names are never used in a technical context, so blanks and symbols can be used. This field is mandatory.

possibleValues

List

n/a

Specifies a list of possible values for the parameter. The values are entered as pairs of strings. The first string is the actual display name. The second string is a value, which is used when the value is selected in a modification rule.

variableName

String

n/a

This is the name that is used in the code. This field is mandatory.

Actions

The available actions are described below. Some actions apply to the list, while others are specific to selected modification methods.

List Actions

The following actions are available on the modification methods list. If the action is disabled a tooltip will provide the exact reason for the deactivation. The required permissions are described in detail for each action.

  • Bulk Create Permission

  • Bulk Delete

  • Bulk Export

  • Create

  • List History

Create a new permission on the selected objects. Shows in the result list whether the permission could be granted on the respective object. Only these permissions can be granted that are existing on the underlying object.

A permission in the result list can have three different states, these are:

CREATED

The permission successfully granted on the object.

MERGED

The granted permission already exists on the object and merged with the new permission.

SKIPPED

The permission could not be granted, because of missing administration permission on the object.

The following permissions are required on the list:

  • ADMINISTRATION

  • READ

Delete the selected objects.

The following options are available:

Cascade

Recursively delete depending objects.

When using cascade, dependent objects are deleted first also with cascade enabled. Thus, a cascade deletion is a recursive function that deeply searches for dependent objects and deletes them first. There is only a confirmation for the first object. The dependent objects are deleted without confirmation but only when the user has the DELETE permission.

This feature is only available in development mode. More information about development mode can be found in the chapter User Settings. It should be used with caution.

An object in the result list can have two different states, these are:

DELETED

The object could be deleted.

NOT_DELETED

The object could be not deleted. This may be because the executing person does not have a delete permission on the object or the object is still referenced by others. A detailed reason can be determined with the help of the error message. If the object is still in use, these objects are also displayed.

The following permissions are required on the list:

  • DELETE

  • READ

Exports the selected objects.

YAML

Generates a YAML file containing all the object’s settings. The user has the option to download the export file, or to paste the content in the import dialog. The YAML export is particularly suitable for importing the exported objects again via the XDM UI.

ZIP

This export writes several individual YAML-files. Each YAML-file is stored in a directory according to its type. For example, when exporting a native table backup task template named 'A backup template', a YAML-file 'A backup template.yaml' is created inside the directory /TaskTemplate/native-table-backup-task-template/ of the ZIP-file. This kind of export is suitable for usage in git-repositories together with XDM’s configuration as code feature.

Related and dependent objects can optionally be included in the export. The export dialog has the following options:

Include dependent objects

Dependent objects only belong to the exported object like rules and tasks.

Include permissions

Permissions of each exported object, only when the object supports permissions. Some objects like rules don’t have permissions.

Include referenced objects

Referenced objects exist by their own and are used in the exported object like connections and environments.

Include objects that depend on referenced objects

Also include the dependent objects of the referenced objects. E.g. the rules of a modification set or the rules in an application model version.

Objects on which the user does not have READ permission are not exported. This includes dependent and referenced objects. However, the reference to an object will be exported. For example a connection object would refer to the credential, even if the user does not have READ permission on the credential. The definition of the credential object itself will not be part of the export file. This can lead to issues during the import, because the connection cannot be created without an existing credential.

The following permissions are required on the list:

  • READ

Creates a new object in the current list. Depending on the object type either a popup dialog is shown for the most important settings, or the complete object is shown in edit mode. The dialog provides the option to create the object and remain in the current list or to switch to the newly created object in edit mode to perform further changes.

The following permissions are required on the list:

  • CREATE

The history list tracks all modifications made to objects within it. A new record is added each time an object is created, edited, or deleted. A record indicates who made the change, which object was affected, and when the change was made.

For more information about the concept of the history refer to the history concepts.

The following permissions are required on the list:

  • READ

Object Actions

The following actions are available on specific modification methods. In order to execute the action, the user must possess the necessary permissions for the object. The permissions required for each action are described individually. If the user does not have these permissions, the action will be disabled and the tooltip will provide the exact reason for the deactivation.

  • Check

  • Delete

  • Duplicate

  • Edit

  • Event List

  • Export

  • Migrate code

  • Object History

  • Permission Check

  • Usage

  • Uses

This action validates the object and its dependencies, reporting configuration errors that could cause issues during task or workflow execution. The validation will cascade through the child objects of the checked objects and objects referenced by them.

For instance, if an installed application of an environment is checked, the check will process the application model, the specified version, the connection, modification sets, and involved modification methods. If an object has rules, all active rules will be checked. The modeling connection and version, including their modification sets and methods, will also be checked. Deactivated objects will not be included in recursive checks, but can be checked individually if the check is executed on the object itself.

Checks often require additional information from the context of the objects being checked, such as necessary connections or custom parameter values. The check will gather information from the objects being checked and use it to perform checks on child objects. Any required additional information must be provided before the check begins. The check queries the user to provide these missing information.

Database object checks

For all rules which reference database objects such as tables, columns, etc, the check verifies that the those objects exist in the database system. If a connection can be inferred from the context, then this connection is used. If no connection is available in the context, it must be specified before the check is executed.

Connection checks

For objects which configure access to external systems, such as connections or storage locations, the configuration check verifies that access can be established using the given credentials. Furthermore, additional operations on database connections are performed to check whether the credential user has the necessary authorization to access relevant database objects. In particular, the credential user’s permission to read source tables and write to target tables is verified. Similarly, for storage locations the check verifies that the credential user has permission to write to the working directory.

Code checks

For all entities containing code segments, such as modification methods or condition scripts, the syntax for the code is checked. This does not check, however, whether at run time all necessary variables are likely to be available.

The following permissions are required:

  • READ

Delete the object. If the object is still used by another entity, an error message is displayed, and the object is not deleted. The delete operation must be confirmed in a separate popup.

The following options are available:

Cascade

Recursively delete depending objects.

When using cascade, dependent objects are deleted first also with cascade enabled. Thus, a cascade deletion is a recursive function that deeply searches for dependent objects and deletes them first. There is only a confirmation for the first object. The dependent objects are deleted without confirmation but only when the user has the DELETE permission.

This feature is only available in development mode. More information about development mode can be found in the chapter User Settings. It should be used with caution.

The following permissions are required:

  • DELETE

  • READ

Will create an exact copy of the current object with a different display name in the same list. Users can decide whether they want to copy child objects like rules, permissions or tasks. It is only possible to select complete classes of objects and not to select individual child objects. Copied child-objects will preserve their display name. The default is to copy all child objects.

The following permissions are required:

  • CREATE

  • READ

Opens the current entity in edit mode.

The following permissions are required:

  • READ

  • WRITE

This list shows all registered events for the object. It includes events that are specific to the object, or for that type.

The following permissions are required:

  • READ

This action allows to export XDM objects in different formats in order to import them via export or CasC in another environment.

Refer to configuration of export for more information.

Related and dependent objects can optionally be included in the export. The export dialog has the following options:

Include dependent objects

Dependent objects only belong to the exported object like rules and tasks.

Include permissions

Permissions of each exported object, only when the object supports permissions. Some objects like rules don’t have permissions.

Include referenced objects

Referenced objects exist by their own and are used in the exported object like connections and environments.

Include objects that depend on referenced objects

Also include the dependent objects of the referenced objects. E.g. the rules of a modification set or the rules in an application model version.

Include implicit created objects

Implicit created objects are tasks or workflows which were automatically created for execution. These objects won’t be exported by default, but can be included by setting this flag. When exporting implicit objects, make sure that the Include dependent objects flag is also enabled.

Objects on which the user does not have READ permission are not exported. This includes dependent and referenced objects. However, the reference to an object will be exported.

For example a connection object would refer to the credential, even if the user does not have READ permission on the credential. The definition of the credential object itself will not be part of the export file. This can lead to issues during the import, because the connection cannot be created without an existing credential.

The following permissions are required:

  • READ

This action migrates the legacy script modification method code to the new standard non-legacy code and deactivates the legacy mode.

If you use the JavaScript 'includes()' function on arrays or lists in the method, you need to change this manually to 'indexOf() != -1'. As 'includes' is no longer available.

If you also use loose typing, please note that this may no longer be possible in the new function type. For example, it is no longer possible to overwrite the type of declared variables.

After using this action, it is recommended to look over the migrated code again to check whether everything has been migrated correctly.

Please pay particular attention to checking the libraries used. As these are not taken into account during migration.

The following permissions are required:

  • READ

  • WRITE

The history displays all changes made to the respective XDM object, including any changes made to its rules.

Each change record includes information about the operation performed (e.g. CREATE, UPDATE, DELETE), the timestamp, and the user responsible for the change.

For more information about the concept of the history refer to the history concepts.

The following permissions are required:

  • READ

The check verifies that the current user has the authorization to access the object. The check can also be performed for a specific user or role, if needed. By default, the check is performed using the current user’s credentials. It is then applied to child and referenced objects.

Additional permission checks are applied when these can be inferred from the context in which the check was started. For example, if the check is performed on a table copy task, the referenced source and target connections are checked to determine whether the given identity has source or target usage permission respectively.

The following permissions are required:

  • READ

The Usage List shows all objects that refer to the current object. It provides an overview of the relationships and makes it easy to track these relationships.

The following permissions are required:

  • READ

The Uses List shows all objects that the current object uses. It provides an overview of the relationships and makes it easy to track these relationships.

The following permissions are required:

  • READ

Appendix

Performance of legacy vs. non-legacy methods

Modification methods implemented in JavaScript exhibit significantly lower performance compared to legacy-based methods.

This is due to the underlying JavaScript engine that interprets the code. Therefore, we strongly recommend switching to modern, non-legacy methods based on Groovy. Performance analyses have shown that Groovy methods are not only faster than JavaScript methods but also outperform the previous legacy methods.

Available API in modification methods

The XDM ModificationContext API

The ctx object contains contextual information about the environment in which the Groovy or JavaScript code is running. The ctx object is always of type ModificationContext.

A complete list of possible methods for the ctx object is documented under ModificationContext API docs.

The XDM Database Model API

The ModificationContext API returns Column and Table objects. These are documented in the Database Model API docs.

The XDM MappingTableUtils API

This is recommended if XDM is running in Kubernetes. The required data is read from a database, not from the file system. A complete list of possible methods is documented under MappingTableUtils API docs.

The XDM ModificationUtils API

This is the current standard when using H2 tables. A complete list of possible methods is documented under ModificationUtils API docs.

The XDM HashUtils API

By default, you can use the HashUtils in your modification methods. These provide hashing functions that can be used for masking algorithms. For more details click here.

The data object

The data object contains the contents of the columns of a row. You can both read the contents of the columns and modify it.

The data type of the data element depends on the data type of the column that the element represents:

  • If the column type is numeric (for example, integer values or floating-point values), the data type of the data element is the Java data type java.math.BigDecimal.

  • If the column type is a string type (for example, fixed length strings or varying length strings), the data type of the data element is the Java data type java.lang.String.

  • If the column type is a date or time type (for example, date values or timestamp values), the data type of the data element is HighPrecisionTimestamp. This data type is derived from java.sql.Timestamp and is able to handle an arbitrary precision.

A detailed overview of how data types are being mapped for each supported database system in XDM can be found in the corresponding chapter in the object reference for data type mappings.

It is not possible to use values that exceed the value range of the database data type that is represented by the element of data. For example, a column that is able to store 16-bit signed integer values typically has a value range of -32,768 to +32,767. The data array will accept any BigDecimal number for this column, but when the XDM task is running, the database will return an error message if you try to use a value that is outside the above value range.

The elements of the data array are numbered starting with 0. That is, the first column of the table can be accessed using data[0], the second column of the table can be accessed using data[1], and so on. In order to determine the position of a column inside the data array, use the indexOf method of the ctx object, which is available in every modification method.

The columnName and columnIndex variables

Modification methods can be applied to a table, or to a single column of a table. If a modification method is a column-based method, the Groovy/JavaScript code has access to two additional variables:

  • columnName contains the name of the table column to which the method was applied.If the modification method is applied through a modification rule that has a wild card character in its column pattern, the variable columnName contains the actual name of the column without any wild card characters.

  • columnIndex contains the position of the column to which the method was applied inside the table. You can use the variable columnIndex as an index to the data array. This allows you to access the contents of the column for the row that is being processed.

If the modification method is a row-based method, these two variables are not available to the Groovy/JavaScript code.