XDM User Manual – Overview

This introductory section outlines the purpose and scope of the XDM platform. It presents an overview of the system’s capabilities, its modular architecture, and key concepts such as environments, models, and task orchestration. The section serves as a foundation for understanding how different parts of the platform interact and support data integration workflows.

The accessibility section details how the XDM platform addresses usability for all users, including those with impairments. It describes compliance with accessibility standards, available keyboard navigation support, and screen reader compatibility. These features ensure the platform is operable within a wide range of accessibility contexts.

This section provides a chronological record of platform changes across versions. It documents newly introduced features, improvements to existing functionality, deprecated components, and resolved issues. The changelog helps users track the evolution of the platform and assess compatibility with their environments.

The Setup section provides comprehensive guidance for installing and configuring the XDM platform. It begins with planning steps for a proof of concept, which help define the project scope and assess suitability. The requirements subsection outlines all technical prerequisites, including supported platforms and database engines. Information on licensing clarifies the different packages and how they apply in practice.

Installation procedures are described in detail, covering standard environments as well as Kubernetes deployments. Post-installation steps ensure system functionality is verified. Configuration topics include setting environment variables, defining custom properties, and enabling logging and security parameters.

Updating the platform is addressed through a structured update process, and specific instructions are provided for upgrading the PostgreSQL database backend. Backup and restore mechanisms are outlined for operational safety. Finally, this section covers ongoing maintenance tasks and how to integrate XDM into Jenkins-based CI/CD pipelines.

The Tutorial section provides step-by-step guidance for using key features of XDM through practical scenarios. It starts with a description of the sample environment, which serves as the basis for all tutorials and includes predefined models and tasks.

Subsequent tutorials cover infrastructure objects such as systems and containers, followed by exercises for setting up native table copy tasks. Modification methods and sets are explained through examples of how to apply logic to data during processing. The application model structure is detailed alongside the concept of environments.

Row-level processing tasks demonstrate how logic can be applied individually to each data row, including conditional flows and branching. The section concludes with custom parameters, which allow runtime configuration of tasks and environments.

This chapter focuses on practical scenarios and core operational concepts of working with the XDM platform. It provides in-depth explanations and examples that support day-to-day activities and help users effectively manage and execute data integration tasks.

The section introduces key administrative concepts, such as the role of environments, models, and system containers, and explains how these building blocks interact. Common concepts, including task composition, script binding, and parameter usage, are presented to establish a shared understanding of platform behavior.

A subsection on comparing structures describes how different data structures can be analyzed and validated across environments, ensuring schema consistency. The process of copying full tables is documented with techniques to optimize performance and manage metadata during bulk transfers.

Data generation tools within XDM are explained, highlighting their use in test environments and simulation scenarios. The section also covers how subsets of data can be isolated and processed using filters and parameterized execution.

Advanced data manipulation techniques are detailed in the modification section, which explains the application of transformation logic through scripting. The concept of hooks is introduced in this context to show how scripts can be injected at specific stages of a task or workflow.

The chapter describes how workflows and workflow templates are defined and executed. It includes how hooks and modification methods are combined to orchestrate multi-step processing. Integration with the Data Shop is explained, where users can trigger and schedule tasks, optionally using technical users and time-based controls.

Practical tooling such as the Icebox is introduced to support temporary data storage or manual inspection tasks. Additionally, this section documents how XDM clients interact with the platform, providing details on authentication, task triggering, and configuration of endpoints.

Overall, this chapter provides operational clarity for users who regularly define, run, and maintain data processing workflows within the XDM environment.

The Object Reference chapter provides a structured description of all configuration objects available within the XDM platform. Each object type is documented in a dedicated section.

For every object, the documentation typically outlines the object’s purpose, structural components, key attributes, and configurable parameters. It also explains how the object interacts with other components in the system and under which conditions it is used in processing workflows or administrative contexts.

Using a consistent format, the object reference enables technical users to understand the definition, usage, and lifecycle of each object type. This includes examples, field constraints, and system behaviors triggered by specific configurations. Whether modeling data, defining tasks, or integrating systems, the object reference serves as a foundational lookup for all reusable XDM entities.

The Script Reference is a technical resource for defining and executing custom logic within the XDM framework. It begins with an introduction to supported scripting languages (primarily Groovy and JavaScript), execution contexts, and platform-specific conventions.

An API reference documents the available functions and their parameters, grouped by usage context. This includes database interaction, logging, task manipulation, and system utilities. Practical script examples illustrate common patterns such as calling REST APIs, executing SQL, and integrating with Java libraries.

The reference also includes examples of modification methods, use of persistent memory, and working with data from mapping tables or local files. A dedicated section describes task stage hooks, where scripts can be bound to lifecycle events such as data loading, transformation, or post-processing.

This chapter consolidates implementation-oriented documentation and practical configuration examples for working with XDM in more specialized or advanced scenarios. It provides reusable guides, technical how-tos, and configuration snippets that support customization and automation efforts.

The section begins with curated guides that cover topics such as calling host jobs from task stage hooks and dynamically managing environments during runtime. Other guides explain how to manipulate JSON data stored in JSONB columns, set identity values, or move reusable logic into script libraries to support cleaner and more maintainable solutions.

Additional guides explore the use of files in task start conditions, setting dummy files for transformation steps, and interacting with the Structure Compare Task framework to compare environment states. Examples include writing binary (BLOB) data into XDM files and scheduling logic for technical users.

The resource area provides detailed examples for common transformation patterns, such as formatting strings, handling geo-coordinates, validating social security numbers, and generating UUIDs. There are also examples related to modification methods, postal data formats, and country-specific identifiers.

Supplementary scripting and integration examples cover sending emails, reading CSV files into the file bridge, executing REST APIs, and processing content from icebox generations. The documentation also addresses deletion strategies for expired workflows, cleanup tasks, and execution reporting templates.

Together, these guides and references serve as a practical toolbox, allowing developers and administrators to extend the platform in consistent and reusable ways. They also offer templates for common operational requirements and contribute to a more automated and robust deployment strategy.

This chapter focuses on the integration of external data sources—such as CSV files or VSAM datasets—into the XDM platform using the File Bridge server. It outlines installation steps, SQL usage patterns, and configuration of virtual tables.

The section begins with an overview of how the File Bridge Server operates and how it enables SQL-like access to structured flat files. General usage scenarios are described, including reading and modifying CSV content, handling encoding formats, processing binary content, and accessing VSAM datasets. Special attention is given to data typing nuances such as NULL handling, boolean flags, and timestamp formatting.

The installation guide describes how to deploy the File Bridge Server across platforms including Linux/UNIX, Windows, and IBM z/OS. It includes instructions for both Docker-based and manual setups, as well as details on startup procedures, system prerequisites, and connection validation.

SQL support in the File Bridge is documented in depth, including statements like CREATE TABLE, SELECT, UPDATE, DELETE, and GRANT. Each command is described with required authorizations, syntax rules, and limitations. The section also lists built-in functions for string manipulation, bitwise logic, date formatting, and environment access.

Finally, the chapter explains how to create virtual tables within XDM based on external definitions. It demonstrates how COBOL copybooks can be converted to SQL schemas and how table definitions for CSV files can be dynamically injected into the File Bridge using stage hooks within the XDM workflow engine.

Together, these components allow users to seamlessly incorporate flat file systems into the broader data processing architecture, using familiar SQL constructs and integrating with other tasks and environments in XDM.