Requirements for an XDM Installation

Installation Overview

XDM can be installed using two approaches:

  • Docker Compose / Podman Compose on a dedicated Linux server or virtual machine.

  • HELM Chart on a Kubernetes or OpenShift environment. (needs Cloud Support License)

Both installation methods provide an orchestrated container landscape containing several services, jointly forming the complete system.

Component Overview

The following containers/pods are typically included in an XDM installation:

  • web-ui: Provides the user interface accessible from any modern web browser.

  • core-server: Central service for configuration, user management, and orchestration.

  • dataflow-server: Coordinates task execution and data pipelines.

  • xdm3-runner: (only in cloud installations on Kubernetes/OpenShift) Executes data processing tasks.

  • xdm-db: PostgreSQL instance for admin and status data.

  • neo4j: Stores configuration and modeling data.

  • elasticsearch, grafana, tempo, loki, prometheus: Services for monitoring, logging, and metrics.

  • Optional: generator-source, file-sink, graph-store, and others depending on feature requirements.

Access to XDM is always via the web-ui container/pod in the browser. All backend connections to application databases are handled by internal services using the respective connectors.

--- config: theme: neutral --- flowchart LR %% User on the left subgraph users["Users"] User["User
(Web Browser)"] Automation["Automation
(REST Client)"] end %% XDM system in the center (Container Runtime and pods/containers) subgraph Center["XDM System
(on Container Runtime: Docker/Podman/Kubernetes/OpenShift)"] webui["web-ui
(User Interface)"] core["core-server"] dataflow["dataflow-server"] runner["xdm3-runner
(cloud only,
one per execution)"] xdmdb["xdm-db
(internal PostgreSQL)"] neo4j["neo4j"] others["others..."] end %% Application databases on the right AppDBs[("Application Databases
Db2 z/OS
Db2 LUW
Db2 for iSeries
Oracle DB
MS SQL Server
PostgreSQL
Snowflake
BigQuery
IBM IMS/DB
...")] %% Support systems at the bottom subgraph Support["Supporting Systems"] direction LR admindb["PostgreSQL AdminDB (optional)"] git["Git CasC Repository (optional)"] ldap["LDAP Service (optional)"] oauth["OAuth/OpenID Service (optional)"] end %% Connections User -- "HTTPS/Browser" --> webui Automation -- "REST/API" --> webui webui -- "REST/API" --> core core --> dataflow dataflow --> runner core --> xdmdb core --> neo4j core --> others %% Support systems connect to core services core -.-> admindb core -.-> git core -.-> ldap core -.-> oauth %% Application databases interaction (data extraction/application) dataflow -- "JDBC/SQL/Utilities" --> AppDBs runner -- "JDBC/SQL/Utilities" --> AppDBs

Configuration Artifacts

  • For Docker/Podman installations, environment variables are set for each service individually in the docker-compose.yml file.

  • For HELM-based installations, all system configurations are defined centrally in a values.yml file.

Communication Flow Overview

  • web-ui → core-server: User interaction, control, and status queries.

  • core-server → Database connectors (e.g., PostgreSQL, Neo4j): Access to application databases. (internal)

  • core-server → dataflow-server → application-databases : Task and pipeline execution initiation and management.

Hardware Requirements Docker/Podman installation

The following minimum requirements apply for an XDM installation on a dedicated server or VM:

  • Docker server/VM:

    • 16 GB RAM

    • 10 GB disk space for containers

    • 80 GB disk space for task files (may contain temporary data, dependent on amount of processed data)

    • 6 CPU cores

  • For larger environments (VM for XDM services):

    • 40 GB RAM

    • 10 GB disk space for containers

    • 1 TB disk space for task files (may contain temporary data, dependent on amount of processed data)

    • 16 CPU cores

The number of required CPU cores depends on the number of tasks that are executed in parallel. It is recommended to have at least 6 CPU cores. As a basic rule, XDM requires one CPU core for the XDM core module (the XDM core module manages that tasks and settings and provides the user interface) and one CPU core for each task that is executed in parallel.

Hardware Requirements HELM Chart installation

The requirements on memory and CPU are controlled per Pod. The default setup is contained in the HELM chart. Refer to the following chapter to get an estimated on how much memory is required per service. Each task is executed in a separate Pod, therefore a calculation of required CPU based on the expected number of parallel tasks is not necessary.

Recommended Memory per Service

Service Minimum RAM Recommended RAM

dataflow-server

3GB

16GB

core-server

2GB

8GB

web-ui

512MB

512MB

neo4j

3GB

6GB

xdm-db

512MB

1GB

elasticsearch

2GB

4GB

grafana

512MB

1GB

tempo

512MB

1GB

loki

512MB

1GB

prometheus

512MB

1GB

generator-source

2GB

4GB

file-sink

1GB

4GB

graph-store

1.5GB

4GB

The actual memory requirements for task execution depend on the task configuration and data volumes. These recommendations can be used as limits for the respective Pods.

Software Requirements

  • Supported operating systems for Docker/Podman installations:

    • Red Hat Enterprise Linux 7.x-9.x, CentOS 7.x-9.x, Debian 10

    • Docker CE or Docker EE version ≥ 20

    • Docker Compose ≥ 1.23 or Podman ≥ 4.2 plus Podman Compose ≥ 3.4

  • Kubernetes: Current distribution (e.g., Vanilla 1.30+, OpenShift 4.12+).

  • Modern web browser: Chrome version 72 or higher, Firefox version 70 or higher (JavaScript must be enabled).

Network Requirements

Overview

XDM provides a web-based user interface. To access this interface, the following general network requirements must be met:

  • The web UI must be reachable from all required clients (e.g. users' web browsers, integration services, automation tools).

  • Access to the web interface should be encrypted using SSL/TLS, especially in production environments.

  • Access to the web interface should be restricted to trusted networks or authenticated users (e.g. via firewalls, reverse proxies, or identity-aware access control).

Deployment-Specific Access

Docker Installation

For Docker-based installations, the Web UI container listens on TCP port 4280 by default.

  • By default, port 4280 is published on the host system.

  • The host port mapping can be customized via the Docker run configuration or your container orchestration tooling.

  • Ensure that:

    • the configured host port is open in the host server’s firewall, and

    • any upstream firewalls (e.g. corporate firewalls, cloud security groups) allow access to this port from the required client networks.

If TLS termination is performed by an external component (e.g. reverse proxy, load balancer), ensure that this component can reach the Web UI container on the configured port.

Kubernetes Installation

For Kubernetes-based installations, the Web UI is provided by a dedicated Pod which is deployed as part of the default Helm chart.

  • The Helm chart deploys an Ingress resource to provide external access to the XDM user interface.

  • By default, HTTP port 80 is used to access the XDM UI via the Ingress endpoint.

  • For HTTPS, custom ports, or more user-friendly URLs (hostnames, paths), adjust:

    • the Ingress resource (hosts, paths, TLS secret), and

    • the Ingress controller configuration as required.

Ensure that:

  • the Ingress controller is reachable from the required client networks, and

  • all relevant firewalls and (if used) Kubernetes NetworkPolicies allow traffic from the Ingress controller to the XDM Web UI service.

Infrastructure and Connectivity

Firewalls

  • The XDM user interface must be accessible through all relevant firewalls from:

    • users' local web browsers, and

    • any services or automation tools that need to interact with the XDM UI or API.

  • XDM itself requires network access through firewalls to all integrated systems, including but not limited to:

    • database systems used by XDM,

    • directory services (e.g. LDAP/Active Directory),

    • authentication/authorization services (e.g. identity providers, SSO),

    • external systems and applications with which XDM exchanges data (e.g. via REST, message queues, or other protocols).

DNS and Name Resolution

  • DNS resolution must be available for all hostnames that XDM needs to access (e.g. databases, LDAP, external services).

  • If the web interface is published via a hostname, this hostname must be resolvable in the DNS used by client systems.

TLS/SSL

  • For production environments, it is strongly recommended to expose the web interface only via HTTPS.

  • Ensure that:

    • valid TLS certificates (e.g. from a trusted CA) are provided,

    • TLS is terminated either directly by the XDM web UI or by an upstream component (reverse proxy, Ingress controller, load balancer),

    • client systems trust the certificate chain used (root/intermediate CAs are installed where required).

Database Connectors

Each supported database is connected via a dedicated connector. Configuration is handled through environment variables per service (for Docker Compose/Podman), or centrally via the values.yml (for HELM).

Db2 z/OS Connector

  • Supported versions: V11, V12, V13

  • JDBC driver: ≥ 4.26.16, or HIT JDBC from 4.12.413 (with limitations)

  • Required privileges: insert, update, delete, and optionally create table, index, trigger, etc. See details per connector.

Db2 LUW Connector

  • Supported versions: 9.7 through 11.5

  • JDBC driver: ≥ 4.26.16

  • Required privileges: align with Db2 z/OS requirements

Db2 for iSeries Connector

  • Supported versions: 7.5+

  • JDBC driver: ≥ 4.26.16

  • Required privileges: align with Db2 z/OS requirements

Oracle DB Connector

  • Supported versions: 12c, 18c, 19c, 21c

  • JDBC driver: ≥ 12.1.0.2.0

  • Required privileges: table and schema access as necessary

MS SQL Server Connector

  • Supported versions: 2012-2022

  • JDBC driver: ≥ 14

  • Required privileges: table and schema access

IBM IMS/DB Connector

  • Supported versions: 11-15

  • JDBC driver: ≥ 14.0

PostgreSQL Connector

  • Supported versions: 10-17

  • JDBC driver: ≥ 42.4.1

Snowflake Connector

  • Supported versions: 8.41.2+

  • JDBC driver: ≥ 3.10.0

BigQuery Connector

  • JDBC driver: Simba JDBC ≥ 1.3.0.1001

In each case:

  • The connector requires network access to the JDBC port of the target DB; the user account must have appropriate rights as per the connector requirements.

  • Detailed catalog table access information is provided in the technical appendix.

Additional Requirements

  • Internet access (for pulling Docker images; alternatively, images may be provided offline using docker load)

  • Network and firewall configuration: Ensure access from browser to web-ui, between internal services, and from connectors to target databases

  • Optionally, enable TLS/HTTPS for secure web interface and backend service connections

Connector permission requirements

The required permissions depend on the task types to be used. For all data copy operations, there are two different modes of operation: Native and Compatibility. While compatibility completely relies on SQL statements to extract and insert data, native mode uses database-specific utilities to perform these operations. Therefore, the required permissions depend on the selected mode of operation.

For each database that is used as the source or target of an XDM task, the following requirements must be met:

  • A suitable JDBC driver must exist in the driver directory of the XDM installation (see Driver Configuration)

  • The JDBC port of the database must accept incoming connections and must not be blocked by a firewall

  • The user ID that is used to connect to the database must have read access to the database meta data (catalog / information schema). A detailed list of catalog tables accessed by XDM can be found under this paragraph.

For databases that are used as a source database for an XDM task, the following requirements must also be met:

  • The user ID that is used to connect to the database must have read access to the tables that participate in the XDM task

  • Native Table Copy tasks only: The database server must be accessible via FTP, SFTP or FTPS

  • Native Table Copy tasks only: If the task is configured to stop the source environment for the duration of copy process, the user ID that is used to connect to the database must have the authorization to stop and start the source tables.

  • Native Table Copy tasks only: If table data is supposed to be extracted using the built-in utilities of the database (e.g., UNLOAD or EXPORT), the user ID that is used to connect to the database must have the authorization to execute these utilities.

For databases that are used as a target database for an XDM task, the following requirements must also be met:

  • The user ID that is used to connect to the database must have read and write access to the tables that participate in the XDM task

  • If the task is configured to create missing target tables, or to recreate target tables that have structural differences, DROP and CREATE authorization for tables, indexes and other database objects is required.

  • Native Table Copy tasks only: If the task is configured to stop the target environment for the duration of copy process, the user ID that is used to connect to the database must have the authorization to stop and start the target tables.

  • Native Table Copy tasks only: If table data is supposed to be inserted using the built-in utilities of the database (e.g., LOAD), the user ID that is used to connect to the database must have the authorization to execute these utilities. In addition, authorization to execute database-specific commands and utilities such as REPAIR or SET INTEGRITY is required.

Accessed tables in catalog/information schema per DBMS

The following tables of catalog/information schema are accessed by XDM. The user ID used to connect to the database must have read access to these tables.

Db2 z/OS Db2 LUW Oracle DB

SYSCHECKS
SYSCOLAUTH
SYSCOLUMNS
SYSDATABASE
SYSDUMMY1
SYSFOREIGNKEYS
SYSINDEXES
SYSINDEXPART
SYSKEYCOLUSE
SYSKEYS
SYSKEYTARGETS
SYSRELS
SYSSEQUENCEAUTH
SYSSEQUENCES
SYSSEQUENCESDEP
SYSTABAUTH
SYSTABCONST
SYSTABLEPART
SYSTABLES
SYSTABLESPACE
SYSVIEWDEP
SYSVIEWS

SYSIBMADM.ADMINTABINFO
SYSIBMADM.CONTAINER_UTILIZATION
SYSIBMADM.MON_TBSP_UTILIZATION
SYSCAT.BUFFERPOOLS
SYSCAT.CHECKS
SYSCAT.COLAUTH
SYSCAT.COLIDENTATTRIBUTES
SYSCAT.COLUMNS
SYSCAT.DATAPARTITIONS
SYSCAT.DATAPARTITIONEXPRESSION
SYSCAT.DATATYPES
SYSCAT.INDEXCOLUSE
SYSCAT.INDEXES
SYSCAT.INDEXPARTITIONS
SYSCAT.KEYCOLUSE
SYSCAT.PERIODS
SYSCAT.REFERENCES
SYSCAT.SEQUENCEAUTH
SYSCAT.SEQUENCES
SYSCAT.TABAUTH
SYSCAT.TABCONST
SYSCAT.TABDEP
SYSCAT.TABLES
SYSCAT.TBSPACEAUTH
SYSCAT.TABLESPACES
SYSCAT.VIEWS

ALL_COL_PRIVS
ALL_CONS_COLUMNS
ALL_CONSTRAINTS
ALL_IND_COLUMNS
ALL_IND_EXPRESSIONS
ALL_IND_PARTITIONS
ALL_INDEXES
ALL_LOBS
ALL_OBJECTS
ALL_PART_INDEXES
ALL_PART_KEY_COLUMNS
ALL_PART_TABLES
ALL_SEQUENCES
ALL_SYNONYMS
ALL_TAB_COLS
ALL_TAB_COMMENTS
ALL_TAB_PARTITIONS
ALL_TAB_PRIVS
ALL_TABLES
ALL_TABLES ALT
ALL_XML_TAB_COLS
DBA_DATA_FILES
DBA_SEGMENTS
DBA_TAB_PRIVS
DBA_TS_QUOTAS AUTH
OWNER_EXTENTS
USER_EXTENTS
USER_TABLESPACES
MS SQL Server PostgreSQL

schemas
indexes
data_spaces
database_principals
tables
columns
database_permissions
synonyms
database_files
filegroups
types
identity_columns
index_columns
foreign_keys
foreign_key_columns
check_constraints
key_constraints

information_schema.columns
information_schema.column_privileges
information_schema.key_column_usage
information_schema.referential_constraints
information_schema.role_table_grants
information_schema.table_constraints
information_schema.tables
pg_catalog.pg_am
pg_catalog.pg_attribute
pg_catalog.pg_class
pg_catalog.pg_constraint
pg_catalog.pg_depend
pg_catalog.pg_index
pg_catalog.pg_namespace
pg_catalog.pg_tables
pg_catalog.pg_tablespace
pg_catalog.pg_trigger
pg_catalog.pg_user