Task reports

Introduction

This document describes all task reports that are generated by XDM and what their relevance is for the various task types.

The purpose of this document is to give the user better understanding of what results one can expect in the task reports, and how one can use these to check whether the task is running correctly.

Each task report description contains:

Description
Columns in the report
Row dependencies, if there are any

The following are the various task stage objectives and the reports they generate.

Tailoring

Column Exclude Rule Report

The column exclude rule report contains information about the Column Exclude Rules. used and, therefore, which columns are to be excluded in the subsequent task run.

Report columns

The following columns are displayed in the Column Exclude Rule Report.

Column Specification Rule Report

The column specification rule report contains information about the Column Specification Rules. used in the task. These rules determine which columns will be selected for data extraction and thereby enhance the filter options of the specified task.

Report columns

The following columns are displayed in the Column Specification Rule Report.

Credential Report

The credential report shows the list of credentials which are used to authenticate XDM with a database or server.

Every credential is listed with the corresponding type. This indicates whether it is used for a connection or a storage location.

Report columns

The following columns are displayed in the report.

Property

The name of the credential type. This indicates whether the credential is used for source or target, storage location or connection, and can also have an index.
For example:

tos_user: defines the target operating system user credential for a storage location.
sdb0_user: defines the source database user credential for the first of multiple source connections in a task.

Value

The name of the credential. This is typically a user name.

Data Apply Rule Report

The data apply rule report contains the information about Data Apply Rules used in the task. These rules define the order in which related tables are processed in the target environment and how modification rules are propagated.

Report columns

The following columns are displayed in the Data Apply Rule Report.

Data Relation Rule Report

The data relation rule report contains information about Data Relation Rules. used and, therefore, how table relationships are handled in a task run.

Report columns

The following columns are displayed in the Data Relation Rule Report.

Exclude Rule Report

The exclude rule report contains information about Exclude Rules. used in the task. These rules specify which objects should not be considered for selection.

Report columns

The following columns are displayed in the report.

Extract Key Rule Report

The extract key rule report contains information about Extract Data Rules. used and, therefore which columns are written to the output file.

Report columns

The following columns are displayed in the report.

Ignore Difference Rule Report

The ignore difference rule report contains information about the Ignore Difference Rules. used in a structure compare task, and, thereby, which objects are to be ignored in the subsequent task run.

Report columns

The following columns are displayed in the report.

Mapping Rule Report

The mapping rule report contains information about Mapping Rules. used by a task to change the target names of selected tables, indexes, sequences, and other objects.

Report columns

The following columns are displayed in the report.

Modification Rule Report

The modification rule report contains the information about Modification Rules. used and, therefore, which table content is modified by the task.

Report columns

The following columns are displayed in the report.

Reduction Rule Report

The reduction rule report contains information about Reduction Rules. used in a task. These rules restrict the number of rows copied to the target environment.

Report columns

The following columns are displayed in the report.

Selection Rule Report

The selection rule report contains the information about Selection Rules. used and, therefore, which objects should be selected in a task execution.

Report columns

The following columns are displayed in the report.

Task Properties

The task properties report contains information about property settings used in the task run.

Report columns

The following columns are displayed in the report.

Category: The category to which the property belongs.
Property: The property name.
Value: The value used in the task run.

User Defined Primary Key Report

The user defined primary report contains information User Defined Primary Keys.

Report columns

The following columns are displayed in the report.

Fetch source objects

Source Objects Report

The source objects report contains information about the tables that were selected in the source environment using the defined selection rules or data relation rules. Objects which are matched by an exclude rule are not reported. Tables not listed in the source report are not available in further steps of the task execution.

Report columns

The following columns are displayed in the report.

Application model: The application model for the source table.
Source system: The connection name for the source table.
Source schema: The schema name of the source table.
Source name: The name of the source table.

Data collection

Subset Extraction Report

The subset extraction report contains information about the rows that were extracted using row level processing. For each table the number of extracted rows is listed.

Report columns

The following columns are displayed in the report.

System: The name of the connection for the extracted table
Application model: The application model for the source table.
Schema: The schema name of the extracted table
Table: The name of the extracted table
Rows: The number of successful extracted rows

Data Extraction Report

The data extraction report contains information about the rows that were extracted using a database utility or by fetching the data with SQL. For each table the number of extracted rows is listed.

Report columns

The following columns are displayed in the report.

System: The name of the connection of the extracted table
Schema: The name of the schema of the extracted table
Table: The name of the extracted table
Rows: The number of successfully extracted rows

Performance Statistics

The performance statistics report contains information actions performed by the Data Extractor. It is intended to help explain how much time was used by each Data Extractor action in comparison to the overall runtime.

Report columns

For each action there is a column listing the number of seconds spent performing the action and a second column showing the time spent as a percentage of the total time. One row is written every 5 minutes and the times displayed are cumulative.

The following columns are displayed in the report.

Fetch (s): Time spent fetching data from the database.
Fetch (%): Fetch time as a percentage of total time.
Parse (s): Time spent retrieving data from the fetched rows.
Parse (%): Parse time as a percentage of total time.
Modify (s): Time spent modifying data.
Modify (%): Modification time as a percentage of total time.
Convert & write (s): Time spent writing the extracted data, including any necessary data type conversion.
Convert & write (%): Convert and write time as a percentage of total time.
Total (s): Total time used for data extraction.

Fetch target objects

Target Objects Report

The target objects report contains information about the tables that were found in the specified target for the source objects using the defined mapping rules. It shows the source table and the associated target table. Tables not listed in the target report are not available in further steps of the task execution. In some tasks, depending on the drop option setting, the target table can be generated in a later stage.

Report columns

The following columns are displayed in the report.

Application model: The application model for the target table.
Source system: The connection name for the source table.
Source schema: The schema name of the source table.
Source name: The name of the source table.
Target system: The connection name for the target table.
Target schema: The schema name of the target table.
Target name: The name of the target table.

Compare structures

Comparison Result Report

The comparison result report contains the results produced by the Object Comparer. This report contains all comparisons between source and target objects.

Report columns

The following columns are displayed in the Comparison Result Report.

Object type: The type of database object.
Parent object: The parent object of the compared database objects.
Object name: The name of the source database object.
Compared object: The name of the compared target database object.
Result: The result of the comparison. This can be EQUAL, DIFFERENT, or MISSING.
Mapping: Whether the database object was mapped with a Mapping Rule. :noindex:

Structure Compare Difference Report

The structure compare difference report contains information about the structural differences between corresponding source and target objects found in a structure compare task.

Report columns

The following columns are displayed in the report.

Object type: The type of database object.
Parent object: The parent object of the to be compared database object.
Object name: The name of the source object.
Compared object: The name of the corresponding target object.
Result: The result for each compared object. This either DIFFERENT or MISSING.
Mapping: Whether the database object mapped with a Mapping Rule. :noindex:

Columns Values Need To Be Generated For Report

This report contains the results produced by the Object Comparer. This is a listing of all additional target columns, that are not nullable and have no default value. To write into those tables, an additional modification rule with scope target is necessary in a copy task between the source and target objects.

Report columns

The following columns are displayed in the Comparison Result Report.

Object type: The type of database object.
Parent object: The parent object of the compared database objects.
Object name: The name of the source database object.
Compared object: The name of the compared target database object.

Load/Apply data

Apply Row Operations Report

The apply row operations report contains information about rows that were applied to a target table using subset apply. For each target table the number of applied rows is listed.

Report columns

The following columns are displayed in the report.

Application model: The application model for the target table.
System: The name of the connection of the target table
Schema: The name of the schema of the target table
Table: The name of the target table
applies: The number of rows applied with INSERT, UPDATE, or MERGE statements.
deletes: The number of deleted rows
discards: The number of operations that were prohibited by the database system. Typically, these are unique key conflicts or foreign key conflicts. Check the discards report for more details. :noindex:

Discards Report

The discards report contains information about the rows that were discarded by subset apply. Similar discard reasons per table are aggregated.

Report columns

The following columns are displayed in the report.

Application model

The application model for the table.

System

The name of the connection for the table

Table name

The name and schema

Error type

The type of discard. Possible values:

MISSING_PARENT - A row should be inserted in a table with a foreign key constraint, but the respective key was missing in the parent table.
UNIQUE_CONFLICT - A row with an identical key already exists in the table.
REMAINING_CHILDREN - A delete operation in a parent table is not possible because there are rows in a child table which reference it.
CHECK_CONFLICT - A check constraints prohibits the operation.
TRIGGER_BLOCK - A trigger prohibits the operation.

Exception

The message issued by the database system. This may give more details on the constraint that prohibits the operation. The details of this message depend on the database system.

Error type count

The number of identical messages and error types issued per table.

Discard Data Report

The discard data report contains detailed information about the rows that were discarded by subset apply. Unlike in the discards report, similar discard reasons per table are not aggregated.

It lists the table, the exact row key, and the reason why the row was discarded. It lists the first 1000 discarded rows, the value can be increased by using the task runtime property subapply.discardMaxReportLines.

Report columns

The following columns are displayed in the report.

Table name: The name and schema
Discarded data: The data that was discarded.
Reason: The message issued by the database system. This may give more details on the constraint that prohibits the operation. The details of this message depend on the database system.

Data Import Report

The data import report contains information about rows that were imported using a database utility. For each table the number of imported rows is listed.

Report columns

The following columns are displayed in the report.

System: The name of the connection for the loaded table.
Schema: The schema of the loaded table.
Table: The name of the loaded table.
Rows: The number of successfully loaded rows.

Analyse data

Matched Columns Report

The matched columns report contains information about the columns that were found by the configured matchers.

Report columns

The following columns are displayed in the report.

Category: The category of the matcher that matched on the column data
System: The name of the connection for the matched table
Schema: The schema name of the matched table
Table: The name of the matched table
Column: The column on which the matcher had hits
Probability: The percentage of matched data entries in the analyzed rows (shown if task property Use absolute count is false). Only shown if the column’s probability is higher than the configured probability in the PII finder task.
Absolute Count: The number of matched data entries in the analyzed rows (shown if task property Use absolute count is true)

Trace data

Trace Data Report

The trace data report contains information about the data written by an active trace, configured by the trace data configuration property.

Trace Data Modification Report

The trace data modification report is a special kind of trace data report which contains an entry whenever the trace expression is affected by a modification method. The definition of trace data report is also valid here.

Report columns

The report can contain different columns depending on the batch program evaluating the trace expression. The following columns are displayed in the report.

Schema: The schema of the table with the matched trace data expression
Table: The name of the table with the matched trace data expression
Referenced Schema: When the report is written by the subset extractor, contains the referenced table’s schema
Referenced Table: When the report is written by the subset extractor, contains the referenced table’s name
Data Relation Rule: When the report is written by the subset extractor, contains the name of the data relation rule which selected the data.
Trace data expression: The expression as configured in the task
Row data: The data that matched the expression
Unmodified data: When the report is a modification report, contains the data row that matched before the modification
Modified data: When the report is a modification report, contains the data row that matched after the modification

Modification

Modification Summary Report

The modification summary report contains information about the overall modification that was applied per table. There is one row per table and the operation statistics as aggregated over all applied row and column based methods.

For column based methods, the number of modified rows has a verification to be sure that the actual value was modified. For row based methods, a verification is not possible without decreasing the performance of the modification process. Here every written row is counted as modified.

Report columns

The following columns are displayed in the report.

Application model: The application model for the modified table.
System: The name of the connection for the modified table.
Schema: The schema name of the modified table.
Table: The name of the modified table.
Row based modification: The number of rows that were modified by row based modification methods.
Column based modification: The number of rows that were modified by column based modification methods.
extracted: The number of rows that were used as input for the modification process.
modified: The number of rows that were actually modified. Rows that were filtered by rule condition script and rows that were not changed by the modification do not count to this number. Rows affected by row based methods are always counted as modified as checking if the row was modified is too time-consuming in this situation and would decrease the performance.
written: The number of rows written after modification. Modification methods which filter rows or multiply rows may affect this number. :noindex:

Source Scope Modification Report

The source scope modification report contains information about the rows that were modified by a modification rule with scope SOURCE. There is one line per combination of affected table and method. If a table is modified by multiple methods, there will be multiple rows in the report. If a method is used to modify multiple tables, each use will be listed in a single row.

Report columns

The following columns are displayed in the report.

Application model: The application model for the modified table.
System: The name of the connection for the modified table
Schema: The name of the schema for the modified table
Table: The name of the modified table
Column: The name of the modified column, if modification was column based
Method: The modification method used, identified by category and display name
Modified rows: The number of successfully modified rows

Target Scope Modification Report

The target scope modification report contains information about the rows that were modified by a modification rule with scope TARGET. There is one line per combination of affected table and method. If a table is modified by multiple methods, there will be multiple rows in the report. If a method is used to modify multiple tables, each use will be listed in a single row.

Report columns

The following columns are displayed in the report.

Application model: The application model for the modified table.
System: The name of the connection for the modified table
Schema: The name of the schema for the modified table
Table: The name of the modified table
Column: The name of the modified column, if modification was column based
Method: The modification method used, identified by category and display name
Modified rows: The number of successfully modified rows

Foreign Key

Foreign Keys Outside of Selection Set Report

The foreign keys outside of selection set report contains information about those foreign keys of the tables in the source object report, where either the referenced table or the base table is outside of the selection set. This can lead to erroneous behavior or data not being copied as expected. Users can use this report to further analyze their task setup.

Report columns

The following columns are displayed in the report.

Constraint: The name of the foreign key constraint
Schema: The schema of the base table
Table: The name of the base table
Ref. Schema: The schema of the referenced table
Ref. Table: The name of the referenced table