Backup and restore an XDM instance

Overview

An XDM instance consists of several parts that are important, and which should be backed up. During the backup process XDM should be stopped to avoid inconsistency. This means there should no task be executed, or objects created, changed, or deleted.

The backup and restore process includes two things:

  1. The configuration files

  2. The XDM admin database

Configuration files backup

One opportunity to install XDM is the Kubernetes installation. Therefore, the Kubernetes helm charts are used. To back up them, it is not necessary to copy them to another save place because they are normally stored in a git repository.

Another opportunity is the docker compose installation. Within this installation the configuration files describe how and where XDM should run. These files are important to be able to start XDM and build an instance. The configuration contains the configuration files as well as license files or JDBC drivers. These files should be backed up via a file backup process for the server. The configuration consists of the following files and directories:

  • xdm-config

  • xdm-backups

  • xdm-data

  • xdm-tasks

  • docker-compose.yml

  • docker-compose-user.yml

To back up these configuration files and directories, copy them to another save place.

The four configuration directories xdm-config, xdm-backups, xdm-data, xdm-tasks have to be mounted as volumes in the docker-compose.yaml. Through the mounting the configuration directories are available from the host machine. For more information see volumes and mount points.

Configuration files restore

To restore the configuration files and directories, copy them into the directory where XDM is installed and replace the existing files.

Backup and restore process for Docker

Database backup

The database contains all objects and processes defined in XDM together with the historical information to these objects.

To prepare a backup, shutdown XDM and only start the XDM db:

docker-compose down

docker-compose up -d xdm-db
This is only an example how an backup can be done. This example uses the pgdump command of the postgres database. Please have a look at the postgres pgdump command documentation to get more information.

To back up an XDM instance, create a zipped backup file named xdm-db-$(date +%Y-%m-%d).sql.gz and put it into the directory admin-backups by running the following command:

docker compose exec -T xdm-db pg_dump -U xdm --clean --insert | gzip > ./admin-backups/xdm-db-$(date +%Y-%m-%d).sql.gz
Verify that the backup file xdm-db-$(date +%Y-%m-%d).sql.gz has been created.

  • the option --clean creates drop commands to delete existing objects in the database

  • the option --insert is used to create single INSERT statements instead of COPY. This makes the backup file larger and slows down the restore process. But it is useful to restore only specific rows of a table instead of the whole table.

  • there are more options that can be used. Please have a look at the postgres pgdump documentation to get a full list.

The previous command can be placed into a .sh file. This allows you to create a cronjob to automatically create backups with the following example expression:

0 0 * * * /opt/xdm/backup-xdm-db.sh

Database restore

Please note that restoring a backup into XDM may take some time.

To restore an XDM instance the following steps need to be performed:

  1. Shutdown XDM and only start the XDM db:

docker-compose down
docker-compose up -d xdm-db

Use the following command to restore the backup to the database.

gunzip < ./admin-backups/xdm-db-$(date +%Y-%m-%d).sql.gz | docker-compose exec -T xdm-db psql -U xdm

Finally, the complete XDM environment can be restarted.

docker-compose up -d
After restarting XDM the neo4j graph database must be rebuilt. To do this, in the XDM web-ui go to Settings → System Configuration → Rebuild Graph .

Backup and restore process for Kubernetes

Database backup

Make sure no objects are created during the pg_dump command is running.
This is only an example how an backup can be done. This example uses the pgdump command of the postgres database. Please have a look at the postgres pgdump command documentation to get more information.

  • Run the following command to create a backup of your database:

kubectl --namespace <namespace> exec -it <postgres-container-name> -- bash -c "export PGPASSWORD='xdm'; pg_dump -U xdm --clean --insert" > xdm-db-backup.sql
Verify that the backup file xdm-db-backup.sql has been created.

  • the option --clean creates drop commands to delete existing objects in the database

  • the option --insert is used to create single INSERT statements instead of COPY. This makes the backup file larger and slows down the restore process. But it is useful to restore only specific rows of a table instead of the whole table.

  • there are more options that can be used. Please have a look at the postgres pgdump documentation to get a full list.

Database restore

Please note that restoring a backup into XDM may take some time.

  • Delete the current xdm-postgres pod:

kubectl delete --namespace <namespace> pod xdm-postgres-0

  • Delete the persistent volume claim for the xdm-postgres pod:

kubectl delete --namespace <namespace> pvc xdm-postgres
This may take some time, depending on the size of the database.

  • Upgrade the xdm environment:

helm upgrade <release> <repository-name> --values values.yaml --namespace <namespace>

  • Import the previously created backup:

cat xdm-db-backup.sql | kubectl --namespace <namespace> exec -i xdm-postgres-0 -- bash -c "export PGPASSWORD='xdm'; psql -U xdm"

  • Restart the xdm-core pod:

Use the command kubectl get pods --namespace <namespace> to get the name of the xdm-core pod. 
kubectl delete --namespace <namespace> pod <xdm-core-pod-name>
After restarting XDM the neo4j graph database must be rebuilt. To do this, in the XDM web-ui go to Settings → System Configuration → Rebuild Graph .