Modification Method Script Examples

The following chapter shows examples of how to implement some general use cases of Entity Modification Methods.

More information about Entity Modification Methods themselves as configuration objects can be found here.

Example of an Entity Modification Method to modify nested Kafka messages

The following example demonstrates how to modify data within nested Kafka messages. The example is based on the sample kafka topic production.departments and its department domain model.

With the Entity Modification Method provided by this example, every data point in messages like the following can be modified:

{
  "dept_no": "d005",
  "dept_name": "Development",
  "dept_managers": [
    {
      "emp_no": 110511,
      "from_date": "2005-01-01",
      "to_date": "2012-04-25",
      "manager": {
        "emp_no": 110511,
        "birth_date": "1977-07-08",
        "first_name": "DeForest",
        "last_name": "Hagimont",
        "gender": "M",
        "hire_date": "2005-01-01",
        "email": "deforest.hagimont@example.com",
        "weekly_work_hours": 20.0,
        "phone_number": "+ 44 9831 2297631"
      }
    },
    {
      "emp_no": 110567,
      "from_date": "2012-04-25",
      "to_date": "9999-01-01",
      "manager": {
        "emp_no": 110567,
        "birth_date": "1984-04-25",
        "first_name": "Leon",
        "last_name": "DasSarma",
        "gender": "F",
        "hire_date": "2006-10-21",
        "email": "leon.dassarma@example.com",
        "weekly_work_hours": 20.0,
        "phone_number": "+77572701457"
      }
    }
  ]
}

To access data points in the nested manager objects, the following Entity Modification Method can be used:

import groovy.json.JsonOutput
import groovy.json.JsonSlurper

/*
 * This function is called once before the processing starts.
 */
def void init() {

}

/*
 * This method is called for each entity.
 * The entity's attributes are provided to the method through the parameter `data`.
 * These map uses the name of an attribute as the key. The corresponding value is converted according to its
 * specified type.
 */
def void apply(Map<String,Object> data) {

    def managersJson = data.get("dept_managers")
    def managers = new JsonSlurper().parseText(managersJson)

    managers.each { Map<String, Object> m ->
        m.manager.first_name = "REDACTED"
        m.manager.last_name = "REDACTED"
        m.manager.email = "REDACTED"
    }

    data.put("dept_managers", JsonOutput.toJson(managers))
}

/*
 * This function is executed once after the modification of all entities is complete.
 */
def void close() {

}

As shown in the code example, the actual modification takes place in the apply method, which is called for each entity. The entity’s attributes are provided to the method through the parameter data. This map uses the name of an attribute as the key. The corresponding value is converted according to its specified type. By calling data.get("dept_managers") a string is returned, containing every manager and their data. That string can then be used to create a map in which the data may be modified. After modifying the map it can then be inserted back into the data map via data.put(..), to override the unmodified message.

The modified result looks like this:

{
  "dept_no": "d005",
  "dept_name": "Development",
  "dept_managers": [
    {
      "emp_no": 110511,
      "from_date": "2005-01-01",
      "to_date": "2012-04-25",
      "manager": {
        "emp_no": 110511,
        "birth_date": "1977-07-08",
        "first_name": "REDACTED",
        "last_name": "REDACTED",
        "gender": "M",
        "hire_date": "2005-01-01",
        "email": "REDACTED",
        "weekly_work_hours": 20.0,
        "phone_number": "+ 44 9831 2297631"
      }
    },
    {
      "emp_no": 110567,
      "from_date": "2012-04-25",
      "to_date": "9999-01-01",
      "manager": {
        "emp_no": 110567,
        "birth_date": "1984-04-25",
        "first_name": "REDACTED",
        "last_name": "REDACTED",
        "gender": "F",
        "hire_date": "2006-10-21",
        "email": "REDACTED",
        "weekly_work_hours": 20.0,
        "phone_number": "+77572701457"
      }
    }
  ]
}