Loading data into IBM InfoSphere MDM

IBM InfoSphere MDM | Master Data Management Server provides a customer lot of abilities to do customization's at any stage of the MDM process. Here are some of the things which I have recently come across and thought would be useful.

The first thing the customer wants to do/see is to have their data loaded onto MDM. This is a very common use case and see how it integrates well with his existing systems. IBM InfoSphere MDM provides various capabilities to achieve the same.

   

This involves two main steps :

1. Create and define data model mapping between the customer data format to MDM data model.
2. Load the data file into MDM.

Step 1

Adaptive Services Interface (ASI) framework MDM provides a graphical, easy-to-use interface to map an existing data model to the predefined data models/transactions of MDM.

The tools required here would only be IBM MDM Workbench that gets installed on any eclipse based IDEs like Rational Software Architect (RSA) or Rational Application Developer (RAD).

The steps to do the same are:

• Open up a new work space in RSA or RAD
• Move to MDM Development perspective.
• Create a new service mapping project. You need to give here various details like Project name, package name, the CBA (Composite bundle archive) this project needs to be associated with and so on.
• The next page gives you an option to choose between these two options.

• Delimited text file - This means the data file you want to load to mdm is in a text file and it has certain fields and a certain delimiter too.
• Importing existing Wsdl or xsd - This means you want to invoke mdm transactions with your existing wsdl or xsd structure.
• Once we create this project the next would be to do the mapping.
• Mapping comes in two parts.
• One is to map the transaction which we want to run as part of this mapping. For example addParty, addPerson or so on. This is done by opening up mapping.mdmxmi file in the project created with service mapping editor.
• Other is to actually map each column of the text field to an existing attribute in MDM data model. For example map the second column to BirthDate field in TCRMPersonBObj. The previous step generates a .map file under transform folder of the project. The .map can now be opened with mapping editor that provides a graphical mapping editor . This actually allows you to connect the left hand side i.e., list of columns in your text file to any field of the entire business objects of MDM data model.

• Once the mapping is done for all the columns in the text file, you need to clean and build all the projects to make sure there are no errors in workspace.
• We need to deploy this CBA to the WebSphere Application Server where MDM is installed and deployed.
• All metadata sqls are generated under sql folder of the project. Based on your database connected to MDM Server installation i.e.,DB2, Oracle or so the sqls need to be run on the MDM databases.
• The WebSphere application server where MDM is deployed needs to be restarted after all these steps.

    So now we are done with creating a mapping project that maps your data file to mdm data model to directly load the data onto MDM.

Step 2

To do this the Customer can utilize the MDM Batch processor framework capabilities.

We need to run MDM Batch Framework providing these parameters.

1. Text file to be loaded onto MDM.
2. The output directory where the logs related to running the batch processor will be created. The logs will have the response from MDM for each record that is being inserted. The response could be a successful record added or a failed record.
3. Additional configurations - In the above project you created under the properties folder , a *batch_extension.properties file would have got created. This needs to be passed as additional configuration.

    Once we run this batch processor a transaction gets invoked for each record in the data file and it actually gets loaded onto InfoSphere MDM