Any large enterprise runs multiple automated systems. Often, the same data sets are processed in several applications as part of the automation of close business processes. For example, in banks, the ABS system is responsible for automating settlement operations, but work on attracting new customers and planning marketing communications takes place in the CRM system, loan applications are processed in specialized software, and a separate software package is responsible for Internet banking.
All these systems work with the same data: about customers, current accounts and bank products, contracts. Each of the applications can change this data, and all the others must quickly learn about the changes. The need arises to create another system that maintains reference data and distributes it to business applications. Such systems are called MDM (Master Data Management).
Any MDM system must perform the following functions:
- Managing the composition and structure of master data: as requirements change depending on business needs, you need the ability to add new types of objects and their properties to the master dataset.
- Data object management: Sometimes you need to edit master data directly from the MDM system. Often this requires reconciling changes according to a business process template.
- Track changes to business application data and automatically upload them to MDM with structure transformation.
- Normalization, or data cleaning: information received from applications should be cleaned of input errors, and some property values should be brought to a standard format. Normalization should be performed using rules configured by the analyst, not by programmed procedures.
- Validation, or format-logic control: the system should allow you to configure a set of rules that all loaded data is checked against. The rules can be informative or blocking: values of object properties that do not meet the blocking rules should not be included in the reference objects.
- Deduplication and linking of data objects describing the same business object: for example, cards for the same customer may be retrieved from different systems, and duplicate cards may exist in some specific systems
- Generating reference records: the system should allow you to customize the rules for generating reference data objects. Such objects should consolidate the most recent, reliable, verified information about each business object
- Distribution of master data to source systems: The MDM system should have a well-developed API so that it is easy to integrate with business applications that need to receive master data in a synchronous or asynchronous (subscription-based) manner. The same block of functions should also allow the creation of showcases of reference data
- Data analysis: includes various types of statistical analysis as well as viewing reports on the application of data validation rules. Data stewards should be able to view data objects that have not passed format-logical control and correct them directly in MDM or from the source systems.
Often MDM systems are divided by their purpose depending on what kind of data they process.
- PIM (Product Information Management) - systems for processing data on products, nomenclature of manufactured or used products. They have developed functionality in terms of storing the values of product properties, assigning them position codes, in which each group of symbols means certain characteristics. Management of physical units and conversion of property values from one unit to another can be realized.
- CDI (Customer Data Integration) is the most common type of MDM systems designed to process customer data. It can evolve towards Customer 360 class systems that store not only basic, rarely changing data about customers, but also consolidate all information about interactions with them. CDI systems contain functions of integration with state data sources on individuals and legal entities, rules for cleaning contact information.
Another way to classify MDM systems is by their purpose in business processes: there are operational MDMs, which are focused on distributing data between applications in real time, and analytical MDMs, which only accumulate data from all business applications for the purpose of analysis.
At DataVera, we consider both classifications obsolete: a modern MDM solution should process any data regardless of the subject area, and perform both analytical and operational functions. Our DataVera EKG Platform product does just that.
Implementing MDM creates the following business benefits:
- Reduction of losses due to possible data errors. For example, a manager may make a wrong decision when working with a client due to data about the client not updated in time, which will result in direct losses or lost profit. At the level of the whole enterprise, the analysis of cleaned, reference data will allow making informed management decisions.
- Replacing multiple "point-to-point" integrations between automated systems with centralized data exchange via MDM will make business processes more reliable and faster, as well as reduce operational costs to support integrations and better manage IT budgets.
- The ability to manage the structure of core corporate data from a single point will accelerate the customization and refinement of business applications in accordance with business requirements. This will enable faster innovation and digital transformation of the company.
- Having a ready set of reference data will accelerate the implementation of new business applications and reduce the cost of such projects. This will enable businesses to more quickly apply the most modern and efficient IT solutions to outpace competitors.
DataVera EKG Platform meets all of these functional requirements and allows you to realize these benefits.
For example, DataVera EKG Platform implements the following validation and data cleansing rules:
- Bringing full name and other string values to a single case: upper case, lower case, CamelCase
- Removing unnecessary, repetitive and special characters (spaces, hyphens, etc.)
- Correction of keyboard layout errors during input - for example, replacing English letters "a", "o", "c" etc. with corresponding Cyrillic characters, and vice versa
- Normalization and verification of Kazakhstani mobile and city phone numbers, verification of telephone codes
- Control of repetitive characters in phone numbers, documents, IINs and other codes: the value of such fields cannot consist only of 0, 1 or other characters
- Checking whether the IIN corresponds to gender and date of birth
- Parsing and verification of dates: birth, document issuance, etc.
- Parsing of postal addresses, canonical address compilation, CATO code determination or checking if the CATO code matches the address (this function is available on our website as separate service).
- Detection of duplicate data objects by various criteria: customers - by IIN or phone number, documents - by type and number, etc.
The analyst can configure any other rules of data cleaning, validation and deduplication in the system without interfering with the program code. Rules can include conditions - for example, depending on the presence of values of some object properties, other properties become mandatory.
Our product, created by a Kazakhstani vendor, is successfully implemented in large financial organizations. Our team has extensive experience of MDM implementation in industry, at enterprises of different sectors.
We invite you to read more information about our MDM solution, MDM implementation services, and contact us to discuss your challenges.