MDM-systems (Master Data Management) is a class of software products that has been on the market for more than 20 years. The functionality of most solutions of this class is approximately the same. It includes mechanisms for data synchronization between business applications and MDM, data consolidation and cleansing, support for master data change reconciliation processes.
Data storage in MDM can be organized in different ways. Typically, MDM uses a relational DBMS to store the master records it contains. The master data structure is considered to be quite simple and stable, and the volume is considered to be relatively small, thus avoiding complex storage solutions.
However, in practice, this simplicity is often achieved by artificially narrowing the scope of MDM and simplifying the data structure, which reduces its business value. Examples can be found in two of the most popular MDM applications - customer (counterparty) data governance and product data governance.
One of the first steps in implementing MDM is data structure modeling. When trying to describe the structure of customer information, the analyst immediately faces the question: how to reflect the variety of counterparty types? They can be divided into groups by entity - for example, individuals, legal entities and individual entrepreneurs; by role - buyers, suppliers, partners. When describing individuals, each of them can act as an employee, a family member, a buyer or a user of certain products. Each particular counterparty in practice fulfills several roles at once. A natural person can register as an individual entrepreneur, and after some time stop registration, while his properties as a person (name, date of birth) remain unchanged, and properties as a sole proprietor (date of registration, registration number) appear and disappear. It is easy to see that not only the content of data on counterparties changes over time, but also the set of properties applicable to each particular counterparty.
Within the relational model, this problem can be solved in several ways: creating a single table with a large number of columns corresponding to all possible counterparty properties; splitting the counterparty information between several tables, as a result of which several DBMS records will correspond to each counterparty; using columns storing JSON collections, which actually takes the data storage model beyond the relational paradigm. All these methods have their disadvantages and complicate data governance.
When modeling product information, typical challenges are:
- The need to fit all products into the same hierarchy of product groups, which results in the mixing within the same hierarchy of categories allocated on different classification bases.
- The need to select only one main category in which each product is included - when in fact it would be correct to include the product in several categories, allocated within both one and different classification hierarchies.
- Use of various "crutch" solutions for storing property values applicable only to certain types of products.
All this complicates the work with product information and actually leads to the fact that the data in the MDM system does not adequately reflect reality. The information value of the data is impoverished, and at the same time the complexity of data governance increases, which from the business point of view significantly reduces the value of the MDM system.
An excellent way to avoid such problems is to use ontology modeling methods and technologies when creating MDM. Such products are available on the market. They perform all the standard functions of an MDM system, but offer a number of significant advantages.
| Classic MDM | MDM using ontologies |
|---|---|
| Data structuring | |
MDM inherits the limitations of the relational data warehousing model, such as the need to assign each object to only one entity type, the lack of ability to specify multiple values for each property of each object. Most often, a single category hierarchy is used to structure each directory. If an MDM-system tries to overcome these limitations, there are "crutch" solutions that significantly complicate the processes of working with data and negatively affect the speed of the system. | MDM uses an ontology-based data storage model in which:
The use of ontologies allows not to simplify the content of data, to reflect all the variety of different roles of modeled objects and points of view on them. The data structure can be easily managed. |
| Exchange with business systems in terms of model structure | |
Most often in classical MDMs there is no possibility to get data structure description via API. Moreover, there is no possibility to change it. | In ontology technologies, there is no gap between the data structure and the data itself, they are technologically homogeneous. This allows data model descriptions to be distributed to business applications on a subscription or on-demand basis. Applications can modify the data structure using APIs. This extends the scope of the MDM system by adding to it the functions of RDM (Reference Data Model) - a repository of a reference enterprise data model. |
| Temporality | |
As a rule, an MDM system stores only the current (actual) state of information objects. The history of values, if any, is available only as metadata, not as MDM data itself. It is rarely possible to work with the history of values of object properties via API. | MDM-system based on ontologies can reflect changes in classification affiliation and property values of objects both at the level of the ontology model itself (there are a number of techniques for this purpose) and at the level of metadata. In any case, it allows working through the API of the MDM-system not only with the description of the current state of business objects, but also with any previous state. This significantly increases the analytical value of MDM data. |
| Data integrity control and enrichment | |
In classical MDM, integrity control is implemented using programmed mechanisms that allow you to describe the patterns that data must conform to. At best, rule application engines such as Drools are used. | The most important feature of ontologies is the possibility of machine-readable representation of information processing rules in the ontology itself. Rules of logical integrity control (format-logical control), rules of information enrichment (logical inference) are formulated in terms of the ontology model structure and are automatically applied to all information stored in MDM. Rules are an integral part of the ontology model, allowing them to be propagated into business applications and applied on their side. |
| Support during operation | |
Business requirements are constantly changing, resulting in the need to modify the data structure. For classical MDM this can be a problem - also because the data structure is often explicitly linked to the structure of API calls, and changes in the data structure result in the need to make changes to the code of exchange adapters or integration procedures between MDM and business applications. | The structure of data in the ontology world is as easy to edit as the data itself. Information about structure changes is propagated to business applications. Integration procedures are created so that changes in the data structure do not affect them - at most, it may be necessary to set up rules in the model for mapping ontology elements to business application data schema elements. This greatly simplifies and accelerates the implementation of new business requirements. |
Thus, MDM-systems using ontological modeling technologies and methods have a number of important advantages over systems based on traditional relational data storage scheme. These advantages do not turn into disadvantages: in terms of speed and volume of stored data ontological MDM is ahead of classical systems due to the use of data virtualization technologies, absence of "compromise" solutions in structuring information. Ontological MDM is easy to extend to additional data domains and applications. Its support is much simpler and cheaper than the maintenance of cumbersome relational MDM - in fact, to perform many types of work with such products requires the participation of programmers of the vendor or integrator, while the support of ontological MDM is mainly carried out by analysts.
Let us list several projects known in the world practice on the use of ontological tools for solving MDM tasks. The American company TopQuadrant reports about implementation of the Reference Data management system in the agro-industrial sector. Metaphacts reports on the implementation of MDM at a Swiss energy company, and the implementation of its tools for product and component data governance at a German industrial company. The same vendor's solution for an international financial services provider actually serves as Customer MDM (Customer 360).
DataVera specialists are ready to provide services for implementing ontology-based MDM solutions. From software deployment and integration with data sources to model building, data cleansing, and organizational procedures for data governance, we have the expertise to handle all types of work required in the implementation process. If your company has multiple business applications that handle data about core objects such as products or customers - you need an MDM system. If you already have an MDM, but it is too expensive to support and does not solve all functional tasks - think about switching to ontology-based MDM. Such a solution can not only meet business requirements, but also become the basis for data-centric transformation of the entire IT infrastructure of the enterprise.
