Process-Centric Data Quality Services and MDM
For the past few years, master data management (MDM) has been a hot topic among enterprise architects and data management professionals. But many of these evangelists have felt like rebels without a cause because their MDM initiatives have been primarily IT driven with minimal business sponsorship and participation. In other words, the business expects high quality data, but hasn’t taken much accountability in delivering it.
Data quality (DQ) management, a more mature technology and competency than MDM - and a required core capability for any MDM solution - has also struggled to be embraced by the business as a top priority. More often than not, DQ has been relegated as a supporting downstream batch application for ETL, data warehousing and business intelligence applications, but not for a broader cross-enterprise data architecture including upstream transactional systems and processes.
But Forrester now sees this trend changing for the better, as highlighted by my colleague, Alex Cullen, who recently published important research on “The Top 15 Technology Trends EA Should Watch.” A key theme Alex identifies is around Process-Centric Data and Intelligence. Alex believes - and I agree with him - that both MDM and real time data quality services will become critical enablers to support real-time operational and decision-making processes.
Organizational competency in the definition and implementation of DQ rules, processes and technologies that can effectively cleanse, standardize, enrich, match, and merge critical data is a foundational step to moving towards true MDM: the real-time delivery and synchronization of a single, trusted view of enterprise reference data to both operational and analytical environments. Why? Because you can’t bi-directionally synchronize master data if you haven’t implemented the DQ rules required to derive a master record.
Unfortunately in many instances the after-the-fact batch cleansing of data in a data warehouse is often too little too late in ensuring high quality data can be delivered via MDM or otherwise. This is why we see real-time DQ Services as a critical evolution for data architects to embrace and evangelize: embed DQ validation and cleansing logic within the upstream transactional processes that capture and maintain this data in the first place. Not only will this of course improve the trustworthiness of the data used for analysis later, but even more transformational – it will improve the efficiency and quality of the upstream transactional processes themselves.
A simple example:
A “Ship To” postal address is typically captured as part of an order management process. Before real-time DQ Services are implemented to ensure that the addresses captured meet formatting standards and are in fact deliverable, order fulfillment runs a very real risk of being delayed – leading to potential postage penalties and negative customer satisfaction due to the delays. But DQ Service can significantly reduce this risk by recognizing these DQ issue while the customer is still engaged in the process and best able to respond.
As further evidence of this trend toward transactional data trustworthiness, in my just-published “Trends 2009: Master Data Management” research I share the results of a recent Forrester MDM readership survey where over 64% of respondents said their planned or implemented MDM solution must support real-time subscription requirements, and not just scheduled batch loads. Therefore it’s not too surprising that almost 79% of those surveyed that have decided upon an architectural deployment style for their MDM solution selected a Hub, SOA, Federated, or Registry approach since all of these are optimized to support real-time, transactional MDM requirements.
But don’t expect this transformation to real-time DQ and MD to happen overnight. The complexity and invasiveness of these data services to the streamlined transactions that are managed make many EA’s wary, especially since most organizations still have extremely immature data governance programs. In the same readership survey, nearly 46% of companies rate their data governance maturity as low or very low, while another 29% rate their maturity average, seeing significant participation from just a few key business stakeholders. Data governance – an organizational and cultural shift – is the first trend EA’s must follow and evangelize in order to deliver the process-centric data and intelligence their organizations are demanding.
Related Post
