Data Harmonization

Data Harmonization is normally confused with Data Integration, Data Blending, Data Warehousing, Data Marts etc. Yes, all do the same steps of connecting and bring all required data from all the disparate sources in one central location, but Data Harmonization goes one step ahead.

Data Harmonization standardizes data coming from multiple source systems in one unified format, so it becomes a bridge between data and business requirement. When we say, one unified format, it means data is converted in such a way that it can become accurate for someone.

Data Harmonization can be of many types.

• Harmonizing data in one location for a specific department can be called a Data Mart.
• Defining standard naming conventions, data type, codes coming from multiple sources in one unique format can be Harmonizing data for Master Data Management.
• Gathering Metadata details of all source systems, data lakes, data warehouses, data marts up to reports and dashboards is Harmonizing data for Data Lineage.

Ideal steps to conduct Data Harmonization is to

• Understand the vision of business requirements like once you convert data, it will be used for what purpose.
• Set the baseline i.e., identifying all the relevant source systems, tables and even columns from where data will come.
• Involvement of business stakeholders is a must to have step as without functional knowledge garbage will flow in.
• Define standard processes and actions like ETL or ELT or Manual steps etc.
• Regular reconciliation and monitoring are also a key step to not compromising data quality.
• Define metadata at each step of the harmonization process.

Marketing department, if don’t have Harmonized data can perform poorly. For Example, marketing department heavily depends on product name, product codes, product sales locations etc. If all relevant details are not Harmonized, means, if a marketing department don’t have information like which products are sold in which location, on what date, at what time, by male or female, was the buyer an adult or a kid, was there any event going on at the point-in-time, was there any public holiday etc., all these information help marketing department to plan accurate marketing campaigns, mobile message, marketing events etc.

Please note the above highlighted details might not be relevant for a Finance department. For Finance department, Data Harmonization steps or required details will be totally different.