Data Scrubbing,Cleansing andCleaning

All these terms like Data Integrity, Data Quality, Data Scrubbing, Data Cleansing and Data Cleaning are very interrelated.

• Cleaning: You eat something, and you get something on your face, you clean your face. You are painting something, and you get splashes of paint on your face, you clean your face. You are walking and you feel the dust, you wash and clean your face.
  SAME WAY, data cleaning can be done at a very high level by Business Users themselves as it requires little effort e.g., export data in excel, clean it, and send it downstream.
• Cleansing: You went on safari or played a football match on muddy ground, now you must use some face cream to cleanse your face plus you have to spend 10-15 minutes. Then it is better to ask someone or ideally go to the parlor for good face cleansing.
  SAME WAY, data cleansing can be done by business users themselves but ideally IT SME(s) can write SQL queries at the database level to cleanse data so next time when you extract, you get the right cleansed data.
• Scrubbing: This is the best level to achieve the highest Data Quality. Coming back to the same example, you need to attend a wedding or birthday party or any kind of valuable event, now you will go to a proper parlor, you will find the best worker, you will select the best brand of creams (based on your pocket vs the event you are attending) to scrub your face skin. You will make sure your face has been properly cleaned, cleansed, and scrubbed, and you will make sure you look ready for the event.
  SAME WAY, data scrubbing is to make sure your data is near to perfect, you will find tools that can do this for you, you will find good resources for this task, you will make sure the findings are implemented from Source to Downstream systems.

There can be multiple reasons to conduct Data Scrubbing i.e., wrong data entry, when two or more databases are merged, there are no, or a smaller number of standards defined in the system or very old data which is not obsolete. During this exercise, corrupt and bad data is removed from the mainstream of data and parked separately for investigation.

As per the survey, in any given hour of the day, almost five dozen companies will change their addresses, nearly a dozen will change their name, and over 40 new businesses will open.

And as per statistics,

• Businesses lose up to 20% of their revenue because of bad data quality.
• Employees waste up to half of their productive time dealing with routine data quality tasks