Data Validation

Data Validation is about verifying or validating the correctness and quality of data against the source. In other words, reconciliation of imported and processed data with its source data.

Data Validation can be done before pulling data and as well after data is processed but must be done before passing data to end users.

Sometime, data changes its shape after transformation from source to target, in this situation Data Validation is conducted against business rules implemented during transformation.

Types of Data Validation

• Constraints Validation checks the accuracy of all types of constraints like primary key, foreign keys, unique, not null, values check and query performance constraint.
• Structured Validation checks the structures of tables and columns.
• Data Range Validation checks the allowable data range to be allowed for processing e.g., if only latest data is allowed to be process the only latest data should get processed.
• Code Validation checks the quality of the syntax. There are ways to write quality and optimized code.
• Data Type Validation checks the type of data to be process e.g., if a data type of a column is Date then only data format with date values should be processed.