Data Provenance

Share this topic

Data Provenance

From the book I Am Datapedia!, published here free by the author.

Part of I Am Datapedia!

Data Provenance and Data Lineage. Logically both do the same thing but at different levels of granularity.

Diagram

Description automatically generatedData Lineage is about knowing the Data Lifecycle but with respect to databases, tables, columns, business rule/ ETL rule etc. For example, data comes from which database, table, column and went into which database, table, column with what business rules incorporated during transformation process.

Data Provenance is at a higher level, it’s about Data Lifecycle with respect to inputs, entities, systems, and processes. For example, data comes in which format, from which source system, its related to which entity, what is the process used like real-time processing, batch processing or event based etc., any analytical modelling done, any data enrichment took place, any data cleaning, data quality rules, metadata management, master & reference data management etc., took place during the data lifecycle.

‘Data Lineage is mostly used by IT folks whereas Data Provenance is used by Business Users or Management.’

Express your reaction

Your Take on This Insight?

Share how this content resonates with you.

Comment
Useful0
Insightful0
Actionable0
Thought-provoking0
Inspiring0

0 reactionsSign in to react or comment

Comments (0)

No comments yet.

    Free account

    You can read every topic without one. A free account lets you react to this topic, comment under it and search the Knowledge Hub. Reacting and commenting need a confirmed e-mail address.