Knowledge Hub

Data has become one of the most powerful words in our modern vocabulary. It drives our world, impacting decisions in industries, governments, and our daily lives. But what exactly is data? How do we define it in a way that captures its full impact and utility?

Mustafa Qizilbash
“Features and Activities of an Object is Data.”

This phrase encapsulates data’s essence. But let’s break down what it really means.

Origins and Traditional Definitions

The word "data" originates from the Latin word datum, which means "something given". This concept persists in how we perceive data today: it’s a foundational input, a starting point. In French, for example, data is referred to as données, highlighting its roots as something "given" or granted for analysis.

If you look up data in Wikipedia, you might find it defined as "individual facts, statistics, or items of information, often numeric." More formally, data is a set of values related to qualitative or quantitative variables about one or more entities or objects. In this sense, each "datum" represents one specific value or measurement.

In the world of data and databases, the term entity is foundational yet often misunderstood. To build reliable data systems, it is essential to understand what an entity is and how it is structured. This chapter explores the concept of an entity and breaks it down into its three core components: Entity Type, Entity Attributed and Entity Instance.

Let’s decode it.

What Is an Entity?

An entity is anything that can be uniquely identified and described. In data modelling, an entity typically refers to a real-world object, person, concept, or event that holds relevance within a system, such as a student, a book, or a transaction.

Entities are the fundamental building blocks of structured data models. To model and manage them effectively, we categorize entities into two levels:

• Entity Type: the blueprint or template
• Entity Attribute: the properties
• Entity Instance: the real-world, data-filled object

1. Entity Type – The Blueprint

Table, Column and Rows are one of the most basics of Data World but it’s worth explaining.

Let’s try to understand with an example, let’s say family is a table, how? Consider, you have 6 members in your family i.e., Grandfather = 1, Grandmother = 1, Father = 1, Mother = 1, Son = 1, Daughter = 1

Above is a standard average traditional family for most of us. Now, each of above members will have Name, Relationship and Date of Birth, right? These details or information are called properties and attributes of each family member and are called Columns or Fields in a table.

Now, when we say Name of the eldest person in family is John, his relationship in family is Grandfather, and his Date of Birth is 1^st January 1950. These details or information about one person in a family is called a Row.

Same way, when we have details or information about all the family members, we can say, we have multiple rows.

Primary Key is also known as Pkey, and Foreign Key is also known as Fkey. These are one of the most important terms without which a proper Data Model cannot be created.

We have understood what a table is. Now, the next step is to understand, if there is more than one table then how to join both. In the topic ‘Tables, Columns and Rows’ we used an example of table ‘Family’. Let’s use the same table here with an additional column i.e., Birth Location.

Table Name: Family

Now let’s say, to save space in Family table as we can see ‘United State of America’ has been used many times so how about creating another table where let’s store lengthy country names and in Family tables, let use country code instead with there is another table called as Birth Location.

Table Name: Birth Location

Now, let’s update the Family table to reflect country code instead of country name.

Table Name: Family

Now, the problem is column ‘Birth Location’ have country code so how to find out which country name refer to ‘001’. Let’s declare ID in Birth_Location table as Pkey and Location_ID in Family table as Fkey.

In data management, there are three fundamental data types: structured, semi-structured, and unstructured. These data types can originate from numerous sources, including databases, mobile applications, social media platforms, sensors, logs, CCTV footage, radio signals, and more. While structured and unstructured data are widely recognized, the classification of semi-structured data often leads to debate.

Structured Data

Structured data refers to information organized in a predefined format, typically consisting of rows and columns. For instance, imagine an “employee-contact-details” table containing fields such as employee code, email, mobile, LinkedIn URL, Instagram, and Twitter. Although not every employee will have complete details, the table structure remains intact when exported, with each row containing a set number of columns, even if some fields are empty. For example, a dataset with 6 columns and 10 rows will consistently display this format, regardless of missing data entries.

Good data in the data world possesses several essential qualities that contribute to its reliability and effectiveness. Firstly, it is accurate, ensuring that the information it represents aligns with the real-world context it intends to portray, free from errors or inconsistencies. Completeness is another hallmark of good data, implying that it includes all necessary details without any missing or incomplete records. Consistency is maintained throughout, following standardized formats and structures, facilitating uniformity and coherence. Relevance ensures that the data is directly applicable to its intended purpose, meeting the specific goals of data analysis or application. Timeliness is a key aspect, as good data is up to date, reflecting the most recent and relevant information. Reliability is associated with trustworthiness, ensuring the data is unbiased and collected through reliable methods. Validity is maintained by adhering to predefined rules and standards, and accessibility ensures that authorized users can easily retrieve and utilize the data.

If you have read Edition 2 you know what Data what Metadata is. Now the big question is, are both must-to-have or there can be a possibility to have one only.

Questions:

• Do we need Data when we have Metadata?
  • My response is, No, we don’t.
• In other words, can there be Data with Metadata?
• Again, my response is, No.
  • As one of friend Daniel Lundin said, ‘One person Metadata is Data for another person’.

Exciting topic right….. Yes, it is, as we know size of data is getting huge to humongous lolz 😊.

Let’s decode it…..

Here, we are only referring to Raw Data i.e., which is coming from source systems e.g., OLTP applications or social media or unstructured datasets like video, audio, PDF, email etc., into next datastores e.g., Data Lake or in Data Warehouse etc.

So, when it’s out in the next storage, do we need to keep raw data for life or for certain time e.g., to fulfill regulatory requirements or we can purge it once Metadata has been extracted?

Question: Can there be Data with Metadata?

As discussed in a different context, specifically in the comparison between Data and Metadata, my friend Daniel Lundin aptly pointed out that 'One person's Metadata is another person's Data.' This statement underscores the necessity for a profound grasp of the fundamental principles of Data.

Let's delve into the interpretation...

In the discussion on 'Data vs Metadata,' as illustrated in the accompanying image, our attempt was to delineate the distinction between Data and Metadata. To simplify comprehension, we labeled Column Names as Metadata and the actual content as Data.

Let's delve deeper and explore the question: Can there be Data without Metadata? Disclaimer: In my understanding, the answer is NO; it's not possible. Before we delve into further details, let me quickly remind you that traditionally, there are three types of Metadata: Business, Technical, and Operational. The initial image in this discussion illustrates Technical Metadata in the form of Column Names.

Data Modelling is the practice of structuring, describing, and organizing information so it reflects real-world entities, relationships, rules, and contexts in a way that supports business operations, analytics, governance, and AI (Artificial Intelligence).

Most of us, are aware of only few Data Modelling techniques, but in reality, there are around 37 of those, shown in below diagram.

Data Modelling Concepts & Techniques

I will try not to go into details in this topic, as we will have separate topics for each of it. First, one must understand the concept of Normalization and Denormalization to get a grasp of data modelling world.

De-normalization: Surely, all of us has worked on Excel. Excel has columns and rows, as do tables in Databases. Consider having employee details in Excel with columns like employee ID, name, mobile number, address, salary, department etc. All this information related to one employee can be stored in one row which is a Denormalization data model.

A Fact is the central table in dimensional data modelling that stores the measurable outcomes of a business process. These measures are typically numeric and are designed to be aggregated, analysed, and reported across multiple perspectives provided by dimension tables.

The primary goal of a fact table is to capture business events in a structured format that supports performance analysis and decision-making. Facts answer the question: “What happened?” They represent transactions, events, or measurable states that the organization wants to analyse.

A fact table typically contains:

• Numeric measures (e.g., LoanAmount, ApprovalCount, RepaymentAmount)
• Foreign keys to dimension tables (e.g., Customer_ID, Officer_ID, Branch_ID, Time_ID)
• Minimal descriptive attributes of its own

For example, in a loan approval environment, a Fact_LoanApproval table may contain:

• Loan_ID (Primary Key)
• Customer_ID (Foreign Key)
• Officer_ID (Foreign Key)
• Branch_ID (Foreign Key)
• Time_ID (Foreign Key)
• LoanAmount
• InterestRate
• ApprovalFlag