Knowledge Hub

Data has become one of the most powerful words in our modern vocabulary. It drives our world, impacting decisions in industries, governments, and our daily lives. But what exactly is data? How do we define it in a way that captures its full impact and utility?

Mustafa Qizilbash
“Features and Activities of an Object is Data.”

This phrase encapsulates data’s essence. But let’s break down what it really means.

Origins and Traditional Definitions

The word "data" originates from the Latin word datum, which means "something given". This concept persists in how we perceive data today: it’s a foundational input, a starting point. In French, for example, data is referred to as données, highlighting its roots as something "given" or granted for analysis.

If you look up data in Wikipedia, you might find it defined as "individual facts, statistics, or items of information, often numeric." More formally, data is a set of values related to qualitative or quantitative variables about one or more entities or objects. In this sense, each "datum" represents one specific value or measurement.

In the world of data and databases, the term entity is foundational yet often misunderstood. To build reliable data systems, it is essential to understand what an entity is and how it is structured. This chapter explores the concept of an entity and breaks it down into its three core components: Entity Type, Entity Attributed and Entity Instance.

Let’s decode it.

What Is an Entity?

An entity is anything that can be uniquely identified and described. In data modelling, an entity typically refers to a real-world object, person, concept, or event that holds relevance within a system, such as a student, a book, or a transaction.

Entities are the fundamental building blocks of structured data models. To model and manage them effectively, we categorize entities into two levels:

• Entity Type: the blueprint or template
• Entity Attribute: the properties
• Entity Instance: the real-world, data-filled object

1. Entity Type – The Blueprint

Table, Column and Rows are one of the most basics of Data World but it’s worth explaining.

Let’s try to understand with an example, let’s say family is a table, how? Consider, you have 6 members in your family i.e., Grandfather = 1, Grandmother = 1, Father = 1, Mother = 1, Son = 1, Daughter = 1

Above is a standard average traditional family for most of us. Now, each of above members will have Name, Relationship and Date of Birth, right? These details or information are called properties and attributes of each family member and are called Columns or Fields in a table.

Now, when we say Name of the eldest person in family is John, his relationship in family is Grandfather, and his Date of Birth is 1^st January 1950. These details or information about one person in a family is called a Row.

Same way, when we have details or information about all the family members, we can say, we have multiple rows.

Primary Key is also known as Pkey, and Foreign Key is also known as Fkey. These are one of the most important terms without which a proper Data Model cannot be created.

We have understood what a table is. Now, the next step is to understand, if there is more than one table then how to join both. In the topic ‘Tables, Columns and Rows’ we used an example of table ‘Family’. Let’s use the same table here with an additional column i.e., Birth Location.

Table Name: Family

Now let’s say, to save space in Family table as we can see ‘United State of America’ has been used many times so how about creating another table where let’s store lengthy country names and in Family tables, let use country code instead with there is another table called as Birth Location.

Table Name: Birth Location

Now, let’s update the Family table to reflect country code instead of country name.

Table Name: Family

Now, the problem is column ‘Birth Location’ have country code so how to find out which country name refer to ‘001’. Let’s declare ID in Birth_Location table as Pkey and Location_ID in Family table as Fkey.

In data management, there are three fundamental data types: structured, semi-structured, and unstructured. These data types can originate from numerous sources, including databases, mobile applications, social media platforms, sensors, logs, CCTV footage, radio signals, and more. While structured and unstructured data are widely recognized, the classification of semi-structured data often leads to debate.

Structured Data

Structured data refers to information organized in a predefined format, typically consisting of rows and columns. For instance, imagine an “employee-contact-details” table containing fields such as employee code, email, mobile, LinkedIn URL, Instagram, and Twitter. Although not every employee will have complete details, the table structure remains intact when exported, with each row containing a set number of columns, even if some fields are empty. For example, a dataset with 6 columns and 10 rows will consistently display this format, regardless of missing data entries.

Good data in the data world possesses several essential qualities that contribute to its reliability and effectiveness. Firstly, it is accurate, ensuring that the information it represents aligns with the real-world context it intends to portray, free from errors or inconsistencies. Completeness is another hallmark of good data, implying that it includes all necessary details without any missing or incomplete records. Consistency is maintained throughout, following standardized formats and structures, facilitating uniformity and coherence. Relevance ensures that the data is directly applicable to its intended purpose, meeting the specific goals of data analysis or application. Timeliness is a key aspect, as good data is up to date, reflecting the most recent and relevant information. Reliability is associated with trustworthiness, ensuring the data is unbiased and collected through reliable methods. Validity is maintained by adhering to predefined rules and standards, and accessibility ensures that authorized users can easily retrieve and utilize the data.

If you have read Edition 2 you know what Data what Metadata is. Now the big question is, are both must-to-have or there can be a possibility to have one only.

Questions:

• Do we need Data when we have Metadata?
  • My response is, No, we don’t.
• In other words, can there be Data with Metadata?
• Again, my response is, No.
  • As one of friend Daniel Lundin said, ‘One person Metadata is Data for another person’.

Exciting topic right….. Yes, it is, as we know size of data is getting huge to humongous lolz 😊.

Let’s decode it…..

Here, we are only referring to Raw Data i.e., which is coming from source systems e.g., OLTP applications or social media or unstructured datasets like video, audio, PDF, email etc., into next datastores e.g., Data Lake or in Data Warehouse etc.

So, when it’s out in the next storage, do we need to keep raw data for life or for certain time e.g., to fulfill regulatory requirements or we can purge it once Metadata has been extracted?

‘Intuition is the ability to understand something without any need for conscious reasoning.’

Intuition is the ability to acquire knowledge without conscious reasoning. It includes but is not limited to direct access to unconscious knowledge, and the ability to understand something instinctively, without any need for conscious reasoning.

Now, how does it relate to Data? I would say, Data Intuition is one of the most important capabilities any data analyst, data citizen, data scientist and management should have. Yes, decisions can be taken based on data, but one should not take a decision, if his/ her intuition is not aligned.

For example, there are sales of 200k drinking bottles per month, expected from a particular area. But suddenly, 500k bottles were sold, NOW your data intuition should shake you about the result. Now you can start asking WHY’s, till you have all the evidence of the sales. It could be a fake result just to show higher sales to get a bonus 😊.

DIKW model

‘Data --> Information --> Knowledge --> Wisdom’

Generally, Data, Information, Knowledge, and Wisdom are arranged in the above sequence. How about considering another angle and trying to arrange these 4 terms in different sequence?

For me Information and Knowledge both comes from Data and Wisdom comes from both Information and Knowledge.

Difference between Data and Information?

As we all know, now a days, Data is everywhere either it is generating from System/ Mobile Applications, Online Websites/ Portals, Videos, Audios, logs etc.

• Is all Data equal to Information? No, it’s NOT.
• Can all Data become Information? Yes, it CAN, based on the nature of the business or requirement.

To answer above, one must first understand what Information is. Information is tightly coupled with what one wants to know. You must have heard this question on many occasions or events or counters etc. like What Information do you need?

Data Surfing is always confused with Data Analysis. There is a very thin line between Data Analysts and Data Surfers.

Data Surfing is normally done by non-technical people like management who doesn’t know how to write any kind of code to access data at database level.

Data Surfers normally stays at the level of Dashboards and Reports where they just click here and there to explore data for required information.

Business Intelligence (BI) tools came into being to cater Data Surfers World where data was required to be presented for non-technical people who could view data for decision making.

Era(s) for Data Surfers

• Static Reports: In the beginning, business users used to share their data requirements with IT who then used to create dashboards and reports. Those dashboards/ reports and data was not changeable, it was in paper format or in excel format or in text format etc., means after Go-Live if business want one more column or if latest data was required then the whole development cycle had to restart from scratch.
• Dynamic Reports: In the second era, BI tools came-in to cater for this gap. To start with, dashboards/ reports were still not changeable, but data was directly connected with backend databases, so data was always latest at front-end. This was a huge jump, as now, to view new data developer don’t have to recreate front-end again and again.
• Adhoc Reports: Latest era is where dashboards/ reports are also changeable. Now, end users and management can drag and drop tables and columns in dashboards/ reports.