Knowledge Hub

DIKW model

‘Data --> Information --> Knowledge --> Wisdom’

Generally, Data, Information, Knowledge, and Wisdom are arranged in the above sequence. How about considering another angle and trying to arrange these 4 terms in different sequence?

For me Information and Knowledge both comes from Data and Wisdom comes from both Information and Knowledge.

Difference between Data and Information?

As we all know, now a days, Data is everywhere either it is generating from System/ Mobile Applications, Online Websites/ Portals, Videos, Audios, logs etc.

• Is all Data equal to Information? No, it’s NOT.
• Can all Data become Information? Yes, it CAN, based on the nature of the business or requirement.

To answer above, one must first understand what Information is. Information is tightly coupled with what one wants to know. You must have heard this question on many occasions or events or counters etc. like What Information do you need?

Monetization is about generating benefit/ revenue/ profit from anything.

‘Data Monetization is about generating revenue from Data.’

In this era, social media is a very dominant channel to interact with customers. Platforms like WhatsApp, Facebook, Twitter, TikTok, Instagram etc. all provide free services to their customers. Have you ever noticed, if you are not paying a single penny to them then how they are generating money and how they are running their business?

The response to the above is by Data Monetization. They sell the data that’s generated by their customers. Having said that, please note, there are regulations that they can’t share personal details of their customers with anyone. But that doesn’t mean they can’t sell how many people are viewing what, how many are buying, how many are spending how much on which product category, from which location they are logged in etc.

For example, on Amazon, customers can buy anything they like. Now at the back end, Amazon must be recording what you are buying and what you are selling. Based on this Information, they can very well analyze which product is more viewed on their portal or what is the buying trend of a particular set of products in a particular season. Like, in December, they can see the trend of products related to Christmas are in great demand. They can sell information like WHICH & HOW MANY PRODUCTS WILL BE IN DEMAND IN COMING DECEMBER based on last few years trend to the product owner to plan their production. This information is GOLD for product owners, they can plan their raw material, work force, timeline and many more activities.

Point-to-Point model (PP) connects all applications with each other e.g., if there are 5 applications then there will be many connections as depicted in below picture.

The Hub-and-Spoke model (SP) creates a new centralized connection points called as Hub and all applications will ONLY connect with centralized ONE Hub.

We can clearly see Hub-and-Spoke model is much better in access management, flexible, cheaper, secure etc. as compared to Point-to-Point model.

Examples of Hub-and-Spoke model are Data Warehouse (dimensional), Data Hub, Data Fabric etc. which serves as single point of truth for all enterprise users.

Key Items

• User Access: SP model can management user access matrix at centralized place as compared to PP model.
• Security: As all users will be connecting to one centralized Hub, risk of security breach will be less in SP model as compared to PP model
• Flexibility: In SP model, Hub is easy to incorporate those new requirements which doesn’t need to implement in source systems.
• Cheaper: In SP model, though Hub implementation requires initial investment but once it’s implemented all new requirements cost will have much cheaper impact as all changes will go in Hub as compared to each application in PP model.

This is a principle rather a process to manipulate data, but it holds a very significant value in terms of managing and securing data.

Most of us mix Critical Data Elements with elements or data points identified during Data Quality exercise. Though those are also critical but when we talk about ‘Critical Data Element’ in Data Strategy, we categorize those elements which can have negative impact on the organization.

Critical Data Element can be any regulatory requirements, any legal or compliance requirements, those data elements which can negatively impact the image of company in general public, those data elements which if are found inaccurate make stakeholders’ loss their confidence on data.

Personal Information of customers is one of the most critical data elements in most of the organization. Credit or Debit Card details in the Banking sector are critical. Location details in Telecommunication are critical etc.

This is also a very confusing term in Data domain, a lot of people are not sure how to assure that everyone follows Ethics in their day-to-day activities while interacting with Data. Just like CDE (Critical Data Element), Data Ethics is also a principle and must be managed at organization level and followed at individual level.

Now, the big question is which data falls under the ambarella where Data Ethics must be followed. There are multiple factors involved and everyone has his/ her own level of ethics to handle based on their roles and responsibilities.

Data Ethics is also a term which if not taken care of properly, can damage a company’s image with the general public. To make sure organizations follow ethics, different regulatory bodies in different parts of the world and in different sectors give guiding principles to follow. It doesn’t stop here, if the organization doesn’t end up following all the measures asked-for by those regulatory bodies then they can be penalized with heavy amounts as penalties.

Data Anomalies are about any kind of unusual behavior which can degrade the quality of data. Please note, data quality is not only about dirty data but also about not being able to insert, update and delete required data.

Type of Data Anomalies

• Insertion: If one is unable to enter quality data in database.
• Update: If one is unable to update relevant data or able to update irrelevant data.
• Delete: If one is unable to delete relevant data or able to delete irrelevant data.
• Redundancy: If there are duplicate records in a table.

Data Normalization and database constraints play a vibrant role to make sure data anomalies doesn’t occur. To start with, let’s say there is a table with employee ID, name, department, and salary details. First wrong approach we can noticed i.e., this table is in de-normalized form, by right there should be separate tables for department and salary data using normalization process. Normalization and Denormalization are covered, in detail, in data modelling topic.

Data Surfing is always confused with Data Analysis. There is a very thin line between Data Analysts and Data Surfers.

Data Surfing is normally done by non-technical people like management who doesn’t know how to write any kind of code to access data at database level.

Data Surfers normally stays at the level of Dashboards and Reports where they just click here and there to explore data for required information.

Business Intelligence (BI) tools came into being to cater Data Surfers World where data was required to be presented for non-technical people who could view data for decision making.

Era(s) for Data Surfers

• Static Reports: In the beginning, business users used to share their data requirements with IT who then used to create dashboards and reports. Those dashboards/ reports and data was not changeable, it was in paper format or in excel format or in text format etc., means after Go-Live if business want one more column or if latest data was required then the whole development cycle had to restart from scratch.
• Dynamic Reports: In the second era, BI tools came-in to cater for this gap. To start with, dashboards/ reports were still not changeable, but data was directly connected with backend databases, so data was always latest at front-end. This was a huge jump, as now, to view new data developer don’t have to recreate front-end again and again.
• Adhoc Reports: Latest era is where dashboards/ reports are also changeable. Now, end users and management can drag and drop tables and columns in dashboards/ reports.

“Semantic” is an ancient Greek word: σημαντικός sēmantikós, "significant" is the study of reference, meaning, or truth.

In Data World, Semantic layer is built for the sole purpose of supporting business users.

Semantic layer is to convert complex data models into simple and user-friendly interface for business users.

A semantic layer is the representation layer on top of databases, schema(s), tables and columns in such a way that Business users who have no technical know-how should be able to drag and drop required details on Dashboards. For drag and drop, first, one must assign business names to all databases, schema(s), tables and columns before exposing in BI tools. For example, there can be a column name as Date but the data in this column can be Date_Of_Birth or Transaction Date or Financial_Year_End_Data. Converting data models, column/ table/ schema/ database names as per business understanding is semantic process.

By now, we all know what Semantic Layer is, IT defines data elements for business ease. Then there came the era for Business folks defines data elements themselves based on their own understanding and now the latest approach is going towards Machine Learning Models define data element based on open-ended requirements.

A major difference between first two approaches vs Augmented Analytics is, first two were derived by humans either by IT or Business folks so there is always a factor of human era and at the same time human can be biased base on knowledge and information they possess about the business domain. On the other hand, Augmented Analytics is based on automated Machine Learning which learns the data elements while new data sets keep coming in.

The first approach was static as requirements were always coming from business and implemented by IT folks, so any change had a round robin effect.

Data Island has kind of disappeared as integration technologies have matured.

‘Data Island is a datastore(s) which is/ are not accessible.’

In previous eras when network security was a big concern or even before, when network connectivity was not even there, the datastores were considered as Data Island as external integration was not possible.

Let’s try to understand with examples. We all know WIFI has become a norm in our lives and living without it is like living without Air right? In past when we used to visit a friend’s place we used to greet first, now we first ask, ‘What’s your WIFI Password’ lolz 😊?’. But if you are as old as I am, before 2005 WIFI was not a thing. There were ONLY cables which used to connect our desktops or laptops with network, well I still prefer cable. During that era consider a datastore sitting in different city or region then how to connect? In that situation, we had floppy drives for small data transfer (yeah, I won’t explain what is floppy drive lolz 😊, old folks know what I am talking about. Let’s let newbies find out what is floppy drive), and for huge data we used to have big hard disks and sometimes desktops itself were moved to move data from one location to another location. This datastore, which was not connectable with other datastore, was call Data Island.