Knowledge Hub

Point-to-Point model (PP) connects all applications with each other e.g., if there are 5 applications then there will be many connections as depicted in below picture.

The Hub-and-Spoke model (SP) creates a new centralized connection points called as Hub and all applications will ONLY connect with centralized ONE Hub.

We can clearly see Hub-and-Spoke model is much better in access management, flexible, cheaper, secure etc. as compared to Point-to-Point model.

Examples of Hub-and-Spoke model are Data Warehouse (dimensional), Data Hub, Data Fabric etc. which serves as single point of truth for all enterprise users.

Key Items

• User Access: SP model can management user access matrix at centralized place as compared to PP model.
• Security: As all users will be connecting to one centralized Hub, risk of security breach will be less in SP model as compared to PP model
• Flexibility: In SP model, Hub is easy to incorporate those new requirements which doesn’t need to implement in source systems.
• Cheaper: In SP model, though Hub implementation requires initial investment but once it’s implemented all new requirements cost will have much cheaper impact as all changes will go in Hub as compared to each application in PP model.

This is a principle rather a process to manipulate data, but it holds a very significant value in terms of managing and securing data.

Most of us mix Critical Data Elements with elements or data points identified during Data Quality exercise. Though those are also critical but when we talk about ‘Critical Data Element’ in Data Strategy, we categorize those elements which can have negative impact on the organization.

Critical Data Element can be any regulatory requirements, any legal or compliance requirements, those data elements which can negatively impact the image of company in general public, those data elements which if are found inaccurate make stakeholders’ loss their confidence on data.

Personal Information of customers is one of the most critical data elements in most of the organization. Credit or Debit Card details in the Banking sector are critical. Location details in Telecommunication are critical etc.

This is also a very confusing term in Data domain, a lot of people are not sure how to assure that everyone follows Ethics in their day-to-day activities while interacting with Data. Just like CDE (Critical Data Element), Data Ethics is also a principle and must be managed at organization level and followed at individual level.

Now, the big question is which data falls under the ambarella where Data Ethics must be followed. There are multiple factors involved and everyone has his/ her own level of ethics to handle based on their roles and responsibilities.

Data Ethics is also a term which if not taken care of properly, can damage a company’s image with the general public. To make sure organizations follow ethics, different regulatory bodies in different parts of the world and in different sectors give guiding principles to follow. It doesn’t stop here, if the organization doesn’t end up following all the measures asked-for by those regulatory bodies then they can be penalized with heavy amounts as penalties.

Data Anomalies are about any kind of unusual behavior which can degrade the quality of data. Please note, data quality is not only about dirty data but also about not being able to insert, update and delete required data.

Type of Data Anomalies

• Insertion: If one is unable to enter quality data in database.
• Update: If one is unable to update relevant data or able to update irrelevant data.
• Delete: If one is unable to delete relevant data or able to delete irrelevant data.
• Redundancy: If there are duplicate records in a table.

Data Normalization and database constraints play a vibrant role to make sure data anomalies doesn’t occur. To start with, let’s say there is a table with employee ID, name, department, and salary details. First wrong approach we can noticed i.e., this table is in de-normalized form, by right there should be separate tables for department and salary data using normalization process. Normalization and Denormalization are covered, in detail, in data modelling topic.

Data Surfing is always confused with Data Analysis. There is a very thin line between Data Analysts and Data Surfers.

Data Surfing is normally done by non-technical people like management who doesn’t know how to write any kind of code to access data at database level.

Data Surfers normally stays at the level of Dashboards and Reports where they just click here and there to explore data for required information.

Business Intelligence (BI) tools came into being to cater Data Surfers World where data was required to be presented for non-technical people who could view data for decision making.

Era(s) for Data Surfers

• Static Reports: In the beginning, business users used to share their data requirements with IT who then used to create dashboards and reports. Those dashboards/ reports and data was not changeable, it was in paper format or in excel format or in text format etc., means after Go-Live if business want one more column or if latest data was required then the whole development cycle had to restart from scratch.
• Dynamic Reports: In the second era, BI tools came-in to cater for this gap. To start with, dashboards/ reports were still not changeable, but data was directly connected with backend databases, so data was always latest at front-end. This was a huge jump, as now, to view new data developer don’t have to recreate front-end again and again.
• Adhoc Reports: Latest era is where dashboards/ reports are also changeable. Now, end users and management can drag and drop tables and columns in dashboards/ reports.

“Semantic” is an ancient Greek word: σημαντικός sēmantikós, "significant" is the study of reference, meaning, or truth.

In Data World, Semantic layer is built for the sole purpose of supporting business users.

Semantic layer is to convert complex data models into simple and user-friendly interface for business users.

A semantic layer is the representation layer on top of databases, schema(s), tables and columns in such a way that Business users who have no technical know-how should be able to drag and drop required details on Dashboards. For drag and drop, first, one must assign business names to all databases, schema(s), tables and columns before exposing in BI tools. For example, there can be a column name as Date but the data in this column can be Date_Of_Birth or Transaction Date or Financial_Year_End_Data. Converting data models, column/ table/ schema/ database names as per business understanding is semantic process.

By now, we all know what Semantic Layer is, IT defines data elements for business ease. Then there came the era for Business folks defines data elements themselves based on their own understanding and now the latest approach is going towards Machine Learning Models define data element based on open-ended requirements.

A major difference between first two approaches vs Augmented Analytics is, first two were derived by humans either by IT or Business folks so there is always a factor of human era and at the same time human can be biased base on knowledge and information they possess about the business domain. On the other hand, Augmented Analytics is based on automated Machine Learning which learns the data elements while new data sets keep coming in.

The first approach was static as requirements were always coming from business and implemented by IT folks, so any change had a round robin effect.

Data Island has kind of disappeared as integration technologies have matured.

‘Data Island is a datastore(s) which is/ are not accessible.’

In previous eras when network security was a big concern or even before, when network connectivity was not even there, the datastores were considered as Data Island as external integration was not possible.

Let’s try to understand with examples. We all know WIFI has become a norm in our lives and living without it is like living without Air right? In past when we used to visit a friend’s place we used to greet first, now we first ask, ‘What’s your WIFI Password’ lolz 😊?’. But if you are as old as I am, before 2005 WIFI was not a thing. There were ONLY cables which used to connect our desktops or laptops with network, well I still prefer cable. During that era consider a datastore sitting in different city or region then how to connect? In that situation, we had floppy drives for small data transfer (yeah, I won’t explain what is floppy drive lolz 😊, old folks know what I am talking about. Let’s let newbies find out what is floppy drive), and for huge data we used to have big hard disks and sometimes desktops itself were moved to move data from one location to another location. This datastore, which was not connectable with other datastore, was call Data Island.

Data Silos is always confused with Data Island.

Data Islands are those datastores which are not accessible from external world may be due technology barriers, but Data Silos are those which are only accessible for their relevant stakeholders.

There can be a lot of reasons why data is in silo.

• A particular datastore might be hosting confidential data e.g., finance data, audit data etc.
• A particular data which is not readable e.g., semi or unstructured data sets.
• There is Security or Privacy concerns, so data is not shared among another departments.
• The quality of data is so low that it can’t be shared with management as it may cause wrong decisions and may damage the company’s image.
• Sometime, in development environment different database instances are created for different development teams so if one changes data or table’s structures, other development teams are not impacted.

Why should Data Silos not exist?

There are many ways to achieve data privacy like data generalization, data swapping, data masking, data encryption etc., all these techniques change original data which of course is not desired by most. All these are done to ensure data should not fall into wrong hands.

Data Synthetic is something which will really solve the problem statement like anonymize data before sharing with developers, data science teams for data models etc.

Data Synthetic is about generating new dataset using computer simulations. This dataset contains fake data with the right properties or attributes. For example, customer information is required in almost every application or system, but it can’t be shared to protect individual identity. Data Synthetic tools create a dummy dataset for customers with name, email, contact number, family details etc. but nothing related to original data. This way, developers can build the data pipelines and data scientists can build their data models without thinking of breaching any regulatory or compliance clauses.