Knowledge Hub

We all have heard about Mean and Median values but not Spread.

Before jumping into Spread, let's have a quick definition of Mean and Median which will assist in understanding Spread.

• Mean: The mean (informally, the “average “) is found by adding all the numbers together and dividing by the number of items in the set: (10 + 10 + 20 + 40 + 70) / 5 = 30.
• Median: The median is found by ordering the set from lowest to highest and finding the exact middle. The median is just the middle number: 20.
• Spread: The simple explanation of Spread is e.g., how variable the data is or how spread out the data values are in the dataset, or it is also referred to as a measure of dispersion etc., from the Mean or Median values.

Let’s try to understand with an example. Let’s say we have a dataset with the customers’ age from 20 to 60.

Breakdown

• 8 customers with Age [10]
• 15 customers with Age [20]
• 12 customers with Age [30]
• 7 customers with Age [40]
• 3 customers with Age [60]

DataOps follows the same principles just like DevOps/ DevSecOps but here it has few more elements related to Data domain.

DataOps focuses on data related aspects of operations. It makes sure data is properly integrated with source and target, its flowing-in using automated and best practices data pipelines, its highly secured, it’s of high quality etc.

DevOps doesn’t care whether its data project or mobile application project or any other technology project, but DataOps focuses on Data like where it is coming and where it will go, why it is coming and how it will be used, how much it costed and how much it will monetize etc.

As mentioned above, DataOps includes all the steps or processes involved in DevOps/ DevSecOps but at the same time it also introduces a few additional steps.

DataOps Process Flow

• Regular Integration
• Regular Development
• Regular Build
• Regular Testing
• Regular Deployment
• Regular Delivery/ Run
• Regular Monitoring and Feedback

We all have heard about the Data Driven Organization concept, so here is how it’s done. Data Democratization will make it happen.

‘Data Democratization is the foundation for Data Self-Service Environment.’

Data Democratization/ Data Self-Service is not a tool, nor it’s a technology. It’s about the working behavior of everyone within an organization with respect to data. Traditionally, whenever we need data, we go to IT and ask for some reports or dashboards. We normally tell them, this is our KPI, in other words, we tell them in simple English e.g., we need sales reports about a product from a particular region. Now, it’s IT’s job to find out where relevant data is residing, it’s their job to extract, process and prepare data for business users in the shape of a report or a dashboard. The worrying part is e.g., after IT takes 3 months to prepare a dashboard for Business User and when user says, please include 1 more column 😊, now the whole exercise starts from scratch. Data Democratization fills this gap.

Data Liberalization and Data Democratization are very confusing terms as both work together and most of the time mixed with each other.

‘Data Liberalization ideally is a prerequisite for Data Democratization.’

The aim for Data Democratization is to remove the fear factor about data. Allow everyone to ask analytical questions. To have data access availability for everyone. Allow everyone to raise questions about data and then suggest resolutions, if possible.

Data Liberalization is about providing data in such a way that it should be reusable by users. It can be in any formal i.e., via API, via BI tools, via DaaS etc.

When we say, ‘Aim is to have data access availability for everyone.’ doesn’t mean show everything, of course with access control. Data Liberalization completes the cycle to make sure whatever user is accessing, is USABLE for them rather a pile of junk data.

Data Driven Organization concept can’t be completed just by Data Democratization, Data Liberalization is also a very crucial component to achieve.

Data Literacy is the key for all Data Initiatives.

We know the difference between Business Intelligence (BI) developers and Business Analysts (BA). In previous eras, BA used to talk to business, create KPI(s) and pass it to BI developers to create reports and dashboards. 90% of BI developers would not know what a report or a dashboard is for. They won’t know the impact of good and bad reports. They won’t know the ROI of information this report, or dashboard can produce due to which they don’t care whether BA has provided right KPI(s) or not. This era was not a Data Era, in that era Reports or Dashboards were good to have. Most of the decision making used to happen in reports extracted in excels 😊.

Now time has changed, we have entered the Analytics era as compared to previous era of Analysis. Difference between Analysis and Analytics is simple but tricky i.e., Analysis is about finding Whys’ and What’s whereas Analytics is about finding How’s from a dataset. In other words, in datasets, Analysis search Questions and Analytics propose Answers.

Data Driven Organization is a new Buzz word in market where every organization wants to tag this Key Word with their organization.

Traditionally, we use data to make decisions. Decisions at top level for organizational revenue generation planning or at division level like whether division of a particular product is performing well or is it achieving its planned ROI, is it generating profits for the company.

Above mentioned attributes or properties doesn’t make you Data Driven Organization, above are previous era organizational behavior. Now, the current era is all about Data, everything within an organization must follow data decisions. When we data decisions we are talking about Daily Chores. We are talking about how employees hiring, and firing is done, we are talking about how transfers are done, we are talking about how printer Ink or Papers are orders, we are talking about procurement works, we are talking about how hardware, software or infrastructure resources are provisioned or de-provision.

Monetization is about generating benefit/ revenue/ profit from anything.

‘Data Monetization is about generating revenue from Data.’

In this era, social media is a very dominant channel to interact with customers. Platforms like WhatsApp, Facebook, Twitter, TikTok, Instagram etc. all provide free services to their customers. Have you ever noticed, if you are not paying a single penny to them then how they are generating money and how they are running their business?

The response to the above is by Data Monetization. They sell the data that’s generated by their customers. Having said that, please note, there are regulations that they can’t share personal details of their customers with anyone. But that doesn’t mean they can’t sell how many people are viewing what, how many are buying, how many are spending how much on which product category, from which location they are logged in etc.

For example, on Amazon, customers can buy anything they like. Now at the back end, Amazon must be recording what you are buying and what you are selling. Based on this Information, they can very well analyze which product is more viewed on their portal or what is the buying trend of a particular set of products in a particular season. Like, in December, they can see the trend of products related to Christmas are in great demand. They can sell information like WHICH & HOW MANY PRODUCTS WILL BE IN DEMAND IN COMING DECEMBER based on last few years trend to the product owner to plan their production. This information is GOLD for product owners, they can plan their raw material, work force, timeline and many more activities.

This is a principle rather a process to manipulate data, but it holds a very significant value in terms of managing and securing data.

Most of us mix Critical Data Elements with elements or data points identified during Data Quality exercise. Though those are also critical but when we talk about ‘Critical Data Element’ in Data Strategy, we categorize those elements which can have negative impact on the organization.

Critical Data Element can be any regulatory requirements, any legal or compliance requirements, those data elements which can negatively impact the image of company in general public, those data elements which if are found inaccurate make stakeholders’ loss their confidence on data.

Personal Information of customers is one of the most critical data elements in most of the organization. Credit or Debit Card details in the Banking sector are critical. Location details in Telecommunication are critical etc.

This is also a very confusing term in Data domain, a lot of people are not sure how to assure that everyone follows Ethics in their day-to-day activities while interacting with Data. Just like CDE (Critical Data Element), Data Ethics is also a principle and must be managed at organization level and followed at individual level.

Now, the big question is which data falls under the ambarella where Data Ethics must be followed. There are multiple factors involved and everyone has his/ her own level of ethics to handle based on their roles and responsibilities.

Data Ethics is also a term which if not taken care of properly, can damage a company’s image with the general public. To make sure organizations follow ethics, different regulatory bodies in different parts of the world and in different sectors give guiding principles to follow. It doesn’t stop here, if the organization doesn’t end up following all the measures asked-for by those regulatory bodies then they can be penalized with heavy amounts as penalties.

Before we understand Data Due Diligence, let’s first have a quick refer what is Due Diligence.

Due diligence is an enquiry, inspect, or assessment executed to check evidence or specifics of a matter beneath attention.

For example, when we are buying something we first check e.g., it is made of which country, it’s a new or used item, it’s produced by which company, what’s the quality of the items etc. All these checks and balances come under due diligence process.

Wikipedia

‘Due diligence is the investigation or exercise of care that a reasonable business or person is normally expected to take before entering into an agreement or contract with another party or an act with a certain standard of care.’

There can be many sub processes involved based on domains, businesses, or solution requirements.

A merger and acquisitions (M&A) process is very old process where due diligence is a must, of course when we’re merging or buying a million-dollar company, we need to make sure our decision is correct.