Knowledge Hub

Data Maturity is the key to becoming a Data Driven Organization. There are many factors to judge it.

‘Maturity refers to stability level of an entity or a process or a product.’

Maturity is not about how many stages of process has passed through, it’s about how stable the product is after implementation.

You must have heard people referring to some individual that he is a mature person so let him take his own decision. Same way, decision can be mature or immature which depend on the result from those decisions.

In this era where Data has become the most powerful input for any organization, well it has been since its start. But now technologies are so MATURE that insight from data analytics are taken very seriously and million-dollar or sometimes billion-dollar deals are taken based on Data Maturity.

Everyone remembers the acquisition of WhatsApp right? Multi-billion-dollar deal just based on how many users is using it and what is the potential growth on a simple mobile application.

Data Inventory is a very simple term but sometime get a little bit confused with Data Catalog and Data Assets etc.

Data Assets is about knowing which kind of data we have in our organization with monetary value.

Data Catalog is about knowing system details, databases, tables, and columns and then mapping those with business requirements.

Data Inventory is about consolidating information from Data Asset and Data Catalog with addition details like.

• where this data comes from
• who generated this data?
• when it was generated
• frequency of inserts, updates, and deletes.
• contents details
• security and privacy approaches
• metadata management approaches
• data quality standards
• transformation details
• business rules
• data models
• list of software
• name of all the applications
• licensing details
• data sources details
• target systems details.
• owners of source and target systems

We all maintain inventory of something or the other, either it’s at home or office. A simple example is Passwords Inventory, now a days there are many social media applications then Hotmail, Gmail, etc. We normally maintain all those passwords somewhere safe right so that is your Password Inventory.

Data Supply Chain is not a data capability rather a process.

Just like generic supply chain, Data Supply Chain works in the same manner where it starts from Source Data Stage 🡪 Transformation and Integration Stage 🡪 Consumption Stage

• Source Data Stage is the first stage where raw data is planned and identified i.e., OLTP systems.
• Transformation and Integration Stage is where data is converted based on business requirement in-shape of Data Lake Systems or Data Warehouses
• Consumption Stage is where data is store for business use in-shape of Data Lake Systems or Data Warehouses (OLAP systems) or Semantic Layer etc.

Data Supply Chain is a very critical operational process to manage Data Asset and Liabilities. In the absence of visibility on Data Supply Chain, organization cannot plan ROI as well as TTM which is critical for the Data Monetization.

Note: Data Assets, Data Liabilities and Data Monetization all are explained in separate topics.

Data Lifecycle Management (DLM) is mostly confused with Data Supply Chain which is not 100% correct.

Data Lifecycle Management is about the process from where data was generated till it was removed from the systems whereas Data Supply Chain is about where data come from and where it goes.

There are many sets of steps for Data Lifecycle management, we will be looking into the most commonly used.

Steps in Data Lifecycle Management (DLM)

• Data Generation means the channel where data is getting generated e.g., mobile application, CRM, ERP, HRM, CCTV, Social-Media, Audio(s), Emails, Word document, PDF, Images, Scans etc.
• Data Processing means, what are the business rules based on which raw data is transformed in required format which can be via ETL or ELT or via Data Mining or Data Science Machine Learning Modelling etc.
• Data Reconciliation, Presentation and Utilization means how transformed data is matched with raw data then how it’s been shown on Reports and Dashboard, and lastly for what purpose it has been used.
• Data Storage Lifespan and Extraction means, for how long the data will be kept and what are the means to extract it which can be manual or can be via automated tools.
• Data Removal means how data has been removed and based on which criteria(s).

As the name says all, Data Sharing is about making data accessible to all the relevant stakeholders.

‘Data Sharing is about eliminating Dark Data.’

Data Sharing is one of the key components in Data Driven Organization where organizations aim to build data culture with Data Liberalization.

Data Sharing Channels

• Using Business Intelligence tools
• Using languages like SQL, Python etc.
• Using APIs or direction Using JDBC/ ODBC Connectors
• Using built-in Drivers
• Using Machine Learning or Using Data Mining Models
• Exporting in Excel, CSV, FF formats
• Pushing data to other databases and applications
• Letting other databases and application to Pull data.
• ETL/ ELT tools
• Audit, Compliance or Regulatory Reports etc.

Initially, I planned to write two separate topics on Data Strategy and Data Management but as both are huge areas and the aim of this book is not to train people but rather to make sure readers understand the topics, so decided to combine.

Data Strategy is about defining many elements to make sure when a data solution is implemented, it covers everything to host data, to manage data, to protect data, to maintain data, to make use of data etc.

‘Data Management is one of the Key Element of Data Strategy.’

As the image depicts itself, Data Strategy is about defining all the features and capabilities before the execution process starts. Data Strategy is about tools, processes, implementation planning, maintaining, planning etc., based on business or organization requirements. Data Strategy covers business strategy, organizational roles, data architecture and data management. Data Strategy goals are to innovate, cater to business requirements and address risk and regulatory challenges.

What does "success" mean, and why is it so important to have a clear idea of it when dealing with data solutions?

Success is like reaching a goal or doing something well. When we talk about data solutions, which are ways to manage and use information, it's crucial to know what success looks like. Imagine trying to win a game without knowing the rules or score. Defining success in data solutions helps us understand if we are doing things right, meeting our goals, and making the most of our information. It is like having a map to guide us on the path to doing a great job with data.

Let's break it down further:

• Understanding Success: Success, in general, means achieving a goal or doing something well. It's like hitting a target or accomplishing what you set out to do.
• Applying Success to Data Solutions: In the context of data solutions, we're talking about ways to handle and use information effectively. This could include tasks like collecting, organizing, and analyzing data.
• Importance of Defining Success: Think of it as setting clear rules for a game or knowing the score you want to achieve. Defining success in the world of data solutions helps us clarify what we're trying to achieve and how we'll know when we've achieved it.
• Setting Goals: When you define success for data solutions, you're essentially setting goals. These goals could be related to accuracy, efficiency, security, or other aspects of managing data.
• Monitoring Progress: Knowing what success looks like allows you to keep track of how well you're doing. It's like having a map that shows you where you are on your journey and how far you are from your destination.
• Making Informed Decisions: With a clear definition of success, you can make informed decisions. You know what you're aiming for, so you can choose the best strategies and tools to get there.
• Continuous Improvement: Defining success isn't a one-time thing. It's an ongoing process. You can adjust your goals as needed and keep improving your data solutions over time.

Data Value Realization refers to the importance, usefulness, or significance of data in achieving organizational goals or objectives.

It implies the actualization or fulfillment of something. Putting these together, "Data Value Realization" suggests the process or action of actualizing or realizing the value inherent in data. This involves strategies, practices, or initiatives aimed at leveraging data to achieve tangible benefits or outcomes.

• Effective Use of Data: Implementing practices to ensure that data is effectively used to derive insights, make informed decisions, and drive positive outcomes.
• Monetization of Data: Extracting economic value from data, for example, by identifying opportunities for new revenue streams or cost savings through data-driven initiatives.
• Achieving Business Objectives: Aligning data initiatives with broader business goals and realizing the value of data in achieving those objectives.
• Enhancing Decision-Making: Utilizing data to enhance decision-making processes, optimize operations, and improve overall business performance.

Data Strategy refers to the overarching plan and vision that an organization establishes to manage, govern, and leverage its data assets effectively. It is a high-level, long-term approach that aligns data initiatives with business objectives. It emphasizes the organization's goals, priorities, and desired outcomes related to data. It involves understanding the value of data, defining data governance practices, establishing data quality standards, and aligning data initiatives with business strategies.

Key Components:

• Vision and Objectives: Defines the purpose and goals of leveraging data within the organization.
• Governance Framework: Establishes policies and procedures for managing and securing data.
• Data Quality Standards: Sets guidelines for ensuring accuracy, completeness, and reliability of data.
• Data Lifecycle Management: Defines how data is created, stored, used, and retired over time.
• Technology Roadmap: Outlines the technologies and infrastructure needed to support data initiatives.

In the data world, a "DATA DOOM PILE" is likened to a collection of unorganized or miscellaneous data items that need attention and organization. It's comparable to the concept of having a repository for data that requires sorting, cleansing, or processing at a later time. Similar to how individuals have physical spaces like a junk drawer or cluttered area for miscellaneous items, in the data context, there are repositories, folders, or databases where unstructured or unprocessed data accumulates before being properly managed or analyzed. These are referred to as "data doom piles," reflecting the need for organization and attention in the data management process.

In the data world, a "DATA DOOM PILE" is like a virtual space where various pieces of unorganized data accumulate, waiting to be properly sorted and managed.

Let's explore this concept with examples that we can relate to:

1. Unsorted Data Files on a Computer: Imagine you have a folder on your computer called "Data Doom Pile." In this folder, you may have random files, photos, or documents that you haven't organized yet. These could be data from school projects, downloaded files, or photos from your camera that need categorization.
2. Uncategorized Emails in Your Inbox: Think of your email inbox as a "Data Doom Pile." You receive various emails with attachments, links, and information. Some of these emails may be important, but others are just sitting there, waiting to be categorized, archived, or deleted.
3. Unprocessed Survey Responses: Suppose you conducted a survey with classmates, and their responses are in a spreadsheet. If you haven't yet analyzed or organized the data to draw conclusions, that spreadsheet can be considered a "Data Doom Pile" of survey responses.
4. Raw Data from Science Experiments: In a science class, after conducting experiments and collecting data, you might have a notebook or digital file with raw data. Before you analyze and present your findings, raw data represents a "Data Doom Pile."
5. Images in an Unorganized Gallery: If you have a collection of photos on your phone or computer that you haven't sorted into albums or categorized, that assortment can be your "Data Doom Pile" of images.
6. Unprocessed Sensor Readings: In a science or engineering project involving sensors, you might collect data from various sensors. If you haven't yet processed or interpreted those readings, the dataset can be viewed as a "Data Doom Pile" waiting for analysis.