Knowledge Hub

“Semantic” is an ancient Greek word: σημαντικός sēmantikós, "significant" is the study of reference, meaning, or truth.

In Data World, Semantic layer is built for the sole purpose of supporting business users.

Semantic layer is to convert complex data models into simple and user-friendly interface for business users.

A semantic layer is the representation layer on top of databases, schema(s), tables and columns in such a way that Business users who have no technical know-how should be able to drag and drop required details on Dashboards. For drag and drop, first, one must assign business names to all databases, schema(s), tables and columns before exposing in BI tools. For example, there can be a column name as Date but the data in this column can be Date_Of_Birth or Transaction Date or Financial_Year_End_Data. Converting data models, column/ table/ schema/ database names as per business understanding is semantic process.

Data Inventory is a very simple term but sometime get a little bit confused with Data Catalog and Data Assets etc.

Data Assets is about knowing which kind of data we have in our organization with monetary value.

Data Catalog is about knowing system details, databases, tables, and columns and then mapping those with business requirements.

Data Inventory is about consolidating information from Data Asset and Data Catalog with addition details like.

• where this data comes from
• who generated this data?
• when it was generated
• frequency of inserts, updates, and deletes.
• contents details
• security and privacy approaches
• metadata management approaches
• data quality standards
• transformation details
• business rules
• data models
• list of software
• name of all the applications
• licensing details
• data sources details
• target systems details.
• owners of source and target systems

We all maintain inventory of something or the other, either it’s at home or office. A simple example is Passwords Inventory, now a days there are many social media applications then Hotmail, Gmail, etc. We normally maintain all those passwords somewhere safe right so that is your Password Inventory.

Data Curation is about generating, managing, and maintaining data for future use.

‘Curation comes from the Latin word curae, meaning "care for" -- a meaning it still has today, including in relation to data.’

Data Curation is about taking care of the data from start till end. From extracting till loading. From management to governance. From raw till transformed. From generation till secured and private storage. From open till encryption. As the name says, CARE FOR. Data Curation is also about taking steps to preserve data for future reference as well. Data Curation is the process of preparing data for business and for decision support systems. There is nothing complex about its definition 😊.

Steps for Data Curation

• Understanding the data
• Transformation
• Metadata Management
• Cleansing
• Reconciliation

Stages of Data Curation

• Storing --> Sharing --> Discoverable

Data Contextualization has emerged as a critical factor in dealing with and summarizing vast, rapidly changing, and diverse unstructured datasets. Let's break it down.

We all understand the concept of context. For instance, when you're about to commence a meeting with management, you begin by acknowledging them and stating the purpose of the meeting. This establishes the context, ensuring that the audience knows what to expect and how to interpret the upcoming presentation.

Similarly, in the realm of data, Data Contextualization has become a fundamental process, especially when extracting insights from unstructured data. Unstructured data typically contains a substantial amount of information that is irrelevant to decision-making. The relevance of a data element may vary depending on the specific decision being made.

Here's why Data Contextualization is essential:

• Filtering Irrelevant Data: It eliminates irrelevant data, reducing the overall volume of data. This, in turn, simplifies data processing, cleaning, transformation, and maintenance.
• Enhancing Decision-Making: By contextualizing data, you can combine external data with internal data, providing a more comprehensive foundation for decision-making.
• Providing Clarity: It offers a clear rationale for the data, making it easier to comprehend and analyze.
• Adding Points of Interest: Data Contextualization allows for the inclusion of relevant insights or points of interest about the data, facilitating better understanding.

Semantics are different for everyone. In an organization, each department has its own semantics, to view data. Same data can have different meanings for one department as compared to others.

Salary for HR department is to know who is getting what and when they hire new candidate, they can refer to what others are getting based on role, JD, skill set etc.. But Salary for Finance department can be, to calculate annual expense for the company.

Semantic Layer in Decision Support Systems are based on business requirements, in their language, using their own key words, their codes etc.

Interoperability on the other hand is about exchanging something across systems so others don’t need to regenerate it again and again.

Semantic Interoperability is about exchanging semantic information across systems or across departments or across organizations. But this comes with a challenge whether the receiving party can understand the keywords, symbols, terminologies etc., from the sending party.

In simple words, Graph Theory defines the structure i.e., data model, MetaGraph are the metadata, Taxonomies are the classifications, and Ontologies are the rules.

MetaGraph, Taxonomies, and Ontologies are essential tools in knowledge management, data governance, and AI, each serving distinct yet occasionally overlapping purposes. Below is a detailed breakdown of their characteristics and applications.

MetaGraph

MetaGraph represents metadata within a graph database or knowledge graph, providing a high-level overview of the data's structure, relationships, and properties.

Key Characteristics:

• Focus: Outlines the data model, specifying entities (nodes), relationships (edges), and their attributes.
• Structure: Functions like a schema, defining node and edge types, along with constraints.

Purpose:

• Aids in understanding the graph's structure.
• Supports data governance, lineage tracking, and semantic clarity.
• Enables enhanced searchability and automation by contextualizing graph content.

Data Wrapping is the practice of augmenting raw data with additional layers of value, such as tools, metadata, and context, to make it easier to use, understand, and act upon. This process is crucial in transforming data from a passive resource into an actionable asset.

The Core Elements of Data Wrapping

Data wrapping goes beyond merely presenting raw data, it involves transforming it into a meaningful and actionable resource. Here are the core elements, expanded with practical insights:

1. Metadata and Context

Metadata acts as the "data about data," providing essential context that helps users interpret and utilize data effectively.

• What It Includes: Descriptions of data sources, timestamps, definitions, units of measurement, and relationships between datasets.
• Why It’s Important: Without metadata, users may misinterpret data or struggle to understand its significance.
• Example: A dataset tracking customer orders might include metadata about the region, the time zone in which orders were placed, and the categorization of product types.