Knowledge Hub

Knowledge Graph Modelling is the practice of representing information as a network of interconnected entities and relationships, enriched with semantic meaning. Instead of viewing data as isolated records stored in tables, this approach treats knowledge as a web of connections. Entities become nodes in a graph, and relationships between them become edges. The structure itself reflects how concepts are linked in the real world.

The primary goal of Knowledge Graph Modelling is to create a flexible and interconnected view of knowledge. By explicitly representing relationships, it becomes easier to integrate information from multiple sources, discover hidden patterns, and enable reasoning across domains. This modelling approach is especially valuable in environments where connections matter as much as individual data points.

A knowledge graph models the world as nodes and edges. Nodes represent entities such as Customer, Loan, or Product. Edges represent relationships such as applies for, owns, guarantees, or reviews. Unlike traditional relational models, where relationships are often embedded in foreign keys, the connections in a knowledge graph are first-class elements. They are visible, traversable, and central to the model.

AI / Machine Learning Feature Modelling is the discipline of designing, structuring, and governing features, the measurable variables used as inputs to machine learning models. It defines how features are created, catalogued, versioned, and reused so that model inputs remain consistent, explainable, and trustworthy across the enterprise.

The primary goal of Feature Modelling is to provide a standardized and governed framework for engineering and managing features. Instead of allowing every data scientist to independently create slightly different versions of the same metric, Feature Modelling ensures that features are defined once, documented clearly, and reused consistently across multiple AI use cases.

In practice, raw transactional or external data is rarely fed directly into machine learning algorithms. It is first transformed into structured features such as:

• Customer credit score
• Average repayment delay
• Loan utilization ratio
• Officer approval rate
• Customer income stability index

Entropy term is normally used in Data Science domain.

• Where finding unexpected outcomes is the aim
• Where surprises are welcomed
• Where results are analyzed based on probability ratio, the lower the probability the better it is.
• Where informative information is something which wasn’t known

Types

• High Entropy means, more surprises, more unexpected values are considered more informative.
• Low Entropy means, less surprises, less unexpected values are considered less informative.

‘Entropy is also called as Extreme Disorder of values.’

Referring to the image, we can see at the starting point all the signs are MINUS, then in the middle there are 50/50 signs of PLUS & MINUS and right at the end all the signs are PLUS. At those extremes left, middle and right corners, the Entropy is at the lowest, so no surprises are expected so nothing much for Data Scientists to predict right. But in 2^nd, 4^th, and 5^th circles, it’s difficult to say how many PLUSES(s) and MINUS(s), if values are not visible, at this points Entropy is at the highest.

DLOps is like MLOps and belongs to Data Science domain.

Before we try to understand DLOps as compared to MLOps, one must understand the difference between Machine Learning and Deep Learning. (There is separate topic on difference between Machine Learning and Deep Learning)

‘It won’t be wrong to say, MACHINE LEARNING is for simpler and more structured & semi-structured use cases whereas DEEP LEARNING is for complex and unstructured use cases.’

Let’s try to understand with a simple example. If we need to find out how many cars, bikes or buses there are in structured or semi-structured datasets then it’s relatively easy as data is in rows and columns format. We can have the width, length, and height of each car, bike or bus so using MACHINE LEARNING models we can easily ready this dataset and produce expected output. But what if we give pictures to ML models to extract the same result, will it be possible? The answer is No as ML models normally work on structured or semi-structured datasets. To find out cars, bikes, or buses from pictures, DEEP LEARNING models are used which can recognize the objects in the pictures and find out which presents car or bike or a bus.

We touched base with Data Mining and Data Science, it will be ideal to discuss Data Dredging, Snooping, p-hacking, and Fishing. New terms right 😊?

Data Dredging, Snooping and Fishing all refer to the same behavior of data analysis BUT without proper hypothesis and relationship among datasets.

Data mining finds results based on the correlation of data in large data sets, but Data Dredging, Snooping, p-hacking, and Fishing find results based on chance methodology. In other words, Data Dredging, Snooping, p-hacking, and Fishing share the results which require more investigation. These sometimes bypass Data Mining techniques and come up with immature conclusions. On occasions, it shows more details about something than it contains.

There are two types of Fact Findings in any analysis that ultimately assists in DSS i.e., 1) known facts 2) unknown facts. Data mining techniques extract Known Facts as it’s done only on structured datasets and mostly, we know what's there in structured data like relationships between tables and relationships of data within the same table. But that's Not True, sometimes, even in structured data there are Unknown Facts and that’s where Data Dredging, Snooping and Fishing techniques come into the action.

Concept Drift is used in Data Science practice, and it is mostly unexpected behavior in data. One must have automated ways to keep tracking and identifying Concept Drifts as after Concept Drift, the model will totally not produce or predict the output which was planned. Wrong prediction means wrong decisions.

We all know what a Drift is. Drift is about shifting or moving from one point to another but not in a standard approach. This term is referred in Data Science practice to evaluate tested vs actual performance of Machine Learning (ML) Models.

Concept Drift is also known as class drift, real concept drift, or posterior probability shift. As we discussed in Data Drift, the data elements do not change rather data changes. But in Concept Drift, data elements themselves change.

For example, referring to the same example from Data Drift, model is trained on products but at the same time buying behavior of those products is dependent on external factors like weather. Let’s say, those products are Cold Drinks and it’s obvious in the region where it’s been sold, the weather is expected to be hot and based on the ML prediction, a production of 1M Cold Drinks are planned on weekly basis. Now, what if, due to any reason the weather sees a sudden change and for a week it rains, and the temperature drops unexpectedly, THIS IS CALLED CONCEPT DRIFT, then the whole model’s behavior will be impacted. Due to pleasant whether people will only consume 25% of 1M Cold Drinks which means the ML model must quickly change its training data and predict the production behavior.

Data Skew Issue is one of the main hidden culprits for query performance 😊. It is normally referred to as Skewed data distribution which is used in Data Science domain.

Have you ever noticed, your query is correct, your joins are correct, your all indexes are enabled, you have high data quality, you have data standardization in place but still your query performance is not as expected? You noticed when you loaded new datasets, queries were flying but as time passed, the performance has gone down. You have added more CPU, more RAM, more nodes but NO performance is not like newly added data. It’s because of the Data Skew Issue.

Data Skew Issue is about when data is not distribution evenly. Uneven distribution of data at cluster level causes this issue. It heavily degrades the performance of the queries especially those with joins.

We all have faced a situation where query run exceptionally good till 99% and then hangs for 3-4 hours right lolz yeah, I can understand your pain 😊. Let me tell you the reason, it’s because of Data Skew Issue.

MRKL stands for Modular Reasoning, Knowledge, and Language. This neuro-symbolic architecture is designed to address complex problems by combining elements of neural computation (specifically Large Language Models, or LLMs) with symbolic computation tools such as calculators. (ref: LLM is separate topic)

Neuro-Symbolic Architecture implies that the system combines aspects of neural computation (associated with LLMs) and symbolic computation. Neural computation involves processing information in a way inspired by the human brain, while symbolic computation involves manipulating symbols to perform operations.

LLMs (Large Language Models) are sophisticated artificial intelligence models trained on extensive text data. They surpass in recognizing and producing human-like dialect.

External Tools Like Calculators in addition to neural computation, the system incorporates external symbolic computation tools, specifically mentioning calculators. This suggests a hybrid approach where both neural and symbolic methods are employed.

The widespread adoption of large language models (LLMs) has led to increased expectations for factual accuracy. However, LLMs often generate incorrect answers with confidence. A notable example is when users query general-purpose models like OpenAI’s “gpt-4o” or DeepSeek’s “DeepSeek-V3” about recent events or niche topics; they may decide not to respond or provide fabricated yet convincing responses. The leading industry solution to mitigate this issue is Retrieval-Augmented Generation (RAG), introduced in Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020). RAG enhances LLMs by integrating external retrieval, ensuring more reliable and factual outputs while improving response contextuality and robustness.

Just like how a human is more likely to give a wrong answer when lacking information, AI models are more likely to make mistakes and hallucinate when they are missing context. For a given application, the model’s instructions are common to all queries, whereas context is specific to each query. By incorporating retrieval mechanisms, RAG enables LLMs to interact with real-time information, improving applications in search engines, customer support, academic research, and business intelligence. Unlike traditional search engines, which rely on keyword-based ranking, RAG synthesizes knowledge dynamically, allowing AI-driven solutions to be more adaptable and responsive. This chapter explores RAG’s framework, applications, implementation, security considerations, and future trends.

In today’s rapidly evolving technological landscape, the biggest challenge organizations face is, data integration and maintainability. With the exponential growth of data and the widespread adoption of AI technologies, businesses are increasingly grappling with the need to manage both effectively. Unfortunately, many traditional architectures treat data and AI as distinct entities, leading to a host of challenges that hinder an organization’s ability to innovate, scale, and adapt.

One of the primary issues with the traditional approach is, the fragmentation of data and AI systems into separate silos. Data is often stored and managed in one architecture, while AI models are built, trained, and deployed in another. This separation creates significant friction points.

To overcome these challenges, organizations must embrace a more integrated approach that treats data and AI as interconnected components of a single architecture.