Knowledge Hub

Data Vault Data Modelling belongs to the family of Ensemble Data Modelling.

In discussions about modern data warehousing, terms like Data Vault (DV), Business Vault (BV), Raw Vault, and Information Vault are often used, leading to confusion. However, it’s crucial to clarify there is only one Data Vault methodology, designed to streamline and enhance data warehousing. Originating in 2000, Data Vault was developed as a comprehensive, agile warehousing method that effectively manages complex data requirements. What some refer to as “Business Vault” or “Raw Vault” are not distinct systems but extensions within the core Data Vault model, designed to address specific needs within the same foundational framework. This misconception often creates unnecessary complexity; it’s more accurate to simply refer to “Data Vault” as a single, unified approach.

To fully understand Data Vault modelling, it's helpful to first revisit foundational Data Warehouse concepts. Decades ago, Bill Inmon, known as the "Father of Data Warehousing," introduced the Data Warehouse concept, based on third normal form (3NF) data modelling, which emphasizes organizing data by business entities to reduce redundancy and maintain integrity. Ralph Kimball later popularized the dimensional approach, which uses star or snowflake schemas to simplify data for analytical purposes. This article will focus on the Data Vault model, which is a modern, agile alternative that addresses some of the limitations of traditional models.

Just like Data Vault, Focal Point Data Modelling also belongs to the family of Ensemble Data Modelling.

The modelling process in the Focal methodology is organized into five distinct stages, referred to as the Focal Forms:

• Focal Form One – Identify and define the primary business concept entities.
• Focal Form Two – Identify and define the natural business relationships between the core business concepts.
• Focal Form Three – Identify the attributes of the core business concepts, focusing on capturing terms or attributes that are categorized as descriptive characteristics of the business concepts.
• Focal Form Four – Group the attributes into Descriptor Concepts. This step is unique to Focal, as it emphasizes good definition practices. The task is to categorize attributes into one of 10 predefined Descriptor Concepts, each with a specific definition. For example, one such descriptor concept is Condition, which is a qualifying attribute related to a Focal. A Condition serves two main purposes: (1) it imposes constraints or modifications on the Entity’s characteristics or behaviour, and (2) it specifies requirements that must be met by the Entity. These Conditions are often established through agreements between relevant parties regarding specific Focal members. In practice, Conditions take the form of concrete parameters that influence business relationships or transactions. For instance, in a Contract Focal, the start date acts as a temporal Condition; in a Loan Focal, the interest rate represents a financial Condition; and in an Order Focal, the payment date serves as a temporal Condition that needs to be met.
• Focal Form Five – Atomic Context. Determine which attributes group together at the most granular level, answering a specific atomic question. This step is crucial for physical implementation, as it helps identify attribute sets that consist of a small number (usually no more than five) of related attributes.

Just like Data Vault & Focal Point, Anchor Data Modelling also belongs to the family of Ensemble Data Modelling.

These methodologies aim to design data models that are flexible, scalable, and adaptable to changing business requirements. Anchor Data Modelling focuses on simplifying the Modelling process, improving data governance, and ensuring data consistency while supporting agile data integration and analytics.

The modelling process in the Anchor methodology is organized into below list of elements from which main are Anchor, Knot and Attribute.

The main elements of Anchor Data Modelling, as you mentioned, include Anchor, Knot, Attribute, and Tie. Below is an elaborate explanation of each of these elements:

1. Anchor

The Anchor is the fundamental building block of Anchor Data Modelling. It represents a core entity or concept that exists over time and is independent of other attributes. Anchors are stable and persistent throughout their lifecycle.

The Unified Star Schema (USS) is an advanced dimensional modelling approach that enhances traditional star schemas by introducing a central structure known as the Puppini Bridge. This bridge aligns facts and dimensions into a single, consistent analytical framework, making queries more reliable, intuitive, and resistant to common join errors.

The primary goal of USS is to provide a unified query entry point that eliminates ambiguity when working with multiple fact tables. It simplifies drill-across analysis and ensures business users receive consistent results without needing deep SQL expertise.

In a traditional dimensional warehouse, each business process, such as Loan Approvals, Repayments, or Customer Interactions, typically has its own fact table. While dimensions may be conformed across these stars, combining multiple fact tables in a single query can lead to common modelling pitfalls:

Yes, you are right, you might have not heard, or implemented HOOKS before. This is a new term introduced by Andrew Foad.

HOOK is a technique to Data Warehousing especially proper to Data Lakehouse and virtualized architectures, but having said that it can be applied on any SQL based database. It is a hybrid methodology that borrows features from other methodologies such as Data Vault and Kimball dimensional modelling.

Whereas Data Vault and Kimball methodologies have multiple table types (Hubs, Links, Satellites, Dimension, Facts, etc.), HOOKS has a single structure called a Bag, which is used to wrap existing data warehouse table and exposes Business Keys which align to Business Concepts define in the Enterprise Data Model (EDM).

Bags align with data ingested into the data warehouse from the operational system (source-aligned) and Business Concepts defined in the Enterprise Data Model (business-aligned).

Metadata Modelling is the discipline of designing structured frameworks that describe data about data. It defines how business, technical, and operational descriptors are captured, organized, and related so that data assets can be understood, discovered, traced, and governed consistently across the enterprise.

The primary goal of metadata modelling is to establish a structured and standardized approach to cataloguing and managing metadata. By doing so, organizations can improve compliance, strengthen data governance, enhance data quality, and increase trust in analytical and operational systems.

Metadata modelling focuses on descriptors such as:

• Business definitions
• Data ownership and stewardship
• Lineage (upstream and downstream data flow)
• Data source systems
• Refresh frequency
• Sensitivity classification
• Quality rules and thresholds
• Storage location and format

Unlike ontologies, which emphasize semantic meaning and relationships between business concepts, metadata modelling emphasizes the management attributes of data assets. It does not primarily answer “What does a Loan mean?” but rather “Where does the Loan data come from?”, “Who owns it?”, “How often is it refreshed?”, and “Is it sensitive?”

Security & Access Control Modelling is the practice of defining structured rules and frameworks that determine who can access what data, under which conditions, and in what manner. It formalizes data protection through roles, permissions, entitlements, and policies aligned with organizational security requirements and regulatory obligations.

The primary goal of this modelling approach is to ensure that sensitive information is accessible only to authorized users while still enabling legitimate business access. It balances protection and productivity by translating business security rules into enforceable system controls.

At its core, Security & Access Control Modelling defines entities such as:

• Users (individual identities)
• Roles (grouped responsibilities or job functions)
• Permissions (allowed actions, such as read, write, approve)
• Policies (conditional rules governing access)

Typically, users are assigned to roles, roles carry permissions, and policies define constraints such as time of access, geographic location, or device type.

AI / Machine Learning Feature Modelling is the discipline of designing, structuring, and governing features, the measurable variables used as inputs to machine learning models. It defines how features are created, catalogued, versioned, and reused so that model inputs remain consistent, explainable, and trustworthy across the enterprise.

The primary goal of Feature Modelling is to provide a standardized and governed framework for engineering and managing features. Instead of allowing every data scientist to independently create slightly different versions of the same metric, Feature Modelling ensures that features are defined once, documented clearly, and reused consistently across multiple AI use cases.

In practice, raw transactional or external data is rarely fed directly into machine learning algorithms. It is first transformed into structured features such as:

• Customer credit score
• Average repayment delay
• Loan utilization ratio
• Officer approval rate
• Customer income stability index

As the name says itself, these are traps. If we have Chasm or Fan Traps, then we have a serious problem as these can impact the whole data model.

‘Chasm and Fan Trap lead to Cartesian Data.’

Fan Trap: In Fan Trap, the model ends up having ‘One to Many’ relationship in more than two tables e.g., One Customer having Many Orders and One Order having many Order Products which is logically fine but if physical data model is created like 1) Customer with Order ‘One to Many’ 2) Order with Order Product also ‘One to Many’ then it’s not fine. It’s a Fan Trap.

• Issue: In this scenario, Order Id (Pkey) has been added in Order Products tables as Fkey which means there will be Low Cardinality in Order products tables means there will be duplication of Product in Product’s table which is a wrong approach. In the Products table, Product ID should be unique.
• Solution: One Product can have Many Orders but not the other way. The correct approach is to have Product ID (Pkey) from Order Products table as Fkey in Orders table.

Entropy term is normally used in Data Science domain.

• Where finding unexpected outcomes is the aim
• Where surprises are welcomed
• Where results are analyzed based on probability ratio, the lower the probability the better it is.
• Where informative information is something which wasn’t known

Types

• High Entropy means, more surprises, more unexpected values are considered more informative.
• Low Entropy means, less surprises, less unexpected values are considered less informative.

‘Entropy is also called as Extreme Disorder of values.’

Referring to the image, we can see at the starting point all the signs are MINUS, then in the middle there are 50/50 signs of PLUS & MINUS and right at the end all the signs are PLUS. At those extremes left, middle and right corners, the Entropy is at the lowest, so no surprises are expected so nothing much for Data Scientists to predict right. But in 2^nd, 4^th, and 5^th circles, it’s difficult to say how many PLUSES(s) and MINUS(s), if values are not visible, at this points Entropy is at the highest.