Knowledge Hub

3NF Data Modelling, championed by Bill Inmon, organizes enterprise data in a normalized structure, typically Third Normal Form (3NF), to build a centralized Enterprise Data Warehouse (EDW). The warehouse is subject-oriented, integrated, time-variant, and non-volatile, serving as the authoritative single source of truth for the organization.

The primary goal of a 3NF Data Warehouse is to create a consistent and governed enterprise data foundation. By eliminating redundancy and enforcing referential integrity, the EDW ensures that core entities such as Customer, Loan, Officer, and Product are defined once and integrated across all source systems. This approach prioritizes data quality, traceability, and long-term governance over direct query performance.

In this architecture, data from operational systems is extracted, cleansed, standardized, and loaded into normalized tables. For example, Customer data from a CRM system and Customer data from a Core Banking system are reconciled and integrated under a single enterprise identifier. Loan terms, balances, approval decisions, and officer assignments are stored in separate but related 3NF tables, each with clearly defined primary and foreign keys. Redundancy is minimized, and relationships are explicitly maintained.

Data Vault Data Modelling belongs to the family of Ensemble Data Modelling.

In discussions about modern data warehousing, terms like Data Vault (DV), Business Vault (BV), Raw Vault, and Information Vault are often used, leading to confusion. However, it’s crucial to clarify there is only one Data Vault methodology, designed to streamline and enhance data warehousing. Originating in 2000, Data Vault was developed as a comprehensive, agile warehousing method that effectively manages complex data requirements. What some refer to as “Business Vault” or “Raw Vault” are not distinct systems but extensions within the core Data Vault model, designed to address specific needs within the same foundational framework. This misconception often creates unnecessary complexity; it’s more accurate to simply refer to “Data Vault” as a single, unified approach.

To fully understand Data Vault modelling, it's helpful to first revisit foundational Data Warehouse concepts. Decades ago, Bill Inmon, known as the "Father of Data Warehousing," introduced the Data Warehouse concept, based on third normal form (3NF) data modelling, which emphasizes organizing data by business entities to reduce redundancy and maintain integrity. Ralph Kimball later popularized the dimensional approach, which uses star or snowflake schemas to simplify data for analytical purposes. This article will focus on the Data Vault model, which is a modern, agile alternative that addresses some of the limitations of traditional models.

Just like Data Vault, Focal Point Data Modelling also belongs to the family of Ensemble Data Modelling.

The modelling process in the Focal methodology is organized into five distinct stages, referred to as the Focal Forms:

• Focal Form One – Identify and define the primary business concept entities.
• Focal Form Two – Identify and define the natural business relationships between the core business concepts.
• Focal Form Three – Identify the attributes of the core business concepts, focusing on capturing terms or attributes that are categorized as descriptive characteristics of the business concepts.
• Focal Form Four – Group the attributes into Descriptor Concepts. This step is unique to Focal, as it emphasizes good definition practices. The task is to categorize attributes into one of 10 predefined Descriptor Concepts, each with a specific definition. For example, one such descriptor concept is Condition, which is a qualifying attribute related to a Focal. A Condition serves two main purposes: (1) it imposes constraints or modifications on the Entity’s characteristics or behaviour, and (2) it specifies requirements that must be met by the Entity. These Conditions are often established through agreements between relevant parties regarding specific Focal members. In practice, Conditions take the form of concrete parameters that influence business relationships or transactions. For instance, in a Contract Focal, the start date acts as a temporal Condition; in a Loan Focal, the interest rate represents a financial Condition; and in an Order Focal, the payment date serves as a temporal Condition that needs to be met.
• Focal Form Five – Atomic Context. Determine which attributes group together at the most granular level, answering a specific atomic question. This step is crucial for physical implementation, as it helps identify attribute sets that consist of a small number (usually no more than five) of related attributes.

Just like Data Vault & Focal Point, Anchor Data Modelling also belongs to the family of Ensemble Data Modelling.

These methodologies aim to design data models that are flexible, scalable, and adaptable to changing business requirements. Anchor Data Modelling focuses on simplifying the Modelling process, improving data governance, and ensuring data consistency while supporting agile data integration and analytics.

The modelling process in the Anchor methodology is organized into below list of elements from which main are Anchor, Knot and Attribute.

The main elements of Anchor Data Modelling, as you mentioned, include Anchor, Knot, Attribute, and Tie. Below is an elaborate explanation of each of these elements:

1. Anchor

The Anchor is the fundamental building block of Anchor Data Modelling. It represents a core entity or concept that exists over time and is independent of other attributes. Anchors are stable and persistent throughout their lifecycle.

The Unified Star Schema (USS) is an advanced dimensional modelling approach that enhances traditional star schemas by introducing a central structure known as the Puppini Bridge. This bridge aligns facts and dimensions into a single, consistent analytical framework, making queries more reliable, intuitive, and resistant to common join errors.

The primary goal of USS is to provide a unified query entry point that eliminates ambiguity when working with multiple fact tables. It simplifies drill-across analysis and ensures business users receive consistent results without needing deep SQL expertise.

In a traditional dimensional warehouse, each business process, such as Loan Approvals, Repayments, or Customer Interactions, typically has its own fact table. While dimensions may be conformed across these stars, combining multiple fact tables in a single query can lead to common modelling pitfalls:

Yes, you are right, you might have not heard, or implemented HOOKS before. This is a new term introduced by Andrew Foad.

HOOK is a technique to Data Warehousing especially proper to Data Lakehouse and virtualized architectures, but having said that it can be applied on any SQL based database. It is a hybrid methodology that borrows features from other methodologies such as Data Vault and Kimball dimensional modelling.

Whereas Data Vault and Kimball methodologies have multiple table types (Hubs, Links, Satellites, Dimension, Facts, etc.), HOOKS has a single structure called a Bag, which is used to wrap existing data warehouse table and exposes Business Keys which align to Business Concepts define in the Enterprise Data Model (EDM).

Bags align with data ingested into the data warehouse from the operational system (source-aligned) and Business Concepts defined in the Enterprise Data Model (business-aligned).

As the name says itself, these are traps. If we have Chasm or Fan Traps, then we have a serious problem as these can impact the whole data model.

‘Chasm and Fan Trap lead to Cartesian Data.’

Fan Trap: In Fan Trap, the model ends up having ‘One to Many’ relationship in more than two tables e.g., One Customer having Many Orders and One Order having many Order Products which is logically fine but if physical data model is created like 1) Customer with Order ‘One to Many’ 2) Order with Order Product also ‘One to Many’ then it’s not fine. It’s a Fan Trap.

• Issue: In this scenario, Order Id (Pkey) has been added in Order Products tables as Fkey which means there will be Low Cardinality in Order products tables means there will be duplication of Product in Product’s table which is a wrong approach. In the Products table, Product ID should be unique.
• Solution: One Product can have Many Orders but not the other way. The correct approach is to have Product ID (Pkey) from Order Products table as Fkey in Orders table.

Immutable Data Warehouse has come from immutable data. Data folks are aware of it but don’t use it more often.

‘Immutable means not changeable.’

Now, the question is how it’s relevant to us in Data World. We know when we are inserting new data in a database, at the same time we also have the option to change it by using update or delete commands which means data is mutable.

Mutable feature is a MUST TO HAVE in any database at a technology level, means, any databases or data store MUST allow to update or delete. One should or should not update or delete, can be a business requirement but cannot be technology limitation.

‘By default, Data Warehouse is an Immutable datastore.’

Immutable feature is not only about, not able to update or delete data, it’s about which data should be immutable, and which should be mutable. For this, one must understand the data.

For example, CUSTOMER dataset contains many types of data e.g., name, date of birth, parents, children, address, contact details, jobs, cars etc. In this dataset, you must have judged which information is immutable and which mutable.

A Logical Data Warehouse (LDW) is not a new storage technology but an architectural approach to delivering a unified data warehouse view without physically consolidating all data into one centralized repository. Unlike a traditional physical data warehouse, which extracts, transforms, and loads data into a dedicated storage layer, a Logical Data Warehouse integrates data virtually. It connects distributed data sources and presents them as a single, consistent analytical environment.

The concept does not replace traditional modeling approaches. A Logical Data Warehouse can still support third normal form structures aligned with the philosophy of Bill Inmon or dimensional modeling approaches such as star and snowflake schemas aligned with Ralph Kimball. The distinction lies not in the modeling technique but in the physical location of the data. In an LDW, data may remain in source systems, data lakes, physical warehouses, business vault layers, delta-based platforms, streaming systems, or APIs. The integration happens logically rather than physically.

We touched base with Data Mining and Data Science, it will be ideal to discuss Data Dredging, Snooping, p-hacking, and Fishing. New terms right 😊?

Data Dredging, Snooping and Fishing all refer to the same behavior of data analysis BUT without proper hypothesis and relationship among datasets.

Data mining finds results based on the correlation of data in large data sets, but Data Dredging, Snooping, p-hacking, and Fishing find results based on chance methodology. In other words, Data Dredging, Snooping, p-hacking, and Fishing share the results which require more investigation. These sometimes bypass Data Mining techniques and come up with immature conclusions. On occasions, it shows more details about something than it contains.

There are two types of Fact Findings in any analysis that ultimately assists in DSS i.e., 1) known facts 2) unknown facts. Data mining techniques extract Known Facts as it’s done only on structured datasets and mostly, we know what's there in structured data like relationships between tables and relationships of data within the same table. But that's Not True, sometimes, even in structured data there are Unknown Facts and that’s where Data Dredging, Snooping and Fishing techniques come into the action.