Once you sign in, your details will show here.
Knowledge Hub
Filter (1)
Sort
87 topics · Data Modeling and Design · 10 shown
Show allEvery topic is free to read. A free account also lets you search the topics, react to a topic and comment under it.
- I Am Datapedia!
Data Modeling and Design
NoSQL Data Modelling
NoSQL Data Modelling refers to the practice of designing data structures for non-relational databases such as document stores, key-value stores, wide-column databases, and graph databases. Unlike relational modelling, which is driven by normalization theory and functional dependencies, NoSQL modelling is driven primarily by application access patterns and workload requirements. The goal is not theoretical purity but scalable, high-performance data access.
The primary objective of NoSQL data modelling is to support flexible or schema-less storage, distributed scale-out architectures, and fast, query-driven design. Instead of decomposing data into highly normalized tables, NoSQL models are often intentionally denormalized or embedded to minimize joins and reduce cross-node communication in distributed environments. Data is structured according to how it will be read and written, not according to relational dependency rules.
3 min readMore
- I Am Datapedia!
Data Storage and Operations
Document Databases
Document databases store data as JSON-like documents that can contain nested fields, arrays, and variable structures. Instead of organizing information into rows and columns, they treat each document as a self-contained representation of a business object. The goal is to provide flexible schema design and rapid development by allowing complete entities to be stored and retrieved in a single record.
In a document database, each document typically represents one business object. In a loan approval system, for example, a single document may represent an entire loan application. Customer details, loan officer information, approval status, and repayment history can all be embedded within that same structure. Because the schema is flexible, new fields can be added without requiring costly table migrations or rigid schema alterations. This makes document databases particularly attractive in fast-moving development environments.
5 min readMore
- I Am Datapedia!
Data Storage and Operations
Key–value Databases
Key–Value databases represent the simplest form of non-relational data storage. They store data as a pair consisting of a unique key and its associated value. The key serves as an identifier, while the value contains the data itself. This model eliminates structural complexity and focuses entirely on fast retrieval by key.
The primary goal of key–value databases is to deliver ultra-fast lookups with minimal overhead. They are designed for workloads where the access pattern is simple and predictable, such as caching, session management, token storage, or high-volume read/write operations. Instead of supporting complex queries, these systems optimize for speed and scalability.
In a key–value database, each record is accessed directly through its key. The value associated with that key can be a simple string, a JSON object, or even a binary blob. There is no concept of joins, relational constraints, or query-based filtering beyond key lookup. This makes the model extremely efficient for operations such as “fetch loan details by LoanID.”
3 min readMore
- I Am Datapedia!
Data Storage and Operations
Columnar (Wide-Column) Databases
Wide-Column databases, often referred to as column-family databases, organize data into rows and dynamic columns grouped into column families. Unlike traditional relational tables where every row follows a fixed schema, wide-column systems allow each row to contain a flexible and potentially sparse set of columns. They are optimized for distributed storage and partitioned queries across very large datasets.
The primary goal of wide-column databases is to enable high write throughput and scalable query performance over massive volumes of data. They are particularly well suited for time-series, event-driven, and high-velocity workloads where large amounts of data must be written and retrieved efficiently across distributed clusters.
In this model, data is partitioned by a primary key, for example, CustomerID or LoanID. All related data for that key is stored together within a partition. Within each partition, rows may contain flexible columns that vary between records. The schema is shaped primarily by access patterns. Instead of normalizing entities into separate tables, designers structure data to answer specific queries efficiently.
3 min readMore
- I Am Datapedia!
Data Storage and Operations
Graph Databases
Graph databases store data as nodes and edges, optimized specifically for representing and querying connected information. Nodes represent entities, while edges represent relationships between those entities. Unlike relational systems that organize data into tables and connect them through joins, graph databases treat relationships as first-class citizens, enabling direct traversal across connections.
The primary goal of graph databases is to uncover insights from relationships and networks. They are particularly effective in scenarios such as fraud detection, influence analysis, recommendation systems, and complex workflow analysis, areas where relational joins become expensive and difficult to manage at scale. When the question being asked is fundamentally about how things are connected, graph databases often provide a more natural solution.
In a loan approval context, nodes might represent entities such as Customer, Loan, LoanOfficer, and Branch. Edges define how these entities are connected. For example, a Customer applies for a Loan, an Officer approves a Loan, and a Loan is issued at a Branch. Because relationships are stored explicitly rather than inferred through joins, queries can traverse the network efficiently.
6 min readMore
- I Am Datapedia!
Data Modeling and Design
Event-Driven Data Modelling
Event-Driven Data Modelling structures data around events, discrete occurrences that capture something that has happened, typically with a timestamp and contextual details. Instead of centering the model on static entities such as Customer or Loan, this approach treats events as the primary unit of design. Each event represents a fact in time.
The goal of event-driven modelling is to enable real-time processing, streaming architectures, and event sourcing patterns. By designing systems around events, organizations can react immediately to business activity while maintaining a complete, replayable history of what occurred.
In this model, the focus shifts from storing the current state of entities to recording event streams. Events are immutable and append-only. Once an event is written, it is never updated or deleted. Typical events in a loan approval domain might include LoanApplicationSubmitted, LoanApplicationApproved, or RepaymentMade. Each event contains identifying information, contextual data, and a timestamp.
3 min readMore
- I Am Datapedia!
Data Modeling and Design
Json / Xml Schema Modelling
JSON / XML Schema Modelling is the practice of defining structured schemas for semi-structured data formats such as JSON and XML. These schemas specify the allowed structure, elements, datatypes, and constraints of documents to ensure consistency, validation, and interoperability across systems. Rather than focusing on database storage, this modelling technique focuses on structured data exchange.
The primary goal of JSON / XML Schema Modelling is to provide a clear and enforceable contract for APIs, integration workflows, and document-based systems. When multiple applications exchange data, they must agree not only on the names of fields but also on their structure, required attributes, data types, and validation rules. A schema formalizes these expectations and prevents inconsistent or malformed data from entering the system.
In this approach, schemas define elements, nesting rules, datatypes, required fields, and sometimes value constraints. For example, a Loan schema might define that LoanID is a required string, LoanAmount is a numeric field greater than zero, and Status must be one of a predefined set of values. The schema ensures that every system sending or receiving loan data adheres to the same structural rules.
3 min readMore
- I Am Datapedia!
Data Modeling and Design
API / Service Data Modelling
API / Service Data Modelling is the practice of designing the data contracts and structures that define how information is exchanged between services, applications, and systems. Rather than focusing on how data is stored internally, this approach concentrates on how data is exposed, requested, and consumed across interfaces.
The primary goal of API / Service Data Modelling is to ensure clarity, consistency, and stability in data exchanged across APIs and services. Well-designed API models reduce integration complexity, prevent misunderstandings between systems, and minimize runtime errors. They act as formal contracts between service providers and service consumers.
In this approach, the model defines the structure of inputs and outputs. For example, a Loan Application API may specify required request fields such as CustomerID, LoanAmount, and Term. The response may include fields such as LoanID, ApprovalStatus, and InterestRate. These definitions establish what data must be provided, what data will be returned, and under what rules. The emphasis is not on database tables but on message structures.
4 min readMore
- I Am Datapedia!
Data Modeling and Design
Temporal Data Modelling
Temporal Data Modelling is the practice of structuring data so that changes over time are explicitly captured and preserved. Instead of storing only the current state of information, temporal models record how data has evolved, enabling systems to answer questions about the past, the present, and even planned future states.
The primary goal of Temporal Data Modelling is to enable time-aware queries, regulatory compliance, auditing, and historical analysis. In many industries, especially banking, insurance, healthcare, and government, organizations must demonstrate not only what data looks like today, but what it looked like at a specific point in time.
Temporal modelling distinguishes between different types of time. Valid Time refers to when a fact is true in the real world. For example, an interest rate may be valid from January to March. Transaction Time refers to when the fact was recorded in the system. A correction entered in February about a January event has a different transaction time from its valid time. When both are captured together, the model is called bi-temporal.
3 min readMore
- I Am Datapedia!
Data Modeling and Design
3NF Data Modelling(bill Inmon)
3NF Data Modelling, championed by Bill Inmon, organizes enterprise data in a normalized structure, typically Third Normal Form (3NF), to build a centralized Enterprise Data Warehouse (EDW). The warehouse is subject-oriented, integrated, time-variant, and non-volatile, serving as the authoritative single source of truth for the organization.
The primary goal of a 3NF Data Warehouse is to create a consistent and governed enterprise data foundation. By eliminating redundancy and enforcing referential integrity, the EDW ensures that core entities such as Customer, Loan, Officer, and Product are defined once and integrated across all source systems. This approach prioritizes data quality, traceability, and long-term governance over direct query performance.
In this architecture, data from operational systems is extracted, cleansed, standardized, and loaded into normalized tables. For example, Customer data from a CRM system and Customer data from a Core Banking system are reconciled and integrated under a single enterprise identifier. Loan terms, balances, approval decisions, and officer assignments are stored in separate but related 3NF tables, each with clearly defined primary and foreign keys. Redundancy is minimized, and relationships are explicitly maintained.
4 min readMore
Build a strong foundation in Data & AI
Follow the Data Foundations learning path (Beginner to Advanced).