Knowledge Hub

230 topics · Intermediate · 10 shown

Show all

Every topic is free to read. A free account also lets you search the topics, react to a topic and comment under it.

  1. I Am Datapedia!

    Data Modeling and Design

    NoSQL Data Modelling

    NoSQL Data Modelling refers to the practice of designing data structures for non-relational databases such as document stores, key-value stores, wide-column databases, and graph databases. Unlike relational modelling, which is driven by normalization theory and functional dependencies, NoSQL modelling is driven primarily by application access patterns and workload requirements. The goal is not theoretical purity but scalable, high-performance data access.

    The primary objective of NoSQL data modelling is to support flexible or schema-less storage, distributed scale-out architectures, and fast, query-driven design. Instead of decomposing data into highly normalized tables, NoSQL models are often intentionally denormalized or embedded to minimize joins and reduce cross-node communication in distributed environments. Data is structured according to how it will be read and written, not according to relational dependency rules.

    3 min readMore

    Share this topic

    NoSQL Data Modelling

    React
  2. I Am Datapedia!

    Data Storage and Operations

    Document Databases

    Document databases store data as JSON-like documents that can contain nested fields, arrays, and variable structures. Instead of organizing information into rows and columns, they treat each document as a self-contained representation of a business object. The goal is to provide flexible schema design and rapid development by allowing complete entities to be stored and retrieved in a single record.

    In a document database, each document typically represents one business object. In a loan approval system, for example, a single document may represent an entire loan application. Customer details, loan officer information, approval status, and repayment history can all be embedded within that same structure. Because the schema is flexible, new fields can be added without requiring costly table migrations or rigid schema alterations. This makes document databases particularly attractive in fast-moving development environments.

    5 min readMore

    Share this topic

    Document Databases

    React
  3. I Am Datapedia!

    Data Storage and Operations

    Key–value Databases

    Key–Value databases represent the simplest form of non-relational data storage. They store data as a pair consisting of a unique key and its associated value. The key serves as an identifier, while the value contains the data itself. This model eliminates structural complexity and focuses entirely on fast retrieval by key.

    The primary goal of key–value databases is to deliver ultra-fast lookups with minimal overhead. They are designed for workloads where the access pattern is simple and predictable, such as caching, session management, token storage, or high-volume read/write operations. Instead of supporting complex queries, these systems optimize for speed and scalability.

    In a key–value database, each record is accessed directly through its key. The value associated with that key can be a simple string, a JSON object, or even a binary blob. There is no concept of joins, relational constraints, or query-based filtering beyond key lookup. This makes the model extremely efficient for operations such as “fetch loan details by LoanID.”

    3 min readMore

    Share this topic

    Key–value Databases

    React
  4. I Am Datapedia!

    Data Storage and Operations

    Columnar (Wide-Column) Databases

    Wide-Column databases, often referred to as column-family databases, organize data into rows and dynamic columns grouped into column families. Unlike traditional relational tables where every row follows a fixed schema, wide-column systems allow each row to contain a flexible and potentially sparse set of columns. They are optimized for distributed storage and partitioned queries across very large datasets.

    The primary goal of wide-column databases is to enable high write throughput and scalable query performance over massive volumes of data. They are particularly well suited for time-series, event-driven, and high-velocity workloads where large amounts of data must be written and retrieved efficiently across distributed clusters.

    In this model, data is partitioned by a primary key, for example, CustomerID or LoanID. All related data for that key is stored together within a partition. Within each partition, rows may contain flexible columns that vary between records. The schema is shaped primarily by access patterns. Instead of normalizing entities into separate tables, designers structure data to answer specific queries efficiently.

    3 min readMore

    Share this topic

    Columnar (Wide-Column) Databases

    React
  5. I Am Datapedia!

    Data Storage and Operations

    Graph Databases

    Graph databases store data as nodes and edges, optimized specifically for representing and querying connected information. Nodes represent entities, while edges represent relationships between those entities. Unlike relational systems that organize data into tables and connect them through joins, graph databases treat relationships as first-class citizens, enabling direct traversal across connections.

    The primary goal of graph databases is to uncover insights from relationships and networks. They are particularly effective in scenarios such as fraud detection, influence analysis, recommendation systems, and complex workflow analysis, areas where relational joins become expensive and difficult to manage at scale. When the question being asked is fundamentally about how things are connected, graph databases often provide a more natural solution.

    In a loan approval context, nodes might represent entities such as Customer, Loan, LoanOfficer, and Branch. Edges define how these entities are connected. For example, a Customer applies for a Loan, an Officer approves a Loan, and a Loan is issued at a Branch. Because relationships are stored explicitly rather than inferred through joins, queries can traverse the network efficiently.

    6 min readMore

    Share this topic

    Graph Databases

    React
  6. I Am Datapedia!

    Data Modeling and Design

    Json / Xml Schema Modelling

    JSON / XML Schema Modelling is the practice of defining structured schemas for semi-structured data formats such as JSON and XML. These schemas specify the allowed structure, elements, datatypes, and constraints of documents to ensure consistency, validation, and interoperability across systems. Rather than focusing on database storage, this modelling technique focuses on structured data exchange.

    The primary goal of JSON / XML Schema Modelling is to provide a clear and enforceable contract for APIs, integration workflows, and document-based systems. When multiple applications exchange data, they must agree not only on the names of fields but also on their structure, required attributes, data types, and validation rules. A schema formalizes these expectations and prevents inconsistent or malformed data from entering the system.

    In this approach, schemas define elements, nesting rules, datatypes, required fields, and sometimes value constraints. For example, a Loan schema might define that LoanID is a required string, LoanAmount is a numeric field greater than zero, and Status must be one of a predefined set of values. The schema ensures that every system sending or receiving loan data adheres to the same structural rules.

    3 min readMore

    Share this topic

    Json / Xml Schema Modelling

    React
  7. I Am Datapedia!

    Data Modeling and Design

    API / Service Data Modelling

    API / Service Data Modelling is the practice of designing the data contracts and structures that define how information is exchanged between services, applications, and systems. Rather than focusing on how data is stored internally, this approach concentrates on how data is exposed, requested, and consumed across interfaces.

    The primary goal of API / Service Data Modelling is to ensure clarity, consistency, and stability in data exchanged across APIs and services. Well-designed API models reduce integration complexity, prevent misunderstandings between systems, and minimize runtime errors. They act as formal contracts between service providers and service consumers.

    In this approach, the model defines the structure of inputs and outputs. For example, a Loan Application API may specify required request fields such as CustomerID, LoanAmount, and Term. The response may include fields such as LoanID, ApprovalStatus, and InterestRate. These definitions establish what data must be provided, what data will be returned, and under what rules. The emphasis is not on database tables but on message structures.

    4 min readMore

    Share this topic

    API / Service Data Modelling

    React
  8. I Am Datapedia!

    Data Modeling and Design

    Dimensional Data Modelling

    Dimensional Data Modelling is an analytical modelling approach that structures data into facts (quantitative measures) and dimensions (descriptive attributes) to support reporting, dashboards, and business intelligence. It provides a simplified, performance-oriented representation of organizational data, making analytics accessible to both technical and business users.

    The primary goal of dimensional modelling is to deliver a business-friendly and performance-optimized structure that enables fast and intuitive access to information. Rather than focusing on eliminating redundancy at all costs, dimensional modelling focuses on making data easy to query, aggregate, and understand.

    Dimensional models are organized around business processes. A fact table captures measurable events, such as loan approvals, repayments, or transactions. Surrounding the fact table are dimension tables, which provide context, such as customer attributes, officer details, product types, and dates. Together, they answer two fundamental questions:

    3 min readMore

    Share this topic

    Dimensional Data Modelling

    React
  9. I Am Datapedia!

    Data Warehousing and Business Intelligence

    Star Schema

    The Star Schema is a dimensional modelling design in which a central fact table connects directly to multiple surrounding dimension tables, forming a star-like structure. The fact table stores quantitative measures of a business process, while the dimension tables provide descriptive context for analysis.

    The primary goal of the star schema is to deliver a simple, high-performance, and business-friendly structure for analytics. It is designed specifically for reporting, dashboards, and self-service business intelligence, where clarity and speed are critical.

    In a star schema, the fact table captures measurable events such as LoanAmount, InterestRate, ApprovalCount, or RepaymentAmount. Each row represents a clearly defined grain, such as one loan approval or one repayment transaction. The fact table contains foreign keys that link to dimension tables.

    The dimension tables contain descriptive attributes used for filtering, grouping, and slicing data. Examples include:

    3 min readMore

    Share this topic

    Star Schema

    React
  10. I Am Datapedia!

    Data Warehousing and Business Intelligence

    Snowflake Schema

    The Snowflake Schema is a dimensional modelling design in which one or more dimension tables are normalized into related sub-dimensions, forming a snowflake-like structure around a central fact table. While the fact table remains the analytical core, dimension tables are decomposed to reduce redundancy and improve governance.

    The primary goal of the snowflake schema is to improve data quality, storage efficiency, and reusability, especially for large, shared dimensions, while still preserving dimensional modelling semantics for analytics.

    A snowflake schema typically begins as a classic star schema (Fact + Dimensions). Over time, wide dimension tables may be normalized into smaller, related sub-dimensions. For example:

    • Dim_Customer may be split into:
      • Customer (core identity attributes)
      • Demographics (age band, income band)
      • Employment (industry, employer category)
      • Geography (city, region, country)

    3 min readMore

    Share this topic

    Snowflake Schema

    React

Build a strong foundation in Data & AI

Follow the Data Foundations learning path (Beginner to Advanced).

View learning path