Knowledge Hub

Entity Recognition, commonly known as Named Entity Recognition (NER), is a technique used to identify and extract meaningful entities from unstructured text. It is also referred to as entity identification or entity extraction. While entity recognition has long been appreciated in data modeling, particularly during conceptual modeling when identifying core business entities, it plays an equally critical role in the data science domain.

In traditional structured systems, entities such as Customer, Product, Order, or Invoice are clearly defined within database schemas. However, modern organizations generate enormous volumes of unstructured data from social media posts, emails, PDF files, Word documents, comment columns, chat transcripts, and even video or audio content converted into JSON or text formats. For data modelers, this unstructured layer represents one of the greatest challenges when designing robust data models.

Causality Data in the data world refers to information that helps establish a cause-and-effect relationship between different variables or events. In other words, it involves understanding the influence or impact that changes in one variable have on another.

Causality is a fundamental concept in statistics and data analysis, especially when trying to make sense of complex systems or phenomena. Establishing causal relationships is important for drawing meaningful conclusions and making informed decisions. However, it is essential to word that correlation does now no longer imply causation. While two variables may be correlated, it doesn't necessarily mean that one causes the other.

In the context of data science and analytics, establishing causality often involves advanced statistical methods and experimentation.

Here are a few key points related to causality data:

• Experimental Design: Controlled experiments, where researchers manipulate one variable while keeping others constant, are often used to establish causality. This allows for the isolation of the variable of interest and helps draw conclusions about its impact.
• Observational Studies: In cases where controlled experiments are not feasible, observational studies are conducted to analyze existing data. However, establishing causality in observational studies can be more challenging due to potential confounding factors.
• Counterfactuals: Causality often involves comparing what happened to what might have happened in the absence of a particular intervention or change. This comparison is known as a counterfactual analysis.
• Temporal Order: Causality requires that the cause precedes the effect in time. Observing a temporal order between variables is an important aspect of establishing causation.
• Statistical Techniques: Various statistical techniques, such as regression analysis, propensity score matching, and instrumental variable analysis, are employed to analyze data and infer causal relationships.

Inference Data refers to the data produced by ML process of drawing conclusions, making predictions, or generalizing information based on the analysis of a features dataset. It is a crucial step in extracting meaningful insights from data and often involves the application of statistical methods.

Let's decode it.....

Imagine you want to know something about a big group of students, like all the students in your school. Instead of asking every single student (which could be too much work), you ask a smaller group. This smaller group is like a sample. In simpler terms, inference is like making smart guesses about everyone by looking at a few representative friends first. It's a bit like saying, "I can figure out what most people like, by asking a few of them because they're probably a lot like everyone else."

Inference data is a process of drawing meaningful conclusions about a larger dataset based on observed data, and it is a fundamental aspect of statistical reasoning and data analysis in various domains.

Do We Really Need Data Modelling?

In recent years, a recurring question has surfaced in data conversations: Do we really need data modelling anymore? With cloud-native platforms, data lakes, AI pipelines, and schema-on-read architectures dominating the narrative, some assume modelling is optional, or even obsolete.

The reality is far simpler. There is no such thing as data without a model. Whether designed consciously or not, every system that stores data follows a structure. That structure, explicit or implicit, is a data model.

When we build operational applications, we design backend databases. These typically follow normalization principles, often in Third Normal Form (3NF). Customer data is stored separately from orders, orders from payments, and relationships are maintained through keys and constraints. This structure prevents redundancy, preserves integrity, and ensures consistency. That is data modelling in action.

Data Dividend is something everyone must know either IT Professional or End User.

We all know what a Dividend is in the Stock Market. It's when a user buys a Stock of a particular company and keeps it for a certain time, the company pays the user a certain amount in Quarterly or Annual basis. The reason is the company is keeping users' money for a particular duration. At the backend, the company is investing that money in its business to generate revenue. Then a certain percentage of that revenue is shared with the user in the shape of a Dividend.

This era is to generate customer data either via mobile applications or via social media portals like Facebook, Twitter, Instagram, Amazon, Alibaba, etc. Users use these applications and, in a way, give their DATA instead of MONEY like their buying/ selling behavior, their search trend, their family/ friends interactions, etc. to above-mentioned companies which then use that DATA for marketing & advertisement, for selling services, to production houses, to industries, to inventory houses, etc. In this cycle, all these companies are making Fortunes, and nothing has been shared with the ENTITY i.e., end-users who are the core participants for the data generation.

To understand Data Algorithms, let's quickly know what an algorithm is. It is a procedure consisting of multiple steps with a set of instructions for a desired result. It is independent of any language, in other words, an algorithm can be implemented in any language.

Algorithm Characteristics

• Unambiguous: It should be clear and unambiguous. Each of its steps, and its inputs/outputs should be clear and must lead to only one meaning.
• Input: It should have 0 or more well-defined inputs.
• Output: It should have 1 or more well-defined outputs and should match the desired output.
• Finiteness: It must terminate after a finite number of steps.
• Feasibility: It should be feasible with the available resources.
• Independent: It should have step-by-step directions, which should be independent of any programming code.

Now from a data point-of-view, how the algorithm behaves.

• Search: Algorithm to search an item in a data structure.
• Sort: Algorithm to sort items in a certain order.
• Insert: Algorithm to insert an item in a data structure.
• Update: Algorithm to update an existing item in a data structure.
• Delete: Algorithm to delete an existing item from a data structure.

Data Contextualization has emerged as a critical factor in dealing with and summarizing vast, rapidly changing, and diverse unstructured datasets. Let's break it down.

We all understand the concept of context. For instance, when you're about to commence a meeting with management, you begin by acknowledging them and stating the purpose of the meeting. This establishes the context, ensuring that the audience knows what to expect and how to interpret the upcoming presentation.

Similarly, in the realm of data, Data Contextualization has become a fundamental process, especially when extracting insights from unstructured data. Unstructured data typically contains a substantial amount of information that is irrelevant to decision-making. The relevance of a data element may vary depending on the specific decision being made.

Here's why Data Contextualization is essential:

• Filtering Irrelevant Data: It eliminates irrelevant data, reducing the overall volume of data. This, in turn, simplifies data processing, cleaning, transformation, and maintenance.
• Enhancing Decision-Making: By contextualizing data, you can combine external data with internal data, providing a more comprehensive foundation for decision-making.
• Providing Clarity: It offers a clear rationale for the data, making it easier to comprehend and analyze.
• Adding Points of Interest: Data Contextualization allows for the inclusion of relevant insights or points of interest about the data, facilitating better understanding.

Graph theory is a mathematical framework that is widely applied to solve real-world problems involving interconnected data. It’s used in various fields such as:

• Computer Science: For tasks like network analysis, designing routing algorithms, indexing databases, and building AI models.
• Social Networks: To analyse relationships and influence across social media platforms.
• Biology: For mapping genetic networks and studying protein interactions.
• Transportation: To improve route planning and manage traffic flow efficiently.
• Project Management: To visualize tasks, dependencies, and project schedules.
• Graph theory offers powerful tools for representing, analysing, and solving problems that involve properties, structures and relationships between data elements. It focuses on studying graphs, which consist of vertices (or nodes) connected by edges (or links).
• Vertices (Nodes): These are the basic points in a graph. ‘One node is called Vertex whereas Multiple nodes are called Vertices.’
• Edges (Links): These are the lines connecting the points.