Knowledge Hub

Data Surfing is always confused with Data Analysis. There is a very thin line between Data Analysts and Data Surfers.

Data Surfing is normally done by non-technical people like management who doesn’t know how to write any kind of code to access data at database level.

Data Surfers normally stays at the level of Dashboards and Reports where they just click here and there to explore data for required information.

Business Intelligence (BI) tools came into being to cater Data Surfers World where data was required to be presented for non-technical people who could view data for decision making.

Era(s) for Data Surfers

• Static Reports: In the beginning, business users used to share their data requirements with IT who then used to create dashboards and reports. Those dashboards/ reports and data was not changeable, it was in paper format or in excel format or in text format etc., means after Go-Live if business want one more column or if latest data was required then the whole development cycle had to restart from scratch.
• Dynamic Reports: In the second era, BI tools came-in to cater for this gap. To start with, dashboards/ reports were still not changeable, but data was directly connected with backend databases, so data was always latest at front-end. This was a huge jump, as now, to view new data developer don’t have to recreate front-end again and again.
• Adhoc Reports: Latest era is where dashboards/ reports are also changeable. Now, end users and management can drag and drop tables and columns in dashboards/ reports.

As much as Data Filtering is simple as much as it’s so important in Data World.

‘Data Filtering is about reducing the number of rows while pull data from a table.’

On Cloud: In Cloud, resources i.e., compute power or RAM are just click away. In fact, now a days these can automatically increase based on your query requirements. So, if you are using Cloud then, STRICKLY, no one is allowed to run any query either in PROD, UAT or Dev environment without any filter, means in a SQL command one must use WHERE clause as each query is attached with a dollar value. In Cloud, filtering becomes THE FACTOR to maintain resources utilization at minimum, considering as mentioned each resource costs a dollar value.

On-Premises: In contrast of Cloud, resources i.e., compute power or RAM in on-premises systems is limited. To increase resources on premises there is a need for a complete buying cycle by procurement which takes months. With static resources, if someone runs a query without a filter, that query can hold onto all the available resources, locking the rest of the queries. This can create delays in providing other datasets to top management, means delays in decision making, means losing business opportunities, means loss and no one like losses right. In nutshell, in on-premises solutions, filter also is very critical. At the same time developers has the liberty to us any kind of query in development environment as size of data is limited and static.

Data Aggregation is the most used term in our daily lives whether it’s in office or in home, we use it everywhere.

‘Data Aggregation is about converting details into summarized format.’

For example, if someone asks you how many months there are in a year? You won’t be taking names of all the months rather you will answer, there are 12 months.

Referring to the image, if your manager asks how many installations our team has done, you rather sending left side details, you will send right side summarized report to your manager i.e., in-total there are 6 iOS and 3 Android installations.

As we normally use databases or data stores to store data, there are different aggregation functions normally used to summarize the data.

Aggregation Functions as per ISO Standards

• COUNT
• SUM
• AVG
• MIN
• MAX

Few other Aggregation Functions

• APPROX_COUNT_DISTINCT
• CHECKSUM_AGG
• COUNT_BIG
• GROUPING
• GROUPING_ID
• STDEV
• STDEVP
• STRING_AGG
• VAR, VARP

Canned Data is something which is readily available at one click. A dataset prepared based on business requirements i.e., if required in pre-aggregated format, in one accessible format.

The aim of canned data is to have a dataset that needs no more cleansing, no more wrangling, no more reconciliation etc. Aim is, User to just click it and data should be served.

Imagine Canned food, which are prepared and seals to be used anytime, anywhere, by anyone. Most of the canned food is ready to eat. Same is with Canned data, ready to use. OLAP Cubes are a perfect example of canned data where pre-calculated and pre-aggregated data is stored in a multi-dimensional table, ready to be consumed anytime.

‘Canned data can be refreshed with latest data using same existing data element.’

Canned Reports

Just like Canned Data, Canned Reports has the same approach where static designed reports are built for non-tech savvy stakeholders, who just want to click once and see the expected required output.

Adhoc Reports

Whereas Adhoc reports are for tech savvy users who know how to play with the BI tools and who also have business knowledge of data element shared in the tool. Using Adhoc reporting, users can build their own reports, dashboards by drag and drop exposed tables and columns. Users can slice and dice the reports. Users can drill-in and drill-out.

Tools like

• Oracle Forms, SSRS etc. are considered as Canned Reports generators.
• Tableau, Power BI, Qlik etc. are considered as Adhoc Reports generators.
• OBIEE, Cognos etc. are considered as Semi Canned and Semi Adhoc Reports generators.

We have already gone through Analysis and Analytics. Analytics refers to the systematic analysis of data or statistics to uncover meaningful patterns, insights, and trends. There are several types of analytics, each serving specific purposes and addressing different aspects of data analysis.

The major types of analytics include:

• Descriptive Analytics: It describes what has happened in the past, providing a historical overview. For example, summarizing sales data to understand total revenue for a specific period.
• Diagnostic Analytics: It focuses on why a particular event or trend occurred by identifying its root causes. For example, investigating a drop in website traffic to determine the reasons behind the decline.
• Predictive Analytics: It predicts future outcomes or trends based on historical data and statistical algorithms. For example, using machine learning models to predict customer churn or stock prices.
• Prescriptive Analytics: It recommends actions to optimize outcomes based on predictive analytics results. For example, suggesting marketing strategies to increase customer retention based on predictive insights.
• Other types of analytics include:
• Text Analytics (Text Mining): It analyzes unstructured text data to extract insights, sentiment, or patterns. For example, analyzing customer reviews to understand sentiment and identify common themes.
• Spatial Analytics: It analyzes geographic or location-based data to uncover spatial patterns and relationships. For example, mapping crime rates in a city to identify high-crime areas.
• Social Media Analytics: It analyzes social media data to understand trends, sentiments, and user interactions. For example, monitoring brand mentions on social media to assess brand sentiment.
• Web Analytics: It analyzes website data to understand user behavior, track performance, and optimize user experience. For example, tracking page views, bounce rates, and click-through rates on a website.
• Business Intelligence (BI): It involves the use of tools and processes to analyze and visualize business data for decision-making. For example, creating dashboards to monitor key performance indicators (KPIs) in real-time.
• Big Data Analytics: It analyzes large and complex datasets to uncover hidden patterns and insights. For example, analyzing vast amounts of sensor data for predictive maintenance in manufacturing.
• Customer Analytics: It analyzes customer data to understand behavior, preferences, and trends. For example, creating customer segmentation based on purchasing patterns.

Aggregation Functions are used to perform calculations on a set of values to return a single summarized result. These functions are often applied to columns in a database to derive useful insights from the data.

For better understanding:

1. Aggregation Functions in the Data World:

In the realm of data management and analysis, aggregation functions play a crucial role. These functions are specialized tools used to process and summarize data, particularly when dealing with large datasets. Instead of looking at individual data points, aggregation functions allow us to analyze groups of data and distill meaningful information.

2. Performing Calculations on a Set of Values:

Aggregation functions operate on sets of values, which are typically found in columns of a database. Instead of working with individual records, these functions work with groups or entire columns of data. This allows for the calculation of summary statistics, totals, averages, and other aggregated measures.

Turning Insights into Impact

Data storytelling is the art of translating complex data into a compelling narrative that drives action. Just like a well-structured story, data storytelling involves characters, context, risks, and resolution.

1. Engage Key Stakeholders (Characters)

Every good story has characters, and in data storytelling, these characters are your key stakeholders. They include business leaders, data teams, regulators, and end-users. Each stakeholder has different interests and priorities, so engaging them effectively ensures that data initiatives align with real needs.

Example (Banking Sector):

Imagine a bank wants to improve customer satisfaction. The key stakeholders might include:

• Bank Managers (who need insights to improve customer service),
• Customers (who want quick and secure banking services),
• Loan Officers (who need better data to approve or reject loan applications),
• Regulators (who ensure the bank follows financial laws).