Knowledge Hub

Epistemology, a term rooted in the Greek word episteme, meaning “knowledge,” and logos, meaning “discourse” or “communication,” explores the concept of “justified knowledge.” It addresses questions like “What do we know?”, “How can we claim to know something?”, “What validates our beliefs?”, and “How do we ascertain the certainty of our knowledge?”

In the realm of data, these questions take on a new dimension, especially in the Big Data era, where we store vast amounts of raw data, structured, semi-structured, and unstructured. Data science teams apply machine learning or deep learning models to extract insights, turning raw data into knowledge. However, to ensure that these insights align with their origins, a “justification of knowledge” is essential, ensuring that what we extract truly reflects what was expected. This process emphasizes evidence-based validation, where qualitative research relies on the quality of design and data collected.

Scope Creep is a known term among Project Management but is worth discussing it among data folk as scope creep can also impact data architecture and data modelling.

‘Scope Creep are those new requirements which are never formally added in project scope or if added, are without estimating impact of project deadline.’

Scope Creep means adding new requirements without analyzing impact on time and cost. I personally follow the thumb rule of TCS.

‘TCS: Time, Cost and Scope.’

Whenever 1 of the above mentioned 3 pillars of project management is impacted, the remaining 2 WILL also be impacted. Scope Creep is referred as e.g., a new requirement is added to the project scope, but the Project Manager didn’t estimate how much more effort is required and will there be any additional cost involving it. At the end of the day, it may impact Project Go-Live date which no one wants.

Reasons of Scope Creeps

• Deficiency of a Vibrant and Thorough Scope
• Absence of a Stakeholders Concurrence 
• Weak Project Sponsor Decision Management 
• Contradictory Business Users Views
• Less participation of all stakeholders during Project Lifecycle
• Bad Management of Risk and Issues Proactively
• Crossing Prioritizing Requirements
• Lack of Change Management Process
• Bad Efforts Estimation
• Bad Evaluation of new Requirements
• Missing Stakeholders
• Non-identification of Out-of-Scope earlier in Project Lifecycle
• Vague Acceptance Criteria(s)
• Poor Assumptions Management
• Poor Constraints Management
• No Clear Project Deliverables

Hype Cycle of Data Management is confused with Data Management which is not correct as one is about solution and other about enterprise.

Hype Cycle of Data Management is about knowing the pattern of technologies like how past tools and technologies performed, and which are new technologies in trend.

Hype Cycle of Data Management assist organizations to plan its technologies roadmap, it assists to take decisions like which tools and technologies to adopt and which not to adopt.

There are many a times when organizations are reluctant to adopt new technologies and they end up like Kodak and Nokia.

There are many companies which prepare these kinds of statistics and share with CTO(s), CDO(s), CFO(s), CEO(s) etc.

Yes, you are right, you might not have heard about this approach. It is something new which Myself is working on.

Let’s decode it…..

Productionizable Viable Product (PVP): A Revolutionary Approach to reduce TTM (Time to Market) of adopting cutting-edge Data & Analytics Technologies while minimizing CAPEX (Capital Expenditure).

‘PVP focuses solely on new data & analytics initiatives rather than enhancements.’

Productionizable Viable Product (PVP) approach represents a distinctive approach in optimizing IT project execution by employing a singular Production environment throughout the project lifecycle. This streamlined architecture is designed to alleviate the complexities associated with managing multiple environments. To address data privacy concerns, the PVP architecture integrates controlled measures such as data masking or anonymization during the initial phases of development and testing, ensuring adherence to regulatory standards. The incremental provisioning strategy is a pivotal aspect, commencing with the provision of minimum required services or tools in the Production environment. The architecture emphasizes iterative testing cycles within the Production environment, allowing for continuous validation of integrated code sets through incremental Testing. This cyclic Testing in Production ensures thorough scrutiny before the final release to end users. In essence, the PVP architecture provides a structured and controlled framework, optimizing workflows, reducing complexities, and enhancing the reliability of Analytics project execution.

Foresight and Futurology are used interchangeably. And yes you are right this new term is not something we have never encountered before.

Let’s decode it…..

Data: Refers to information, facts, or statistics collected for analysis, reference, or computation.

Futurology: It's the study or soothsaying of possible futures. It involves examining current trends and making prognostications about unborn developments.

Combining these," Data Futurology" suggests concentrating on the future of data, in terms of how data is collected, reused, anatomized, and its impact on colorful aspects of society or technology.

Let go little deeper about conception of" Data Futurology," which involves the study and soothsaying of possible futures related to data.

Let's break it down:

• Study or Forecasting of Possible Futures
• This refers to the practice of examining current trends and making predictions about future developments. It involves analyzing data and information to anticipate how various aspects of a particular field might evolve over time.
• Data Futurology
• This term specifically focuses on the future of data. It encompasses the entire data lifecycle, including how data is collected, processed, and analyzed. Additionally, it explores the potential impact of data on different aspects of society and technology.

Approaches play a vital role in the failure of many projects in the IT World. We all have heard about POC, Protype, Pilot, MVP, SIT, UAT, and Production implementations, but I am sure, most of us might have not given a thorough thought of aftereffects, if one is interchanged with other.

Let’s try to understand everything in the upcoming topics but to give a broader perspective all above approaches have their own implementation strategies. Most of the projects fail when e.g., started with a POC and once the POC is successful, move the same in Production. One must not do that. This normally happens when business push implementers to achieve tight timelines.

Please note, small level of implementation which in our example, POC might be successful with limited infrastructure or software, but 99% chances are when it goes in Production, it’s bound to fail. Same is true for Prototype, Pilot and MVP environments. One must plan infrastructure and software based on, if plans to be in Production.

POC stands for Proof of Concept. POC normally follows Sales Context of the product.

In POC, output is most important, not technology. In POC, the implementers make sure output is fulfilling customer requirements and at the same time convince business that this is the solution to go with.

The Goal of POC is to make sure whether Idea is Feasible or not.

‘POC is built and run for Business Users.’

Protype is the next stage of POC. In POC, one has test that yes Idea is Feasible, and it will benefit business agenda.

Now, in Prototyping, tools and technologies comes into picture where one starts converting Ideas in Reality, where it will show how the end-result will look like. It will show, how product will look like. It will show the real end solution.

Prototype is not built considering that business users will jump in and start using it. Prototypes are normally for technical folks to evaluate the end results with dumpy but relevant data.

’Prototype is built and run for Technical Users.’

New name of Pilot is MVP 😊. MVP is the next stage of Prototyping where Tools and Technologies are tested, and end solution is tested by technical folks. Now in MVP, subset of actual requirements is considered, subset of real data is used, subset of hardware and infrastructure is used, subset of software is used, and limited number of features are implemented.

Once MVP is ready, it’s released to actual users to start using it. This gives business users a taste of the actual solution. The biggest advantage of this approach is, with small implementation, with small investment, business can taste the blood and take a decision to go to the next stage or drop the solution.

Now a days, many organizations divide complete solution into multiple MVP(s) i.e., by implementing solutions in chunks, this is also supported by Agile approach where product gets ready in shorter timeframes and then keeps maturing with time. In MVP, just like actual implementation, three environments are spin i.e., DEV, UAT and PROD, along with all the steps like DevOps, CI/CD, Metadata Management, Data Quality, Data Governance, Data Lineage, Master and Reference Data Management, and many other steps which are required to go to Production.

In MVP stage where actual Tools and Technologies, and SIT, UAT, and PRODUCTION environments are implemented on a small scale.

Once MVP is successful, the next stage is to gather complete business requirements, a complete designed is made, teams are built, project plan is made, end-to-end SOW is signed, development is conducted, testing is done i.e., UT, SIT and UAT, and solution goes live.

On contrast with POC, Prototype and Pilot/ MVP, in this stage a complete Infrastructure, Software and Data is prepared and used.

UT stands for Unit testing and is done by developer during development phase for every piece of code developed. The better the UT is, the better the product will be.

SIT stands for Systems Integration Testing. Now a days, SIT is disappearing, SIT is the first step to save cost which is NOT a good idea. I am sure in a decade, newbies in IT World might not even know what actually SIT is. Once complete solution is ready, it’s moved to another environment which do have all the possible components of the actual solution but in term of size, it is small e.g., if based on data there is 10 servers and 100 licenses are required, SIT can be run with 1 server and with 1 license. In SIT, we make sure code can run end-to-end, user access is properly set, security protocols are in-place, jobs dependencies are in-place etc.