Knowledge Hub

On-Premises and Cloud both compete but at backend both operate the same way.

Since years and years Cloud is trying to make a mark but some or the other reason it couldn’t. Now things have changed, in recent era, Cloud has become arguably the biggest buzzword after the Covid-19 pandemic in 2019/20, as organizations had to enable Cloud services for their employees to support their businesses.

Let’s try to understand the core difference, to use IT solutions there are always two options either to have all the software(s) on On-Premises or on Cloud.

On-Premises: When an organization buys servers and infrastructure, and places those inside their own location or building under their own control, managed by their own internal teams, it is called On-Premises.

Cloud: Now when someone else places servers and infrastructure in their location or in their building and manages it for customer where customer accesses those servers and infrastructure via internet, it is called Cloud.

In IaaS, online compute power and storage is provided to customers in the shape of Virtual Machines (VS). Customers must bring their own software, tools and IT Solutions and then manage the environment, from installing/ upgrading software, HA, DR, compute power, storage standard etc., on its own. Cloud Service provider charges based on compute and storage usage.

Cloud example segregating IaaS:

If you only expect Me (Service Provider) to buy laptops (Infrastructure or Servers) with ONLY having Operating System of your choice and give you access and all rest of the activities like to install your special software(s) on it, install security software(s), enable software encryption, replaces normal hard disk with encryption disk, install all MS Office software(s) on it for daily activities, install database on it, configure Disaster Recovery and High Availability Strategy like taking backup on an external disk and keep moving data on periodic basic on second laptop so just in case if first laptop crashes, work can be continued on second laptop, periodic upgrading all the software(s) on both laptops etc. etc. etc. all will be take care by yourself is IaaS.

We have discussed differences between On-Premises and Cloud then IaaS where only Hardware is rented by Cloud providers. Next stage of this paradigm change is to rent Containers at a small scale, to save cost and for better management of resources. CaaS (Container as A Service) is one level lower than IaaS to rent resources on Cloud.

As the name says itself, CaaS (Container as A Service) is about deploying code in a container and deploying that container in Cloud. This saves us to manage hardware on our own in Cloud i.e., IaaS, so we can deploy our containers and underneath it, the hardware will be managed by Cloud providers. Most of the Cloud providers are offering CaaS.

You are using Agile Methodology then you need CI/ CD (explained separately). In Agile, most of the time we have multiple teams working in silos on different modules on the same project and can be on the same deliverable. CaaS can be very helpful in this methodology where different teams are able to develop and deploy their code set any time using CI/ CD pipelines.

As we move from IaaS to CaaS to PaaS, it becomes confusing. In IaaS and CaaS, we established that Infrastructure/ Servers and Storage is provided by Cloud providers and rest of all the activities are managed by customer team, itself. Now, let’s try to understand PaaS where all software(s) are made available for customers by the Cloud provider. Software(s) are already installed and ready to use. Customers don’t have to manage any compute or storage; everything is managed automatically by Cloud provider.

If you expect Me (Service Provider) to buy laptops (Infrastructure or Servers) with all the possible software(s) pre-installed and made those available to you via Internet, ready to be used in few clicks. Along with software(s) all rest of the activities like to install security software(s), enable software(s) encryption, replaces normal hard disk with encryption disk, install all MS Office software(s) on it for daily activities, install database on it, configure Disaster Recovery and High Availability Strategy like taking backup on an external disk and keep moving data on periodic basic on second laptop so just in case if first laptop crashes, work can be continued on second laptop, periodic upgrading all the software(s) on both laptops etc., all will be taken care by Me (Cloud Provider) is PaaS.

We have discussed IaaS and CaaS where only Infrastructure and Container level resources are rented on Cloud. Next stage of this paradigm change is to deploy Functions ONLY and pay for resources required to run those only. FaaS (Function as A Service) is one level lower than CaaS to rent resources on Cloud.

One of the biggest challenges to date is, whenever there is a change in the requirement, the whole code-set must redeploy. When code changes, it was always challenging to redeploy ONLY the changed code as there are always dependencies. Breaking the bigger code set into smaller pieces of code sets i.e., functions, can assist in mitigating this challenge wherever possible. Agree, in Data Pipelines it might be difficult but worth having this option to be utilized whenever possible.

As the name says itself, FaaS (Function as A Service) is about deploying ONLY Functions in Cloud. This saves us to manage Infrastructure on our own in Cloud, now we can deploy our Functions ONLY and underneath it the infrastructure will be managed by Cloud providers. Most of the Cloud providers are offering FaaS.

As we move from IaaS and PaaS to SaaS, it becomes very confusing segregating PaaS and SaaS, but please hold on to your horses’ 😊 we are going to nail it down in this topic.

To understand this example, pre-requisite topics are 1) ‘On-Promises vs Cloud’ and 2) IaaS and 3) PaaS.

In IaaS, we established that only Infrastructure/ Servers and Storage is provided by Cloud providers then in PaaS we establish along with Infrastructure/ Servers and Storage, Software(s) BUT EMPTY Software(s) are also made available along with rest of all the activities to be managed by the Cloud providers EXCEPT NO IMPLEMENTED SOLUTION.

In SaaS, there are pre-built or OOTB Solutions available in the Cloud. Please note, it’s not mandatory that all solutions are owned by Cloud providers, most of the solutions are created and maintained by the third parties. Third Parties use compute and storage from Cloud providers and build their own solutions and sell to customers. Now, customers just need to log-in into the Solution in Cloud as a Service that is why it’s called Software as A Service. A solution is owned and its management like installation/ upgradation, compute, storage, HA, DR etc., all are taken care of by the Solution Owner, again which can be by Cloud Providers or any Third Parties. Customers just need to subscribe to the SaaS service and start using the Solution.

Data As a Service (DaaS) is very near to SaaS service in Cloud. We know what SaaS is, it’s about pre-built or OOTB solutions available in the Cloud. Solution providers having their solutions in Cloud with API(s) enabled from where they allow customers to connect and pull data on demand who are then billed based on the data they pulled, this service is called DaaS.

For example, a company builds a solution on cloud where it pulls data from Facebook, Amazon, Ali Baba, etc., and using ML Modelling, predict customers buying behavior, customers surfing behavior, customers segmentation, products categorization, customers demographics etc., this analysis and analytics are then sold as DaaS to organizations in need for these details for marketing or sales planning etc.

The main difference between in SaaS and DaaS is, in SaaS, customer use whole vendor solution on Cloud e.g., customer logs in into vendor solution online and start using it whereas in DaaS service, customer can connect or subscribe to pull the data ONLY.

Please note, AAaaS has nothing to do with Cloud, it can be on On-Premises or On-Cloud. AAaaS is an environment setup for Data Scientists to play around with Machine Learning and Deep Learning Modelling.

Advance Analytics as a Service (AAaaS) is a new concept and still maturing, due to which it is still very confusing. Different companies are offering different approaches, so there is no one best approach.

Two approaches been followed.

1. AAaaS as PaaS: Modelling tools like Databricks, Azure AML etc., are available as pre-installed services (refer to topic: PaaS) where customers can provision these tools in few clicks and simply start developing their Machine Learning or Deep Learning Models to build their own solutions. Here, customers must have their own MLOps along with CI/CD.
2. AAaaS as SaaS: In this approach, Solution Providers hosts pre-built or OOTB Machine Learning and Deep Learning Models. These are offered to customers to consume only. Customers inject their own dataset for relevant data analysis and analytics. Here, customers must follow Solution Providers MLOps along with CI/CD.

I am sure most of us know this term but not sure how accurate you are in terms of its importance.

‘The granularity of data means, the levels of data breakdown.’

Generally, people think having all relevant details about an entity is Granularity, but the question is what level of details?

For example, details can be name, email, and contact number, date of birth, color, blood group, home address, company, role, salary, father name, mother name and so. Is this the lower level of granularity? NO, when we say the lowest level of granularity, we can break down.

• Name into First Name, Middle Name and Last Name
• Address into Apartment Number, Building Name, Street, Area, City, District, Region, Country, Postal Code etc.

In OLTP systems, granularity must go as low as possible and in OLAP systems, it can depend on the business use case.

Now, when we have established what the Granularity is, let's discuss when it's needed. Data Wrangling, Data Mining, Data Science etc., all use raw data for analysis and analytics. And when we say RAW, the lower the level of granularity can be, the better the model result will be for DSS.

As the name says itself, these are traps. If we have Chasm or Fan Traps, then we have a serious problem as these can impact the whole data model.

‘Chasm and Fan Trap lead to Cartesian Data.’

Fan Trap: In Fan Trap, the model ends up having ‘One to Many’ relationship in more than two tables e.g., One Customer having Many Orders and One Order having many Order Products which is logically fine but if physical data model is created like 1) Customer with Order ‘One to Many’ 2) Order with Order Product also ‘One to Many’ then it’s not fine. It’s a Fan Trap.

• Issue: In this scenario, Order Id (Pkey) has been added in Order Products tables as Fkey which means there will be Low Cardinality in Order products tables means there will be duplication of Product in Product’s table which is a wrong approach. In the Products table, Product ID should be unique.
• Solution: One Product can have Many Orders but not the other way. The correct approach is to have Product ID (Pkey) from Order Products table as Fkey in Orders table.