Knowledge Hub

This term is normally known by the Security department and is tagged with Hackers.

‘Data Scraping is also known as Web Scraping.’

The minor difference between both terms is Data scraping means searching the location of the data and then extracting it. In other words, it does not copy and paste rather extract data in a precise and accurate manner whereas Web scraping involves extracting exactly everything from web pages.

Data Scraping or Web Scraping is the process of importing information from a website into a spreadsheet or local file saved on your computer.

Popular uses of data scraping include:

• Research for web content/business intelligence.
• Pricing for travel booker sites/price comparison sites
• Finding sales leads/conducting market research by crawling public data sources (e.g., Yell and Twitter)
• Sending product data from an e-commerce site to another online vendor (e.g., Google Shopping)

It allows us to take unstructured, scattered data from multiple sources and collect it in one place and make it structured for better use.

MLOps is also from the same family of DevOps, DevSecOps, DataOps, etc. Just like DataOps has a focus on data, MLOps focuses on Machine Learning Model’s end to end journey.

As the name says itself, MLOps (Machine Learning Operations) means from the availability of raw data till the ML model is deployed in PROD, this whole journey is called MLOps.

Please note, CI/ CD is a very critical part of MLOps as in Machine Learning world, 90% models do not go to Production. Most of the models are created, trained, and decommissioned due to N number of reasons. CI/CD plays a vital role of moving models from development to training area again and again and once confirmed by the owner, moved to Prod.

As mentioned, MLOps was extracted from the concept of DevOps, it follows the same pattern but has a different agenda.

The key phases of MLOps are: [These are taken from Best Practice on Databricks, Google and few other places.]

DLOps is like MLOps and belongs to Data Science domain.

Before we try to understand DLOps as compared to MLOps, one must understand the difference between Machine Learning and Deep Learning. (There is separate topic on difference between Machine Learning and Deep Learning)

‘It won’t be wrong to say, MACHINE LEARNING is for simpler and more structured & semi-structured use cases whereas DEEP LEARNING is for complex and unstructured use cases.’

Let’s try to understand with a simple example. If we need to find out how many cars, bikes or buses there are in structured or semi-structured datasets then it’s relatively easy as data is in rows and columns format. We can have the width, length, and height of each car, bike or bus so using MACHINE LEARNING models we can easily ready this dataset and produce expected output. But what if we give pictures to ML models to extract the same result, will it be possible? The answer is No as ML models normally work on structured or semi-structured datasets. To find out cars, bikes, or buses from pictures, DEEP LEARNING models are used which can recognize the objects in the pictures and find out which presents car or bike or a bus.

There is a very thin line between ModelOps and MLOps. ModelOps is the next stage of MLOps where focus is more towards automating, retraining, and maturing models on its own.

MLOps mainly focuses on model management, model monitoring and CI/ CD whereas ModelOps mainly focuses on addressing and sharing AI performance to the stakeholders without the need of Data Science and Machine Learning Team members.

Few differences between ModelOps and MLOps.

• MLOps focuses on ML models operationalization whereas ModelOps focuses on decision models and AI operationalization.
• MLOps focuses on continuous cycle of development, implementation, and monitoring of models whereas ModelOps focuses on life cycle management and governance of models.
• MLOps focuses on creating models for AI enabled applications whereas ModelOps focuses on AI transparency and usage of KPIs by business users.

In nutshell, ModelOps supports AI capabilities where involvement of Data Science become less and less as Models run on their own, retrain on their own, deploy on their own, extract business KPIs on their own etc.

AIOps was initiated by Gartner in 2016. It supports DevOps, DataOps, MLOps, DLOps and ITOps together.

AIOps came to surface after the inception of Big Data era where huge data started to flow in, where DevOps, DataOps, MLOps, DLOps rather every kind of operational tasks had tripled. And as we know everyone has recently realized the importance of Data so now everyone wants to progress in data ignoring the fact that automation requires processes and governance etc. That is why AIOps came into picture where everything is handled by the system itself.

AIOps is about assisting DevOps, DataOps, MLOps and ITOps etc., to work seamlessly. AIOps uses algorithms to make things happen like preventing outages, maintaining uptime and assure continuous services. Unlike DevOps, MLOps and ITOps where human interactions are still required, here AIOps makes sure everything runs on its own.

As mentioned, AIOps handles end to end operations on it owns, below are few stages.

These are one of the most confusing terms in the data domain. Both are mostly confused with each other.

The major difference between both is.

• Data Mining is done on ONLY structured data.
• Data Science is done on structured, semi and unstructured data.

Data Science is mostly tagged with Big Data and Data Mining is tagged with Data Warehousing. There are many definitions and comparisons for both in the market as both do machine learning, deep learning, data cleansing, statics, and math algorithms etc.

‘Data Science is the Superset of Data Mining.’

Data Mining is also known as KDD (Knowledge Discovery in Data). First it was performed in excel, then with the introduction of DWH, an automated process started discovering historical patterns and other valuable information from large data sets which are structured.

Now with the introduction of Big Data, Data Lake, NoSQL databases, semi and unstructured datasets, the magnitude of the data size has taken a huge step ahead and Data Science has come onto the surface.

We all know what Machine Learning (ML) models are right? ML is about using raw but structured or semi-structured dataset, then creating feature tables which are no more than traditional tables but with cleansed data. Later, ML models use this cleansed dataset to classify data and produce expected results.

Same way, Deep Learning models is about using raw but unstructured dataset where no one will know what will flow in. Deep Learning models dig out required data from that unstructured dataset to prepare cleansed dataset in shape of feature tables to classify for expected results.

For example, if we need to find out how many cars, bikes or buses there are in structured or semi-structured datasets then it’s relatively easy as in rows and columns format. We can have the width, length, and height so using ML models can easily ready this dataset and produce expected output. But what if we give pictures to ML models to extract the same result, will it be possible? The answer is No as ML models normally work on structured or semi-structured datasets. To find out cars, bikes, or buses from pictures, Deep Learning (DL) algorithm and models are used which can recognize the objects in the pictures and find out which presents car or bike or a bus. Please note, the next stage is supervised and unsupervised learning.

Supervised and Unsupervised Learning categorizes the dataset into two where there are situations where sometimes we know or sometimes, we don’t know what’s in there.

We have discussed this on another topic i.e., Data Classification, Categorization and Data Clustering but as Supervised and Unsupervised Learning are very important to understand Machine Learning, let’s understand it again here.

In Machine Learning, there are Supervised and Unsupervised Learning methods.

Both terms are used for pattern identification before complex algorithms come into action.

Supervised Learning is done based on the uniqueness of the data where e.g., all animals are tagged separately. Whereas Unsupervised Learning is done based on the characteristics we can create clusters e.g., 1) Animals with 2 legs 2) Animals with 4 legs.

• Supervised Learning uses labeled data whereas Unsupervised Learning uses unlabeled data.
• The most popular Supervised Learning algorithms in data mining are the K-Nearest Neighbor and decision tree algorithms.
• The two common Unsupervised Learning algorithms in data mining are K-means clustering and hierarchical clustering.
• Supervised Learning output is known.
• Unsupervised Learning out is unknown.
• Training data is provided for Supervised Learning.
• No training data is provided for Unsupervised Learning.

In the current era, people are confused between AI vs Data Science.

‘This is a very controversial topic.’

If you are clear about Data Science, then understanding AI will not be a problem. In the current era, with huge datasets either its structure, semi, or unstructured data, we create Data Science Models to analyze and predict. So, when it’s done by Humans, it’s called Data Science but when models are matured and are passed on to the systems or machines to run itself, train itself, fix itself, analyze itself and predict itself then it becomes Artificial Intelligence (AI).

AI is much more than that but here aim is not to explain or elaborate in detail like what AI is, here aim is to explain the difference between AI and Data Science at a very high level.

To understand Data Algorithms, let's quickly know what an algorithm is. It is a procedure consisting of multiple steps with a set of instructions for a desired result. It is independent of any language, in other words, an algorithm can be implemented in any language.

Algorithm Characteristics

• Unambiguous: It should be clear and unambiguous. Each of its steps, and its inputs/outputs should be clear and must lead to only one meaning.
• Input: It should have 0 or more well-defined inputs.
• Output: It should have 1 or more well-defined outputs and should match the desired output.
• Finiteness: It must terminate after a finite number of steps.
• Feasibility: It should be feasible with the available resources.
• Independent: It should have step-by-step directions, which should be independent of any programming code.

Now from a data point-of-view, how the algorithm behaves.

• Search: Algorithm to search an item in a data structure.
• Sort: Algorithm to sort items in a certain order.
• Insert: Algorithm to insert an item in a data structure.
• Update: Algorithm to update an existing item in a data structure.
• Delete: Algorithm to delete an existing item from a data structure.