Knowledge Hub

DLP stands for Data Loss Protection/ Prevention which is part of the organizational level security strategy.

‘DLP is to protect or prevent data from getting in wrong hands.’

DLP is normally handled and maintained by the Risk department in an organization where standards and policies are followed to protect data from intruders. We know, the starting point of all the standards and policies start from Regulatory bodies.

The moment you expose your data to outsider via Internet, the DLP comes into action. One must take care of their own data from intruders and hackers. In previous era we had the choices to connect to external world or not, but now NO, we don’t have a choice, we must connect to social media to pull data, we must connect to cloud etc.

Most people confuse Quality and Data Integrity with each other.

The term Data Integrity came from a security point of view, but it came from sharing data in a more effective and easier way, so in a way, it is more than just security. It has multiple components and Data Quality is one of those. In other words, these are not two different terms, nor both compete, rather Data Quality supplements Data Integrity along with other components.

Achieving a higher % age of Data Integrity gives confidence to the business users and management to trust data for daily decision making. Losing Data Integrity means a higher possibility of wrong data, wrong data means wrong decisions, and wrong decisions mean business lose margin or go into loss. Security’s point of view is to make sure the results shared with Public Stakeholders or Regulatory Bodies are correct and not fake. That's the reason there are internal and external audit teams evaluating the Data Integrity of organizations. Based on audit results, more investors invest, more buyers buy, and the company flourishes.

Data Compliance is mostly confused with Data Quality and Data Integrity, which is somehow true, as all these terms do share some common features for ONE GOAL i.e., Data Protection.

Data Compliance is to follow certain regulations and standards of Data Security, Data Privacy and Data Integrity, set by different Regulatory Bodies around the World to protect sensitive data and to make data available in certain ways.

Regulatory Bodies set, if not all but, CERTAIN Data Security, Data Privacy and Data Integrity standards. So, having only Data Compliance measures will not protect data 100%, there are many other standards covered under Data Security, Data Privacy and Data Integrity which are required to be in-place for any organization.

Common Regulatory Bodies Frameworks

We won’t be going into details of regulatory bodies like what is required by each but for the audience, let me mention the most famous ones here.

Data Preservation, Data Security, Data Privacy, Data Integrity, Data Compliance all refer to the protection of the data.

Data Preservation is about securing and maintaining the safety and integrity of the data after it’s been processed, for future reference.

‘Data which is preserved cannot be changed and hardly purged or deleted.’

Things to consider for Data Preservation

• What data to be preserved? Not all data is worth preserving.
• What is the retention period to preserve data? Like for how long to keep data.
• Who will be responsible for preserving data? A custodian is required for data repository.

In the current era with big data having huge semi and unstructured datasets, it’s not worth rather not possible to preserve everything. One must decide what to preserve and for how long.

For example, in datastore like Data Warehouse or a Data Mart or in Data Lake etc., there are certain datasets to preserve. One can start from the list of compliance requirements e.g., traditionally in banking sector, the customers who has left the bank or in other words closed their accounts, their history is preserved or saved for 7 years from the day they closed their accounts.

A term very near to Data Security and Data Privacy. Data Sovereignty is about keeping the data secure within the defined boundaries, especially e.g., certain sets of data should not go cross border for certain sectors.

In Data Security and Data Privacy, we make sure data is only accessed by those who need it or who own it. Within the organization, we maintain an ACL (Access Control List) to record the list of all the users with their level of access they can have on certain data sets. For example, which level of access developers can have, which level of access database administrators can have, which level of access department’s manager can have, which level of access a Business User can have, which of access Cost Centre person can have etc. So, in a way what we are trying to elaborate here is, there can be different levels of access granted to different level of roles in an organization to access data.

Data Virtualization is the virtual consumption layer and connect with anonymous data stores i.e., RDBMS, data appliance, NoSQL, Web Services, SaaS, and enterprise applications etc. As it’s connecting directly with all data stores, the data models of all systems remain intact with no change.

‘It’s considered as one of the solutions for Data Sovereignty problem statement.’

What data stores?

Here data stores are referred as traditional RDBMS or OLTP systems, OLAP systems, Data Warehouse, Data Lake, Delta Lake, Data Vault, Business Vault, Kafka, data appliance, NoSQL, Web Services, SaaS, and enterprise applications etc. Yes, it’s confusing as we all know above data stores serve their purpose for one or the other business requirement then why this another layer on top?

Data Virtualization hosts views connecting directly with all possible data stores. These views can be one-to-one or can be complex SQL written views to pull data from multiple source systems to create a virtualized layer on top of consumer layer to consume.

Data Federation works just like Data Virtualization framework but what is the difference between both.

Relationship between Data Federation and Data Virtualization is just like relationship between Data Mining and Data Science (there is separate topic on Data Mining vs Data Science).

Conceptually, both Data Federation and Data Virtualization works the same way, but Data Federation is restricted to relational database whereas Data Virtualization also includes data appliance, NoSQL, Web Services, SaaS, and enterprise applications etc., as well.

Another major difference i.e., Data Federation doesn’t create a unified data model whereas Data Virtualization does.

Below are a few similar behaviors of both Data Federation and Data Virtualization.

• Single Access Point: Access all data stores from a single environment.
• UAM: Use Access Management at one place.
• Data Quality Management: Data Quality issues can be identified from a single point.
• No Data Movement: No ETL/ ELT is required.
• Ease of use: As users don’t have to write queries in different languages. Users can write in one language, and it will create a wrapper to talk to all the connected systems.
• No Additional Storage: As a virtual copy there is no cost for any separate copy of data, so no additional storage is required.
• Low Risk: As it’s not hosting the data copy as compared to Data Warehouses or Data Lake where data is physically moved, there is no chance of missing any data.
• Better for Analytics: As all the source data models are connected and data is fetched in real like, it makes source data available in real-time for Data Analysts and Data Scientist
• Performance Impact: Some impact on source systems as some queries will run at source.
• Functional Knowledge: Seek dedicated involvement by Data Stewards and Functional consultants.
• No Historical Data: As there is no copy of data hosted so history data is also not maintained. These are best suited along with Data Warehouse in-place.

Encryption is a technique to convert the normal text in the email, text message, databases, data lakes etc. into scrambling format to make it unreadable which is also called 'CipherText'.

‘Encryption and Decryption use to convert and revert data from another format to target confidentiality requirement and use Key to read data.’

This is one of the most known terms in the Data World concerning Security and there is hardly any organization in the World that doesn't use Data Encryption to protect its data from black hawks.

Let's take this opportunity to go one level down.

Which Data to Encrypt

• Data-at-Rest means data sitting in your database, data warehouse, data lake, in-memory in solutions like delta lake, Databricks, snowflake etc. all must be encrypted.
• Data-at-Transit means when data is moving between devices, such as within private networks, through the Internet, or from laptop to thumb drive etc. all must be in encryption mode.
• Data-in-Use means, when data is exposed to end-users which can be in a BI tool or on a website or web portals like Facebook, LinkedIn etc.

Data Encoding and Decoding is a known term but is always confused with Encryption and Decryption.

Just like Encryption and Decryption, Encoding and Decoding convert and revert data from another format using Key.

Importance of Keys

• Encryption and decryption conversion KEYS are used by known entities or individuals.
• Encoding and decoding KEYS are used by unknown entities or individuals i.e., publicly.

Data Encoding and Decoding is normally used by programs or systems internally whereas Encryption and Decryption are used by proprietary expensive software(s).

There are many encoding techniques but major four types are.

1. Analog data to Analog Signals: This technique is used when sender and receiver both are Analog compatible like mic to speakers directly.
2. Analog data to Digital Signals: This technique is used sender is Analog, and receiver is Digital compatible like mic to computer for recording purpose.
3. Digital data to Analog Signals: This technique is used when the sender is Digital, and receiver is Analog compatible like from computer to speakers.
4. Digital data to Digital Signals: This technique is used when sender and receiver both are Digital compatible like data transfer from computer to computer.

Data Subsetting is as simple as taking a small piece of data, with referential integrity in place, for some activity. For example, there is a requirement to build a data science model and it requires data for users aged between 25-35 from X Location having certain buying/ spending patterns. For this requirement, a particular dataset will be extracted for the data science development team to build and train their data models.

Don't confuse Data Subsetting with Data Masking. Data Masking is still required on this smaller piece of the dataset from a security point of view. Having said that, conducting a Data Subsetting exercise is not as simple. One first must understand the functional requirements of the use case, then one must also understand what data is there in the database as whole. Wrongs or partial Data Subsetting will end up producing wrong data model results and will damage trust in data by decision-makers.