Knowledge Hub

Data Compliance is mostly confused with Data Quality and Data Integrity, which is somehow true, as all these terms do share some common features for ONE GOAL i.e., Data Protection.

Data Compliance is to follow certain regulations and standards of Data Security, Data Privacy and Data Integrity, set by different Regulatory Bodies around the World to protect sensitive data and to make data available in certain ways.

Regulatory Bodies set, if not all but, CERTAIN Data Security, Data Privacy and Data Integrity standards. So, having only Data Compliance measures will not protect data 100%, there are many other standards covered under Data Security, Data Privacy and Data Integrity which are required to be in-place for any organization.

Common Regulatory Bodies Frameworks

We won’t be going into details of regulatory bodies like what is required by each but for the audience, let me mention the most famous ones here.

A term very near to Data Security and Data Privacy. Data Sovereignty is about keeping the data secure within the defined boundaries, especially e.g., certain sets of data should not go cross border for certain sectors.

In Data Security and Data Privacy, we make sure data is only accessed by those who need it or who own it. Within the organization, we maintain an ACL (Access Control List) to record the list of all the users with their level of access they can have on certain data sets. For example, which level of access developers can have, which level of access database administrators can have, which level of access department’s manager can have, which level of access a Business User can have, which of access Cost Centre person can have etc. So, in a way what we are trying to elaborate here is, there can be different levels of access granted to different level of roles in an organization to access data.

Encryption is a technique to convert the normal text in the email, text message, databases, data lakes etc. into scrambling format to make it unreadable which is also called 'CipherText'.

‘Encryption and Decryption use to convert and revert data from another format to target confidentiality requirement and use Key to read data.’

This is one of the most known terms in the Data World concerning Security and there is hardly any organization in the World that doesn't use Data Encryption to protect its data from black hawks.

Let's take this opportunity to go one level down.

Which Data to Encrypt

• Data-at-Rest means data sitting in your database, data warehouse, data lake, in-memory in solutions like delta lake, Databricks, snowflake etc. all must be encrypted.
• Data-at-Transit means when data is moving between devices, such as within private networks, through the Internet, or from laptop to thumb drive etc. all must be in encryption mode.
• Data-in-Use means, when data is exposed to end-users which can be in a BI tool or on a website or web portals like Facebook, LinkedIn etc.

Data Encoding and Decoding is a known term but is always confused with Encryption and Decryption.

Just like Encryption and Decryption, Encoding and Decoding convert and revert data from another format using Key.

Importance of Keys

• Encryption and decryption conversion KEYS are used by known entities or individuals.
• Encoding and decoding KEYS are used by unknown entities or individuals i.e., publicly.

Data Encoding and Decoding is normally used by programs or systems internally whereas Encryption and Decryption are used by proprietary expensive software(s).

There are many encoding techniques but major four types are.

1. Analog data to Analog Signals: This technique is used when sender and receiver both are Analog compatible like mic to speakers directly.
2. Analog data to Digital Signals: This technique is used sender is Analog, and receiver is Digital compatible like mic to computer for recording purpose.
3. Digital data to Analog Signals: This technique is used when the sender is Digital, and receiver is Analog compatible like from computer to speakers.
4. Digital data to Digital Signals: This technique is used when sender and receiver both are Digital compatible like data transfer from computer to computer.

Data Subsetting is as simple as taking a small piece of data, with referential integrity in place, for some activity. For example, there is a requirement to build a data science model and it requires data for users aged between 25-35 from X Location having certain buying/ spending patterns. For this requirement, a particular dataset will be extracted for the data science development team to build and train their data models.

Don't confuse Data Subsetting with Data Masking. Data Masking is still required on this smaller piece of the dataset from a security point of view. Having said that, conducting a Data Subsetting exercise is not as simple. One first must understand the functional requirements of the use case, then one must also understand what data is there in the database as whole. Wrongs or partial Data Subsetting will end up producing wrong data model results and will damage trust in data by decision-makers.

Data Rooms or Datastore are both the same entity. Data Rooms are normally used for due diligence of confidential documents like handling Business Mergers or Acquisitions by bidders & advisers, compliance & audit teams.

Traditionally, there are Physical Data Rooms with Physical Documents. These Physical Data Rooms are for restricted people only, secured by guards with a maximum of one or two parties to go in at any given point-in-time which delays the process if multiple parties are using the same Physical Data Room.

With the growing businesses globally, it was a very expensive affair for multiple teams to fly over to be near these Physical Data Rooms. As an alternate and more viable solution, VDRs (Virtual Data Room) came into the picture which are available Online, behind Firewalls and with proper Access Control. VDRs have changed the game, now sellers can invite a bigger audience and teams sitting in one part of the world to participate in a bid process happening in the other part of the world. VDRs have cut down the cost and expedited the process.

DevSecOps is nothing less DevOps but with addition step of security.

Due to the quick and fast deployment cycles, security has become an alarming factor in DevOps, so security has been included right before the creation process and therefore DevSecOps (development, security, and operations) came onto the surface.

Data Entitlement is normally mixed with Data Authorization which is not 100% accurate. There is no doubt, there are few places where both can be used interchangeably but mostly, they should not.

Entitlement is something which you inherit, or which comes as a default or which you get along-with something else whereas Authorization comes because of an action, or it expects an action in future. For example, you are ENTITLED to something and at the same time you are AUTHORIZED to do X, Y and Z things.

Before we understand the difference between Authorization and Entitlement, it’s very importance to understand what Prescriptive and Descriptive behaviors are.

• Prescriptive means, how it should be, and an action is expected.
• Descriptive means, how it is.

‘Authorization has a Prescriptive behavior whereas Entitlement has Descriptive behavior.’

Let’s try to understand with an example, at work you must have heard, you are ENTITLED to 20 annual leaves and maybe 12 medical leaves etc. But when you apply for a leave, your manager has the AUTHORITY to accept or reject your leave.

There are many ways to achieve data privacy like data generalization, data swapping, data masking, data encryption etc., all these techniques change original data which of course is not desired by most. All these are done to ensure data should not fall into wrong hands.

Data Synthetic is something which will really solve the problem statement like anonymize data before sharing with developers, data science teams for data models etc.

Data Synthetic is about generating new dataset using computer simulations. This dataset contains fake data with the right properties or attributes. For example, customer information is required in almost every application or system, but it can’t be shared to protect individual identity. Data Synthetic tools create a dummy dataset for customers with name, email, contact number, family details etc. but nothing related to original data. This way, developers can build the data pipelines and data scientists can build their data models without thinking of breaching any regulatory or compliance clauses.

In simple terms, when we talk about an Embargo, we're usually talking about a rule or limit, especially when it comes to trading or sharing information. Now, if we use "data embargo," it means there are rules or limits on using, sharing, or spreading certain information. This mostly happens for legal reasons, to keep things safe, to protect privacy, or because there are rules to follow.

Let's delve into the concept of an "embargo," especially in the context of data, using simpler language:

1. Embargo Basics: An "embargo" is like a rule that says you can't do certain things. It often happens in trade or when sharing information.
2. Data Embargo: Now, if we add "data" to the word "embargo," it means there's a rule or limit on how we can use, share, or spread specific information. Imagine it's like having a set of rules for how you can talk about or share certain details.

These rules could be there for different reasons: