HTAP

HTAP stands for Hybrid Transactional/ Analytical Processing. It is a new architecture proposed by Garner in 2014.

In previous topics, we have established that there is an OLTP database for operational tasks and then Data Warehouse, Data Lake, Delta Lake, Data Lakehouse, Data/ Business Vault etc. for analysis and analysis. All keep data separately in their own storage for management decisions.

HTAP is proposing to use one system for both transaction and analytical workload. It is offering an in-memory plus columnar approach for analytical processes.

With the advancements in hardware, in-memory and Cloud, Gartner thinks it is very much possible to have both Transactional and Analytical processing on the same database.

Advantages

• No duplication of data as data will not move from the same one operational database.
• Business or Management can make decisions on actual data in real-time, in contrast if data must move from OLTP to OLAP systems.

Challenges

• As this is a very new concept, the skillset is a major challenge.
• How will those datasets be created, which need Business transformation logics to produce business KPI(s)?
• Where data will be cleaned, cleansed, and scrubbed?
• Where data wrangling will be done?
• Data Analysis needs to know the complete operational 3NF data model as there will be a lot of joins required to present data in BI.
• Operational databases are normally row format to support daily operations. How data will be stored in a row as a well columnar format for OLTP and OLAP processing?

HTAP functionality is offered by database companies, such as MariaDB Enterprise, MariaDB SkySQL, Microsoft Azure SQL Database, Microsoft Azure Synapse Link for Cosmos DB, DbAlibaba DRDS, LeanXcale, TiDB, Hubble, MangoDB, Aerospike and many more. [refer from Wikipedia].