vikrant69g blog

Databricks says they unified OLAP and OLTP. I am skeptical.

LTAP is Databricks' answer to running transactions and analytics on the same engine. The pitch is clever. The implementation details matter more.

Abstract diagram showing unified transactional and analytical data processing architecture with overlapping query patterns

Databricks announced LTAP this week. Lake Transactional Analytical Processing. They claim it unifies OLAP (analytics) and OLTP (transactions) workloads on one engine. No more syncing a Postgres write database to Snowflake for reporting. The pitch is simple. Write your order into the lakehouse. Query yesterday’s revenue in the same table. No CDC pipeline. No replication lag. One storage layer, two query modes. I am skeptical because storage and query patterns fight each other. OLTP wants row-level updates with millisecond latency. OLAP wants columnar scans over billions of rows. You can fake both with enough caching and indexing, but you pay for it somewhere. Databricks built this on Delta Lake with row-level deletes and versioning already in place. The new bit is low-latency point lookups. They claim sub-10ms reads for key-value queries while keeping Parquet scans fast. That requires an index outside the Parquet files. Probably a key-value store bolted on. The HN thread is asking the right questions. What is the write throughput? What happens when the transaction log gets massive? Can it replace Postgres for a real application? My guess is this works for operational dashboards and light transactional workloads. It does not replace a proper OLTP database for high-write SaaS apps. But if your company already runs on Databricks and you need a low-latency lookup layer for enrichment queries, this might be cheaper than spinning up RDS. I want to see the index design and the trade-offs they made for versioning. MVCC is expensive in distributed systems. Someone is paying for those point reads, either in storage overhead or query planning. Still, unifying the stack is valuable. Fewer pipelines means fewer things that break at 3am. If LTAP actually works at scale, it saves the operational tax of syncing two systems. That alone might justify the compromises.


Source: Databricks Launches LTAP: A Unified OLAP/OLTP Data Architecture