Why Databricks can’t match ClickHouse Cloud for real-time analytics

SHARED HARNESS

Both systems used the same source dataset, schema, workload definitions, pacing model, query schedule, and result format. Destination-specific adapters handled delivery and timing. Part 1 explains the shared methodology; the Databricks benchmark contract records this provider’s implementation.

DATASET AND PREPARATION

The complete NBBO dataset contains 113,219,565,734 narrow, 12-column rows. Raw tables used (sym, t), stock symbol and event timestamp; daily summaries used (sym, day), stock symbol and UTC day. The summaries maintained the counts, sums, and price minima and maxima used by the aggregate queries.

CONTINUOUS WORKLOAD

Fresh rows were paced toward 1 million per second while aggregate and drill-down queries ran on their ten-minute and hourly schedules. This models data being generated continuously and prepared as it arrives, rather than measuring how quickly an existing dataset can be bulk-loaded or backfilled. The Databricks MV path retained its native snapshot semantics; matching ingestion progress does not make both systems’ summaries equally fresh.

RECOMMENDED STACK AND SIZING

ClickHouse Cloud used a 16-CPU, 64-GiB read node. Databricks used an X-Small Serverless SQL warehouse fixed at one cluster, approximately 16 worker vCPUs; this sizing does not imply identical hardware or memory. Liquid clustering, Predictive Optimization, and triggered incremental MV refresh were enabled. Lakehouse//RT was unavailable in the test workspace, so the accepted comparison uses the Serverless SQL baseline.

① PREPARATION COST

The full ingest includes ClickHouse’s complete two-node HA ingest service and Databricks’ Zerobus ingestion, asynchronous clustering, and MV refresh. Databricks ingestion and refresh use allocated DBU usage; clustering uses the accepted Predictive Optimization DBU allocation. All three appear in the preparation-cost model.

② QUERY COST AND ③ RUNTIME

Each accepted query’s recorded duration is priced at its read-side compute rate and added to accumulated runtime. Databricks uses Query History total duration, excluding result fetch; ClickHouse uses its recorded runner duration. These normalized costs represent accepted query work, not complete warehouse invoices. The query-comparison and pricing notes below detail the timing and boundaries.

TARGET VERSUS OBSERVED RATE

Databricks completed the durable-ingest window in about 31.5 hours, averaging approximately 1 million rows/s. ClickHouse completed its ingest in about 36.75 hours, averaging approximately 0.86 million rows/s. The target was shared; observed progress differed. Databricks’ final reconciliation confirmed all 113,219,565,734 rows with no duplicate surplus. Durability acknowledgment and table query visibility are separate milestones.

REPORTING WINDOW

Per-query latency charts stop at 100 billion rows. Preparation costs cover the full active ingest; accumulated query cost and runtime use the accepted active-ingestion observations, ending at each workload’s shared comparison horizon. Post-ingestion queries and maintenance are outside the headline score. Displayed totals are rounded.

PRICING BASIS

ClickHouse Cloud uses $0.3903 per compute unit (CU) per hour. Databricks uses checked-in public list rates for AWS eu-west-1, Premium: $0.39 per DBU for allocated ingestion and maintenance, and $0.91 per DBU for Serverless SQL. The accepted summary preserves full precision; these are modeled list-price costs, not a historical invoice reconstruction.

① COMPLETE PREPARATION COST

This covers the active ingest of all 113,219,565,734 rows, including the required preparation components.

CLICKHOUSE CLOUD

2 CUs × 36.75 hours × $0.3903/CU-hour = $28.68705, including both HA ingest nodes.

DATABRICKS INGESTION

Allocated Zerobus usage, 1,098.9261167 DBUs × $0.39 = $428.58119. This accepted DBU allocation is used in the headline rather than converting the source’s byte volume into a pricing proxy.

DATABRICKS CLUSTERING

The Predictive Optimization allocation contains 151.8613011 DBUs × $0.39 = $59.22591. This component uses Predictive Optimization’s operation-level DBU allocation. It is a modeled maintenance charge rather than an invoiced amount.

DATABRICKS MV REFRESH

532.8145064 allocated DBUs × $0.39 = $207.79766. Together, ingestion, clustering, and refresh total $695.60475 in the accepted preparation model.

② NORMALIZED QUERY COST

Each accepted query’s recorded duration is multiplied by the per-second read-side compute rate. This models accepted work; it excludes idle capacity and minimum warehouse billing.

CLICKHOUSE CLOUD

8 CUs × $0.3903/CU-hour × (10.312s aggregate + 46.074s drill-down) ÷ 3,600 = $0.04890.

DATABRICKS SERVERLESS SQL

X-Small is priced at 6 DBUs/hour × $0.91/DBU = $5.46/hour. Aggregate work: 613.164s × $5.46/hour ÷ 3,600 = $0.92997.

DATABRICKS DRILL-DOWN WORK

1,133.002s × $5.46/hour ÷ 3,600 = $1.71839. Total normalized query cost: $2.64835.

ALLOCATION BOUNDARY

Databricks ingestion and maintenance are allocated to the producer-active window, 2026-09-18 17:04:53 UTC through 2026-09-20 00:33:42.627 UTC, with an exclusive end. Post-ingestion maintenance, validation/control SQL, database storage, producer infrastructure, network charges, and idle/minimum warehouse charges are excluded. Maintenance was observed after ingestion but that later work is not charged to this score.

③ ACCUMULATED QUERY RUNTIME

ClickHouse: 10.312s aggregate + 46.074s drill-down = 56.386s. Databricks: 613.164s aggregate + 1,133.002s drill-down = 1,746.166s.

COMPARISON BOUNDARY

① covers the complete active ingest. ② and ③ use 189 four-query aggregate batches and 32 two-query drill-down batches. Their row-count comparison horizons differ from the 100-billion-row latency plots. Durability alignment does not establish identical query-visible rows or MV freshness. This is the complete modeled preparation path plus normalized query work, not the full provider bill.

FINAL SCORE

ClickHouse: ($28.68705 + $0.04890) × 56.386 = 1,620.30528. Databricks: ($695.60475 + $2.64835) × 1,746.166 = 1,219,265.82646. The ratio is 752.491×, rounded to 752×, using the full-precision inputs.

The ClickHouse writer-utilization results show ingestion continuing alongside background maintenance. CPU usage remained around 3.1 of the 4 available cores, tracked memory stayed within capacity, background merges processed several million rows per second, and the maximum active-part count per partition stayed around 100. These diagnostics describe the ClickHouse source run reused for this comparison.

Both nodes were retained for high availability, so the preparation cost includes the complete two-node deployment.

THE SAME INGESTION PROBLEM

Applications produce frequent writes that must become efficient storage writes. Both destinations used direct application-to-table ingestion with managed buffering. No Kafka broker, file landing stage, or Auto Loader job was added to the measured path.

DATABRICKS’ MANAGED PATH

The accepted runner used the Zerobus Arrow Flight DoPut interface, with 16 concurrent streams and 50,000-row client batches. Streams rotated every ten minutes; SDK recovery and bounded retries were enabled. Zerobus ingestion compute was allocated and priced separately from clustering, MV refresh, and SQL serving.

CLICKHOUSE’S NATIVE PATH

Ordinary INSERT requests with async_insert enabled were buffered on each receiving node. An adaptive flush timeout responded to incoming traffic. Ordering and pre-aggregation ran on the ingest nodes; the dedicated ingest service shared storage with the independently sized read service.

SHARED PACING

Both adapters read the same Parquet dataset and decoded the same schema under the same target-rate model. Parallelism and client batch sizes were adapted to each destination’s API; they were not forced to be identical.

CLICKHOUSE BATCHES

The client sent 3,000-row batches through asynchronous inserts. Default buffering flushed at the first of three thresholds: 100 MiB buffered, an adaptive timeout between 50 milliseconds and 1 second, or 450 queued insert queries.

DATABRICKS BATCHES

Each Arrow Flight stream sent 50,000-row batches with IPC compression disabled. The target rate was based on logical source rows, with acknowledged durable progress recorded separately from table visibility. Those batches were not staged as files for a later bulk load.

BATCH-SIZE BOUNDARY

These are client delivery batch sizes. Both systems buffer delivery into storage writes, so 3,000 and 50,000 rows do not specify final part or Parquet-file sizes.

RETRIES AND RECONCILIATION

Zerobus used durable stream offsets and SDK recovery. The settled validation confirmed exact row reconciliation and no duplicate surplus. ClickHouse supports insert deduplication for retries, including dependent materialized views. The final row check validates delivery, not equal summary freshness during ingestion.

INCREMENTAL MAINTENANCE

The raw Delta table enabled row tracking, change data feed, and deletion vectors. The MV passed incremental eligibility checks and used INCREMENTAL STRICT. The refresh evidence recorded 1,028 incremental group-aggregate refreshes and zero full refreshes across the collected run, including its post-ingestion observation period.

WHAT WAS TESTED

The serving layer was Serverless SQL X-Small. Lakehouse//RT was not available in the workspace and was not tested. This article describes the accepted configuration and its results, rather than projecting the performance of an unavailable serving product.

DATABRICKS SERIES

The freshness history was polled once a minute. The plotted metric is the time between distinct observed completed refresh IDs: floor(current completed_at − previous observed completed_at), in seconds. The selected active-ingestion evidence contains 1,021 such intervals; compact polling can miss intermediate completions.

SMOOTHING AND READOUTS

A centered 61-observation rolling mean averages about 1.8 minutes across the interval observations. An additional 11-observation display mean and shape-preserving interpolation produce a plotted peak of about 2.0 minutes. Readouts follow that displayed curve. This is a refresh-interval proxy, not a directly polled raw-to-MV watermark lag or a per-query stale-row count.

CLICKHOUSE BASELINE

The zero line is a benchmark-semantic baseline, not a separately polled provider metric. The raw table and incremental MV consume the same flushed insert block, with no independent summary-refresh interval between them. It does not claim zero source-to-query ingestion delay.

Lakehouse//RT is currently in Beta. We have requested access but have not yet received it.

Once Lakehouse//RT is generally available and we have access, we plan to repeat the full CostBench workload with Lakehouse//RT serving the queries and publish the results.

SNAPSHOT CONTRACT

A refresh updates the MV from its source table. The measured aggregate queries read that materialized result directly. They did not force a synchronous refresh before each query or merge a raw-table delta into the answer. The measured latency therefore includes Databricks’ native allowance for stale summaries.

TRIGGER CONTRACT

TRIGGER ON UPDATE schedules refreshes when upstream data changes. AT MOST EVERY INTERVAL 1 MINUTE imposes a minimum interval between triggers, not a one-minute maximum on data age or refresh completion.

INCREMENTAL CONTRACT

INCREMENTAL STRICT controls the maintenance method: eligible changes are processed incrementally rather than silently falling back to a full rebuild. It does not tie MV visibility to each ingested block or make the MV continuously current.

VISIBILITY CONTRACT

The runner’s raw_rows records acknowledged durable ingestion progress. Zerobus can durably accept rows before they are published into the Delta table. A query pair aligned by durable ingestion progress is therefore not proof that both queries saw exactly the same raw-row count or an equally fresh summary snapshot.

COMPARISON BOUNDARY

The headline preserves the tested MV-serving path, including its freshness behavior. A design that forced a refresh or queried and re-aggregated the raw table would have a different latency and cost profile; that work is not included in this result.

QUERY DEFINITIONS

Part 1 describes A1–A4 and D1–D2. Both systems used the same analytical workload and schedule, with SQL adapted to each engine. Counts, sums, and extrema were compared exactly; the approximate-percentile portion of D2 used the contract’s explicit tolerance. MV freshness was not assumed to be identical.

CACHE POLICY

Query-result caching was disabled. Databricks’ compact query evidence reported no result-cache hits for accepted queries. Underlying data caches were allowed to warm normally; the benchmark did not flush them between queries.

TIMING

Databricks latency uses Query History total_duration_ms ÷ 1,000, excluding result-fetch time. ClickHouse uses its recorded query duration. These timing conventions are retained in normalized query cost. Median, P99, and maximum statistics pool unsmoothed per-query observations through 100 billion rows; P99 uses linear interpolation.

LATENCY CHARTS

Each system is plotted at its own observed ingestion progress. Aggregate trends use a centered seven-observation rolling median; drill-downs use five observations, with shrinking edge windows. No outliers are removed. The curves are display smoothing; the statistics and accumulated totals use the original recorded durations.

INTERACTIVE READOUTS

Values below the plots follow the displayed trends at the selected row count. Logarithmic scales keep milliseconds and seconds readable together; Linear shows absolute differences. Each chart has its own playback and row-position controls.

ACCEPTED ACCUMULATED WORKLOAD

The accepted selection contains 189 four-query aggregate batches and 32 two-query drill-down batches per system: 756 + 64 = 820 executions. ClickHouse observations were aligned with Databricks durability progress, through 112.849 billion rows for aggregation and 111.649 billion for drill-downs. The first full-dataset observation and later post-ingestion samples are excluded.

ACCUMULATED CHARTS

Query durations are added at corresponding ingestion counts as step sums, without smoothing or interpolating future executions into earlier positions. Preparation-cost lines allocate the complete-ingest total proportionally by row progress; they are not metered cost-at-time traces. The score sums query durations, not elapsed ingestion time.