Our original benchmark measured one important part of the full real-time analytics path: the cost of continuously turning newly written raw data into query-ready raw data.
Below is the original setup, along with Snowflake's main comments that it helps clarify.
① Dataset and ordering key
We used the ClickBench web analytics dataset, with more than 100 columns and the original multi-column sorting key used by the ClickBench workload.
That choice was deliberate. Many ClickBench queries benefit from this ordering in ClickHouse, so to keep later query performance measurements fair, we used the equivalent clustering key in Snowflake.
Snowflake’s comment: use a simpler clustering key, and avoid the artificial arrival pattern created by repeating the original ClickBench dataset to reach 100 billion rows.
That is fair, and we address both in the expanded benchmark below. But they actually answer different questions: how data arrives, and what physical layout the system has to maintain for the workload.
We ingested data continuously at roughly 1 million rows per second, or about 1 GB of uncompressed data per second.
This was not a maximum-throughput loading test. The ingest rate was intentionally fixed to simulate a real-time workload where new data arrives continuously, and the goal was to use the smallest, lowest-cost Snowflake setup that could sustain that rate while keeping the table query-ready.
Snowflake’s comment: use COPY INTO, Snowpipe Streaming, larger warehouses, and Gen2 warehouses.
Those are reasonable choices for other ingest tests. But in this benchmark, the loading mechanism was secondary. The question was not how quickly Snowflake could finish loading 100 billion rows, but what it would cost to keep up with a fixed real-time ingest rate as common in real-time analytics use cases.
The first benchmark focused on the base table: newly written raw data in, query-ready raw data out. It did not measure materialized views.
Snowflake’s comment: broader production setups should include more of the data pipeline.
Agreed. That is exactly what CostBench adds next: derived-data maintenance, freshness, and reads alongside ingest and raw-data organization.
After the table reached 100 billion rows, we ran the ClickBench query workload once to validate query performance.
So the first benchmark did not measure continuous serving while ingest, clustering, refresh, or other maintenance work continued in the background.
Snowflake’s comment: use Interactive Tables and Interactive Warehouses for high-concurrency, low-latency serving.
Fair enough. Interactive Tables were not part of the first benchmark because that benchmark used the Snowflake path most customers could actually use: standard tables with clustering. Interactive Tables are a newer feature and are currently available only in selected regions.
For the expanded benchmark, we therefore test both paths: the broadly available Snowflake setup using standard tables and materialized views, and an Interactive Tables setup in a supported region.
The expanded benchmark addresses the broader question
Before Snowflake published its response, we had already started applying the new CostBench methodology to a broader full-path benchmark.
This time, we intentionally started at the easier end of the spectrum: a much simpler dataset and ordering key, a cleaner arrival pattern, and continuous reads while data keeps moving, a setup that removes several of the factors Snowflake objected to in the original ClickBench test.
We will repeat this with heavier datasets later. But first, we wanted to measure the full path in the cleanest possible case.
We use a real stock market quotes dataset licensed from a data provider, with access to hundreds of billions of rows.
Compared with ClickBench, it has a much simpler shape: 12 columns, a simple two-column sorting key, and a natural timestamp order.
We ingest the data in strict timestamp order at steady rate of 1 million rows per second. At that rate, the system receives 100 billion fresh rows every 28 hours.
This time, the sorting/clustering key are just two columns: (sym, t) - the stock symbol and the timestamp. This is as natural and simple as it can get.
ClickHouse sorts the raw table by those columns, and Snowflake clusters the raw table by those columns.
Unlike the original benchmark, this run includes continuously maintained pre-aggregations.
That means we also measure the cost of keeping derived data fresh while new rows keep arriving.
This time, queries run continuously while fresh data continues to arrive.
The workload includes two query types: dashboard queries against the pre-aggregated data, and drill-down queries against the raw data with filters matching the raw table’s ordering/clustering key.
We keep the systems’ caches enabled. In benchmarks with static data, we usually disable query-result caches to avoid measuring memory lookups instead of pure engine performance. Here, the data is constantly changing, so caches are much less useful, and leaving them enabled better reflects real-world usage.
We run each query once per round. That mirrors how these queries are used in practice: a dashboard refresh, a user drill-down, or an exploratory ad-hoc query happens once against the latest state of the data.
That simulates the real-time scenario more directly: ingest, maintenance, freshness, and reads all happen at the same time.
For ClickHouse Cloud, we used separate services for ingest and reads, so the write path and query path were isolated from each other.
The client (benchmark driver) is kept intentionally simple and it performs the the same work for ClickHouse and Snowflake.
It reads Parquet row groups directly from existing Parquet files in binary form, combines them into batches of roughly 1 million rows, and sends those batches to the target system. There is no decoding, no decompression, no encoding, and no compression on the client side.
That matters because Snowflake’s response called out client-side cost as a missing factor in the first benchmark. In this setup, the client work is identical for both systems and uses very little CPU and memory, so client-side cost is no longer a meaningful differentiator.
Ingest runs on a dedicated ClickHouse Cloud service with 2 nodes, each using 2 CPUs and 8 GiB of memory.
That was enough to ingest 1 million rows-per-second data stream continuously, sort the incoming data, and update the pre-aggregated data, while keeping the total number of active data parts across the tables at around 60 at any given time via continuous background merges.
This efficiency also matters in practice: it lets us run demos like StockHouse cost-effectively, with the same market data used in this benchmark streaming in live and becoming query-ready as it arrives.
③ Sorted raw data and materialized views
ClickHouse writes the raw data directly to disk in sorted form, using a MergeTree table. This is the table that serves the raw-data drill-down workload.
At the same time, an incremental materialized view maintains the pre-aggregated data in an AggregatingMergeTree table as new rows arrive. This is the table that serves the dashboard workload.
Code: raw MergeTree table, AggregatingMergeTree table, and incremental materialized view.
At any time during the benchmark, the raw MergeTree table remains sorted for fast drill-downs, and the AggregatingMergeTree table stores the pre-aggregated data for the dashboard queries, with no freshness gap to the base table. Both tables stay up-to-date and ready for queries all the time.
Read queries run on a separate ClickHouse Cloud service with 1 node and 16 CPUs.
We use the same amount of read compute for the Snowflake setups, so read-side capacity is aligned across systems.
Note: It is widely understood that a Snowflake Gen2 Small warehouse on AWS uses 16 AWS Graviton3 cores. ClickHouse Cloud also uses AWS Graviton3 cores in AWS deployments, so the comparison aligns both read paths on 16 cores of the same CPU generation.
To simulate a continuous read workload, we run two types of queries periodically throughout the benchmark.
Every 10 minutes, we send 4 dashboard queries against the pre-aggregated table.
Every hour, we send 2 ad-hoc drill-down queries against the raw table. These drill-down queries use filters that match the raw table’s sort order.
We run the same query schedule on Snowflake, with equivalent physical layouts: the raw data is clustered by the same key, and the aggregated data is pre-aggregated in the same way.
Snowflake setup 1: standard tables and materialized views
The first Snowflake setup uses the path most Snowflake customers can use today: standard tables, standard materialized views, and Gen2 warehouses.
The client setup (benchmark driver) is the same as in the ClickHouse setup.
It reads Parquet row groups directly in binary form, combines them into batches of roughly 1 million rows, and sends those batches to Snowflake. This time, the client uses COPY INTO to load the Parquet data into the raw table.
As mentioned earlier, the client does no decoding, no decompression, no encoding, and no compression. The amount of performed client-side work is therefore exactly the same as for ClickHouse.
For ingest, we again chose the smallest Snowflake warehouse that could sustain the fixed 1 million rows per second ingest rate.
But this time, following Snowflake’s recommendation, we use Gen2 hardware: specifically, a Gen2 X-Small warehouse with 8 CPUs.
The goal is still not maximum load throughput.
The goal is the lowest-cost setup that can sustain with the fixed ingest rate to simulate a real-time workload where new data arrives continuously.
The raw data is written into a standard Snowflake table. This is the table that serves the hourly raw-data drill-down workload.
The data ordering is then handled by Snowflake’s serverless clustering service in the background, as in the original benchmark. The goal is to keep the raw table physically optimized for the drill-down queries.
Code: raw Snowflake table.
To include pre-aggregation in the end-to-end benchmark, we use a Snowflake materialized view over the raw table. This view is queried by the dashboard workload.
Materialized views are an Enterprise-only Snowflake feature. Refresh work is handled by Snowflake’s serverless materialized view refresh service in the background.
Code: Snowflake materialized view.
As in the ClickHouse setup, we use separate warehouses for ingest and reads, so the write path and query path are isolated from each other.
For reads, we use a Gen2 Small warehouse with 16 CPUs.
This matches the read-side CPU count used in ClickHouse.
We run the same continuous query schedule as in ClickHouse.
Every 10 minutes, we run 4 queries against the pre-aggregated data, simulating dashboard refreshes.
Every hour, we run 2 raw-data drill-down queries against the raw table.
The second Snowflake setup uses Interactive Tables for both the raw data and the pre-aggregated data.
Interactive Tables are currently available only in selected regions. To test them, we had to create a Snowflake account in a supported region. For that reason, we include both Snowflake paths: the broadly available setup using standard tables and materialized views, and the newer Interactive Tables setup where available.
The client is the same as in the ClickHouse setup and the first Snowflake setup.
It reads Parquet row groups directly in binary form, combines them into batches of roughly 1 million rows, and uses COPY INTO to load the data into Snowflake. The client does no decoding, no decompression, no encoding, and no compression.
As before, ingest uses the smallest Snowflake warehouse that could sustain the fixed 1 million rows per second ingest rate.
We use a Gen2 X-Small warehouse with 8 CPUs.
Incoming data first lands in a standard Snowflake table. The raw Interactive Table is then maintained from that source table by a user-managed warehouse, with refreshes triggered as needed to meet the configured target lag. This corresponds to the standard Interactive Tables pattern.
We use a 10-minute target lag for the raw Interactive Table, which serves the hourly drill-down workload.
(Direct ingest into Interactive Tables is possible through Snowflake-managed ingest paths, which we treat separately because it does not cover the pre-aggregated path measured here.)
Code and docs: standard Snowflake table, raw Interactive Table, target lag refresh behavior, and insert-only limitation.
The pre-aggregated data is also stored in an Interactive Table.
Here, the refresh warehouse regularly transfers the new rows since the previous refresh, aggregates them, and merges the result into the pre-aggregated Interactive Table.
We use a 1-minute target lag, which is the smallest target lag available.
This is the table that serves the dashboard workload, so we configure the freshest pre-aggregation path Snowflake allows.
That lets us compare it with ClickHouse incremental materialized views, which update on the ingest path, and measure what it costs to get as close as possible to that freshness model in Snowflake. Snowflake also offers Interactive Materialized Views; we will test that path separately.
We use a Gen2 X-Large warehouse for refreshes, after a Small, Medium, and Large warehouse could not reliably keep the pre-aggregation refresh within the 1-minute target lag.
Code and docs: pre-aggregated Interactive Table, aggregation query, target lag behavior, Interactive Materialized Views, and refresh-size comparison.
For reads, we use a Small Interactive Warehouse with 16 CPUs.
This matches the read-side CPU count used for ClickHouse and the first Snowflake setup.
Unlike a standard warehouse, the Interactive Warehouse also comes with a large cache. For a Small Interactive Warehouse, that cache is roughly 600 GB, allowing the working set of Interactive Tables to be kept in memory.
Note that Interactive Warehouses have a 1-hour minimum billing duration, and auto-suspend has a minimum setting of 24 hours. In this benchmark, that billing model fits the workload: we are measuring a continuously running real-time analytics service, with ongoing ingest, refresh, and queries.
We run the same continuous query schedule as before.
Every 10 minutes, we run 4 queries against the pre-aggregated data, simulating dashboard refreshes.
Every hour, we run 2 raw-data drill-down queries against the raw Interactive table.
Results: ClickHouse vs. Snowflake standard tables
First, we compare ClickHouse with the Snowflake setup most customers can use today: standard tables, standard materialized views, serverless clustering, serverless MV refresh, and Gen2 warehouses.
Performance: latency and freshness under continuous ingest
Both systems use the same read-side compute for these measurements: 16 AWS Graviton3 cores.
The query schedule is also identical.
This chart shows the 4 dashboard queries we run every 10 minutes against the pre-aggregated data. The x-axis is the number of rows ingested into the raw table, from the start of the run to 100 billion rows after roughly 28 hours. The y-axis is query latency.
All per-query run details are available in the CostBench repository: ClickHouse, Snowflake.
ClickHouse stays almost completely flat as the raw table grows, with dashboard queries remaining in the single-digit millisecond range. That is because ClickHouse updates the pre-aggregated table on the ingest path, so it stays fresh and query-ready continuously.
Snowflake’s standard materialized-view setup behaves very differently. Query latency is much higher and fluctuates heavily: simpler dashboard queries stay around the low seconds, while heavier dashboard queries jump into the high single-digit to tens-of-seconds range.
The reason is freshness. Snowflake materialized views still return results for the latest ingested data, even when the materialized view itself is behind. In that case, Snowflake has to combine the materialized view with the missing rows from the raw table at query time. For a dashboard workload, that means refreshes do not just get slower; they become unpredictable.
The next chart shows the underlying freshness lag directly.
We measured this lag by polling Snowflake’s materialized-view metadata once per minute with SHOW MATERIALIZED VIEWS; the detailed freshness samples are available here - see the behind_by column.
ClickHouse stays current because its materialized views are updated synchronously on the ingest path.
Snowflake’s materialized view is refreshed by a serverless background service with no freshness SLA and no user control over when it runs or how much compute it uses. In this run, the Snowflake materialized view was continuously behind, with lag repeatedly reaching roughly 60 to 72 minutes.
The next chart shows the 2 drill-down queries we run every hour against the raw data. The x-axis is the number of rows ingested into the raw table, from the start of the run to 100 billion rows after roughly 28 hours. The y-axis is query latency.
All per-query run details are available in the CostBench repository: ClickHouse, Snowflake.
These queries use filters that match the raw table’s ordering/clustering key. In other words, both systems get the same physical-layout advantage: ClickHouse sorts by the key, and Snowflake clusters by the same key.
Even then, the pattern is the same as in our initial read-side benchmark: as the dataset grows, the systems diverge. ClickHouse stays roughly flat, largely unimpressed by data size. Snowflake gets progressively slower, reaching several seconds by the end of the run.
Cost: ingest, maintenance, and reads
For cost, we use Enterprise pricing for AWS us-east. This gives Snowflake access to materialized views, which require Enterprise Edition; ClickHouse materialized views are available even in the open-source version.
The next chart shows the total fresh-data path cost for the first 28 hours of the run: ingesting 100 billion fresh rows at a continuous rate of 1 million rows per second, while keeping the raw data sorted/clustered and the pre-aggregated data maintained.
For ClickHouse, the ingest service uses 2 nodes with 2 CPUs and 8 GiB RAM per node. In ClickHouse Cloud pricing, that corresponds to 2 compute units. Over the 28-hour run: 2 compute units × 28 hours × $0.3903/hour = $21.86.
For Snowflake, the ingest warehouse is a Gen2 X-Small warehouse. At 1.35 credits/hour and $3 per credit, the ingest warehouse cost is: 1.35 credits/hour × 28 hours × $3/credit = $113.40.
The remaining Snowflake write-side costs come from serverless services. We take the consumed credits from Snowflake system tables and multiply by the same $3 per credit price: serverless clustering: 54.12 credits × $3 = $162.37; serverless materialized-view refresh: 3.60 credits × $3 = $10.8.
Together, Snowflake’s fresh-data path costs $286.58.
Before query cost is included, Snowflake’s standard setup already costs about 13× more to keep fresh data query-ready.
Finally, we measure the read side: how much runtime and query cost each system needs to serve the same continuous query schedule over the 28-hour run.
For query cost, we use the same simplification as in the initial read-side benchmark methodology: accumulated query runtime × read-side compute price, as if query compute were billed with perfect per-second granularity.
In reality, both Snowflake and ClickHouse Cloud continue billing until a configurable idle timeout is reached. But this normalization compares query-engine efficiency directly:
For a given amount of paid compute time, how much query work can the system complete?
The chart below shows the result.
All per-query run details are available in the CostBench repository: ClickHouse dashboard queries and drill-down queries, Snowflake dashboard queries and drill-down queries.
For query cost, we use the accumulated runtime of the continuous read workload and multiply it by the price of the read-side compute.
ClickHouse uses one read service with 16 CPUs and 64 GiB RAM, which corresponds to 8 compute units: 8 compute units × $0.3903/hour × 70.4 seconds = $0.061
Snowflake uses a Gen2 Small read warehouse with 16 CPUs. At 2.7 credits/hour and $3 per credit, that is $8.10/hour: 2.7 credits/hour × $3/credit × 5,017 seconds = $11.288
That makes ClickHouse about 185× cheaper on query compute for this workload, before we combine it with the fresh-data path cost.
Where do you get the most full-path real-time performance per dollar spent?
To compare full-path real-time cost-efficiency directly, we collapse cost and performance into a single lower-is-better score.
The score is:
(fresh-data path cost + query cost) × total query runtime
This captures the basic cost-performance tradeoff: systems score better when they keep the path cheap and serve queries quickly. Expensive systems score worse. Slow systems score worse. And when a system is both expensive and slow, the two effects compound.
Snowflake’s standard setup scores 969× worse.
That is the compounding effect: Snowflake pays more to keep the path running and still needs much more runtime to serve the same continuous query schedule.
And this is before spending the budget difference. At Snowflake’s spend level, ClickHouse could scale up to larger services and reduce latency even further while still staying below the same total cost.
In the previous section, we measured ClickHouse against Snowflake’s broadly available standard-table path. Next, we repeat the same CostBench workload against Snowflake’s Interactive Tables setup.
The workload, client behavior, ingest rate, query schedule, and read-side CPU count stay the same. What changes is the Snowflake serving path: raw and pre-aggregated data now sit in Interactive Tables, refreshed by a user-managed warehouse and served by an Interactive Warehouse with a large cache.
As a reminder:
Both systems use the same read-side compute for these measurements: 16 AWS Graviton3 cores.
The charts use the same format as above: x-axis is raw rows ingested, up to 100 billion rows after roughly 28 hours; y-axis is query latency.
For dashboard queries, Interactive Tables in combination with an interactive warehouse dramatically improve Snowflake’s latency compared with the standard materialized-view setup. But ClickHouse still stays lower and flatter, despite using the same read-side CPU count.
All per-query run details are available in the CostBench repository: ClickHouse, Snowflake.
The next chart shows what it takes to keep pre-aggregations fresh in the Interactive Tables setup.
We measured this lag by polling Snowflake’s INFORMATION_SCHEMA.INTERACTIVE_TABLE_REFRESH_HISTORY table once per minute`, results are here.
ClickHouse stays current throughout the run because its materialized views are updated synchronously as new rows are inserted.
The raw Interactive Table behaved as expected: with a 10-minute refresh target, it stayed within that target across the tested warehouse sizes. That kept the raw table fresh enough for the hourly drill-down workload.
The harder part was the pre-aggregated Interactive Table. As mentioned earlier, we used the smallest refresh interval Snowflake allows, 1 minute, because this table serves the stock-quotes dashboard workload and is the closest Snowflake can get to ClickHouse materialized views.
A Gen2 Small refresh warehouse fell behind after roughly 3 hours, Gen2 Medium after roughly 8 hours, and Gen2 Large after roughly 15 hours. The red crosses mark the point where each setup could no longer keep the pre-aggregated Interactive Table within the configured 1-minute target.
This is not because Snowflake re-aggregates the entire raw table on every refresh. The refresh into the pre-aggregated Interactive Table is incremental, as confirmed by a system-table query for our setup. Even so, the refresh warehouse still has to process the new rows, aggregate them, and merge the result into the target table before the next refresh is due.
That is why the final setup uses a Gen2 X-Large refresh warehouse. It kept both Interactive Tables within their configured targets over the first 24 hours.
The pre-aggregation lag is still creeping upward. It does not break the 1-minute target on day one, but the curve is not flat. For longer runs, Snowflake would need either more refresh compute or a looser freshness target.
For drill-down queries, Interactive Tables improve Snowflake substantially compared with the standard-table setup.
All per-query run details are available in the CostBench repository: ClickHouse, Snowflake.
But the improvement does not stay flat as data volume grows.
Even though the working set still fits in the Interactive Warehouse cache throughout the first 100 billion rows, Snowflake latency rises with data size. ClickHouse stays much flatter, even without an Interactive Warehouse cache. For one drill-down query, Snowflake is already slower than ClickHouse around 50 billion rows; by the end of the run, both drill-down queries are at or above ClickHouse latency.
Takeaway: Even on a lightweight dataset, after only about one day of a modest ingest at 1 million rows per second, Snowflake Interactive Tables are already at or above ClickHouse latency for the drill-down workload.
Scaling note
The important point is that latency already grows while the working set still fits in the Interactive Warehouse cache.
That raises the next scaling question: what happens when the working set no longer fits, even in the largest available Interactive Warehouse cache?
Snowflake’s Interactive Warehouse model also has a hard 5-second query timeout. Once queries cross that threshold, Snowflake recommends a fallback warehouse. That adds another cost dimension: the fallback warehouse has to be available when needed, and the user-visible runtime includes the failed attempt plus the retry on the fallback compute.
Cost: ingest and refresh
Interactive Tables make the query path cheaper than the standard Snowflake setup, but the cost moves to refresh.
For ClickHouse, the fresh-data path is unchanged: the same ingest service handles ingest, sorting, merges, and materialized-view updates. As before, that costs: 2 compute units × 28 hours × $0.3903/hour = $21.86.
For Snowflake, the ingest warehouse is also unchanged. The Gen2 X-Small ingest warehouse costs: 1.35 credits/hour × 28 hours × $3/credit = $113.40.
The difference is refresh. To keep the pre-aggregated Interactive Table at Snowflake’s minimum 1-minute target, the refresh warehouse effectively has to run continuously. The final setup uses a Gen2 X-Large refresh warehouse. At 21.6 credits/hour and $3 per credit, that costs: 21.6 credits/hour × 28 hours × $3/credit = $1,814.40.
Together, Snowflake’s Interactive Tables fresh-data path costs $1,927.80.
Before query cost is included, that is about 88× more than ClickHouse to keep the fresh-data path query-ready.
For query cost, we use the same per-second runtime normalization described in the standard setup section.
The chart below shows the total accumulated runtime and query cost for the Interactive Tables read workload.
All per-query run details are available in the CostBench repository: ClickHouse dashboard queries and drill-down queries, Snowflake dashboard queries and drill-down queries.
ClickHouse uses the same read service as before: 16 CPUs and 64 GiB RAM, or 8 compute units. 8 compute units × $0.3903/hour × 70.4 seconds = $0.061
Snowflake uses a Small Interactive Warehouse. At 1.2 credits/hour and $3 per credit, that is $3.60/hour. 1.2 credits/hour × $3/credit × 144 seconds = $0.144
ClickHouse is still about 2× faster and 2.4× cheaper on query compute, using the same read-side CPU count.
Now we can compare the full path directly: what it costs to keep fresh data query-ready, and how much runtime the system needs to serve the continuous workload.
Interactive Tables improve Snowflake’s score substantially compared with the standard setup.
But the full path is still much more expensive to run. Snowflake’s Interactive Tables setup scores 180× worse.
That gap matters:
ClickHouse users could spend more on larger compute, beat the Interactive Tables latency by a wider margin, and still stay below Snowflake’s total cost.
The results above also show a more basic difference in how each system keeps derived data fresh while new rows keep arriving.
① Fresh data arrives continuously.
That is the starting point of the whole benchmark: the base table is constantly moving, so pre-aggregations have to keep up with a live stream, not a static dataset.
② Snowflake materialized views refresh asynchronously.
That is exactly what showed up in the standard Snowflake setup: while ingest continued, the materialized view was repeatedly 60 to 72 minutes behind the base table. Snowflake still returns up-to-date query results by combining the materialized view with missing rows from the raw table at query time, but that is also why dashboard latency became slow and unpredictable.
③ Snowflake Interactive Tables use scheduled refresh.
Interactive Tables improve freshness control, but they still work through scheduled refreshes. For pre-aggregations, the smallest target lag is 1 minute. That means the refresh warehouse becomes part of the continuous write path: it has to run at that cadence during ingest, and it has to be sized so each refresh finishes before the next one is due. If it cannot, lag accumulates. As the table grows, maintaining that 1-minute target becomes a scaling and cost problem.
④ ClickHouse incremental materialized views update on ingest.
ClickHouse takes a different path: materialized views are updated synchronously as new data is inserted. The pre-aggregated table therefore stays aligned with the base table instead of waiting for a separate refresh cycle.
⑤ That changes which real-time use cases are possible.
For dashboards, a 1-minute refresh target may be acceptable. For sub-minute decisioning workloads, especially in financial services and fraud detection, where the useful freshness window is measured in hundreds of milliseconds, scheduled refresh is already too late. In those cases, freshness determines whether the system can support the workload at all.
Outlook: more paths and heavier workloads
CostBench is a framework for testing the full real-time analytics path under different system designs.
The next variant we want to test is Snowflake’s managed ingest path, where Snowpipe Streaming can write directly into a raw-data Interactive Table, but cannot directly maintain the pre-aggregated data.
The comparable ClickHouse path is ClickPipes.
That will let us compare managed ingest on both sides without the lightweight client used in this benchmark writing the data directly.
We will also use the same framework to push the read side harder and scale the query workload independently, and test higher concurrency while fresh data continues to arrive.
As mentioned at the beginning, this run deliberately started at the easier end of the spectrum: a small, narrow dataset, a simple sorting key, and an arrival pattern that is favorable to Snowflake.
Next, we will repeat the test at the other end: a much wider, heavier dataset with the kind of shape common in ClickHouse real-time analytics workloads.
That will show both sides of the spectrum: the clean case that gives Snowflake the best chance, and the heavier case where ClickHouse is designed to shine.
And of course, we are open to other setups Snowflake thinks should be tested. The framework is flexible; the important thing is that the comparison stays full-path.
Snowflake’s response helped sharpen the question. Many of the suggestions were valid optimizations: simplify clustering, move reads to Interactive Tables, or tune the refresh path.
But real-time systems are full-path systems. Improving one stage can move the cost somewhere else: faster reads can require more refresh compute, fresher pre-aggregations can require a continuously running refresh warehouse, and better physical organization can increase maintenance work.
So we measured the whole path.
Some results improved. Interactive Tables made Snowflake reads much faster than the standard materialized-view setup. But once we measured the full path - ingest, physical organization, pre-aggregation freshness, refresh cost, and continuous query serving - the tradeoff became clearer.
In the original query-ready-data benchmark, ClickHouse delivered 28× better write-side cost-performance than Snowflake. In this expanded benchmark, ClickHouse delivered 969× better cost-performance than Snowflake’s standard materialized-view setup, and 180× better than Snowflake’s Interactive Tables setup.
That is the main takeaway: real-time analytics is not just about making queries fast, but about keeping fresh data query-ready, keeping derived data fresh, and serving low-latency queries continuously without turning refresh into the dominant cost.
For sub-minute decisioning workloads, the freshness model matters even more. Among the tested paths, only ClickHouse incremental materialized views keep pre-aggregations continuously in sync with the base table without any lag.
ClickHouse is built for cost-efficient real-time analytics across the full path. MergeTree tables keep raw data ordered and query-ready. AggregatingMergeTree tables keep pre-aggregations fresh with no refresh gap. And the query engine keeps latency low as data grows.
Our latest migration story shows the same full-path efficiency in production. Appcues moved real-time customer-facing analytics from Snowflake to ClickHouse Cloud. Across 1.31 PB of data and 410 billion events. P95 query latency dropped from 20+ seconds to under 2 seconds, ingestion latency dropped from 10+ minutes to about 5 seconds, and analytics spend fell even as new workloads were added.
That is the path CostBench is designed to evaluate. We will keep testing more systems, more ingest paths, heavier datasets, and higher concurrency. Stay tuned.
