How BigQuery prepares incoming data
Pause
FOR CONTEXT
How real-time systems keep incoming data query-ready
Fresh-data path
Query-ready data
Newly
written
Ordered
raw data
Ordered pre-
aggregated
Query engine
Query
compute
Fresh
data
continuously
Fast
answers
continuously
1
Store in
columns
2
Order
data
3
Order and pre-
aggregate data
BigQuery
Fresh-data path
Query-ready data
1
Storage Write
API streams
Committed
streams
Clustered table
Incremental MV
Ordered
raw data
Ordered pre-
aggregated
NBBO
quotes
1M rows/s
2
Async
serverless
clustering
3
Async
serverless
MV refresh
Serverless
compute pool
Dynamic slots
Query
results
4 interactive aggs
every 10 min
2 drill-downs
every hour