How Klaviyo cut segment processing from hours to 1 second with ClickHouse

Klaviyo, the $7 billion marketing automation platform powering over 170,000 brands, hit a wall with its segmentation engine a few years back. Computing which customer segments people belonged to was painfully slow -sometimes taking over an hour even with pre-aggregated data. Their original system was a patchwork of Python orchestration, MySQL, and Cassandra that just couldn't keep up.

So in 2022, they decided to rebuild everything from scratch around ClickHouse, and the results speak for themselves: what used to take an hour now happens in about a second.

• Complete migration from a multi-system architecture (Python, MySQL, Cassandra) to a ClickHouse-centered design
• 192-node ClickHouse cluster handling billions of daily updates and tens of billions of segment membership changes
• Smart sharding strategy that isolates customer data while enabling parallel query execution
• Intelligent optimization that only recomputes segments when time-based filters actually change, rather than processing everything on a fixed schedule
• Near real-time segment evaluation across millions of active segments, with most queries completing in under a second

Blog post: https://clickhouse.com/blog/how-klaviyo-uses-a-192-node-clickhouse-cluster-for-real-time-segmentation-and-personalization