Shopify’s Open Source Approach to Network Monitoring with eBPF, Vector and ClickHouse

Sebastian Rabenhorst & Matt Franklin, Shopify
At Shopify, they successfully implemented a scalable, open-source network monitoring solution for the cloud. In this talk, they will demonstrate how they built a network monitoring solution leveraging eBPF, Vector, ClickHouse, and Grafana. This solution enables them to monitor over 30 million network flow, DNS and other networking-related events per second at the container level for thousands of services across hundreds of Kubernetes clusters in the Shopify Cloud.
Related videos
View all Videos
User stories, Meetups
How Equativ powers real-time advertising products with ClickHouse Cloud
Adrien Coulaud and Jean-Denis Costa explain how Equativ uses ClickHouse Cloud for customer-facing reporting, fast lookups, and operational feedback across its advertising products.

Open House, User stories
Precomputing fraud signals with ClickHouse materialized views at Moniepoint
Johnson Adekunle and Abraham Imohiosen, Engineering Managers at Moniepoint, explain how precomputed customer state with ClickHouse materialized views turns fraud checks into lookups, reducing p90 aggregate lookup latency by 93.1%.

Open House, User stories
How Twilio handles a petabyte of logs a day with ClickHouse Cloud
Avleen Vig, Site Reliability Architect at Twilio, explains how ClickHouse Cloud supports 30 million log lines per second at peak, a petabyte of raw logs per day, and 50% annual data growth.

Open House, User stories
Observability for the age of agents: How Vercel uses ClickHouse
Dom Busser and Vincent Voyer explain how Vercel uses ClickHouse Cloud to ingest 100 billion log events a day, serve customer-facing observability, and support agents querying data with SQL.