48.9k
The three pillars of observability are logs, metrics, and traces: the three telemetry types a system emits so engineers can reconstruct its internal state. The framing dates to Peter Bourgon's February 2017 essay "Metrics, tracing, and logging" and now organizes most observability architectures. The pillars describe data types, though, not outcomes, and that gap is where the model breaks.
TL;DR
- Logs are discrete, timestamped event records; metrics are pre-aggregated numeric time series; traces are trees of spans following one request across services.
- Peter Bourgon named the model in February 2017; OpenTelemetry, the CNCF standard, calls the same categories "signals" and added profiles as a fourth in March 2024.
- Each pillar trades context for cost differently: metrics are cheapest and least specific, traces are most structured, logs sit between.
- Storing each pillar in a separate backend forces engineers to correlate by hand during incidents; that manual join work is the model's felt cost.
- Wide events, one high-cardinality record per request, are the convergence point: ClickHouse's own logging platform crossed 100 PB on that design.
What are the three pillars of observability?
The three pillars of observability are logs, metrics, and traces. Logs record discrete events as timestamped text or structured records. Metrics record numeric measurements aggregated over time. Traces record the path of a single request through a distributed system as a tree of timed spans. Together they are the raw material for diagnosing production behavior.
The term comes from Peter Bourgon's February 2017 post, which presented the three as a Venn diagram of trade-offs rather than a checklist, a nuance the industry largely dropped as vendors turned "three pillars" into product architecture. OpenTelemetry, the CNCF project that standardized instrumentation after the OpenTracing and OpenCensus merger in 2019, deliberately avoids "pillars" and calls them signals, a list it treats as open-ended.
What are logs, metrics, and traces?
Logs are immutable records of discrete events ("payment failed for order 8123"), emitted line by line as text or JSON. Metrics are numeric time series ("error rate = 0.4%"), aggregated at write time and stored compactly. Traces are directed trees of spans, each span one timed operation, linked by a shared trace ID across service boundaries.
The same failed checkout request looks different in each pillar:
// Log: one discrete event, arbitrary context
{"ts": "2026-07-28T09:14:07Z", "level": "error", "service": "checkout",
"msg": "payment declined", "order_id": 8123, "user_id": "u_442",
"provider": "stripe", "duration_ms": 1840}
// Metric: an aggregate counter, bounded labels
checkout_payment_failures_total{service="checkout", provider="stripe"} 417
// Trace: one span in a tree sharing trace_id
{"trace_id": "4bf92f35", "span_id": "00f067aa", "parent_span_id": "a3ce929d",
"name": "POST /charge", "start": "…07.120Z", "end": "…08.960Z",
"status": "ERROR"}| Pillar | What it captures | Data shape | Strengths | Where it breaks at scale |
|---|---|---|---|---|
| Logs | Discrete events, one record each | Timestamped text or JSON, arbitrary fields | Richest context; no schema needed up front; the debugging default | Volume grows with traffic; unstructured text is expensive to search at volume; context scattered across many lines |
| Metrics | Numeric measurements over time | Time series: name + labels + values | Cheap to store and alert on; instant dashboards; long retention | Aggregation discards context; high-cardinality labels (user ID, request ID) explode series counts and cost |
| Traces | One request's path across services | Tree of spans sharing a trace ID | Shows causality and where latency lives across service boundaries | Tracing everything is costly, so sampling drops the rare requests you often need most |
Logs vs metrics vs traces: when should you use each?
Use metrics to detect that something is wrong: they're cheap to evaluate continuously, so alerts and dashboards run on them. Use traces to locate where in a distributed request the problem lives. Use logs to explain why, since they carry the richest per-event context. Detection, location, explanation, in that order during most incidents.
The choice is a cost-versus-context trade. A Prometheus counter compresses millions of requests into one number per scrape interval, which is why Google's SRE book builds its four golden signals on metrics. But that compression is lossy: an error-rate metric can't say which customers failed. Traces, descended from Google's 2010 Dapper paper, keep per-request structure but rarely for every request: Dapper itself sampled as aggressively as 1 in 1,024 at high-throughput services. Logs keep everything, and their bill says so. Teams that pre-aggregate to control that bill are trading away exactly the context incidents need, the trade examined in depth in monitoring vs observability.
How do logs, metrics, and traces work together?
The pillars connect through shared identifiers. A metric alert fires; the engineer opens traces filtered to the alert's time window and service; a slow or failed span carries a trace ID; logs stamped with that trace ID explain what the code was doing. OpenTelemetry standardizes those join keys: trace and span IDs in log records, and exemplars linking metric data points to traces.
That's the workflow on paper. In practice, its quality depends entirely on whether the join keys exist end to end: one service that fails to propagate W3C Trace Context headers, or a logger that never stamps trace IDs, breaks the chain. When each pillar also lives in a separate backend (a metrics store, a log search engine, a trace store), the correlation is manual: three browser tabs, three query languages, and an engineer copy-pasting IDs between them at 3 a.m. That correlation pain is the strongest practical argument against treating the pillars as three separate systems.
Is there a fourth pillar of observability?
Several candidates are commonly promoted to fourth pillar: events (giving the MELT model), continuous profiles, and real user monitoring or session data. Profiles have the strongest formal claim: OpenTelemetry announced profiling as a signal in March 2024, with a data model and an eBPF-based agent donated by Elastic. There is no single agreed fourth pillar.
The proliferation is the tell. MELT (metrics, events, logs, traces) adds structured business-significant events; profiles add per-function CPU and memory detail; session replay adds the browser's view. Each addition is genuinely useful, and each also adds another silo to correlate if it ships as a separate store with a separate query surface. Counting pillars is less informative than asking whether the telemetry, whatever its shape, ends up somewhere it can be queried together.
Where does the three-pillars model break?
A team can ship all three pillars (dashboards, log search, and traces) and still fail to answer basic incident questions, because each pillar was pre-aggregated, sampled, or truncated into its own silo. Observability is the ability to interrogate a system's state, and no count of telemetry types guarantees it.
The model breaks in three specific places (this section is ClickHouse's editorial position; the definitions above stand on their own):
- Pre-aggregation loses the questions you didn't plan for. Metrics force you to choose labels up front; any dimension you dropped (customer ID, build hash, feature flag) is unrecoverable at query time.
- Sampling drops the interesting requests. Trace pipelines that keep 1% of traffic routinely lose the one anomalous request an incident hinges on.
- Silos multiply cost and correlation work: three stores mean three ingest pipelines, three retention policies, and manual joins between them.
The convergence answer is wide events: one high-cardinality, high-dimension record per unit of work, carrying what would otherwise be scattered across log lines, metric increments, and span attributes. Stripe described the pattern as canonical log lines, one comprehensive record per request. Structurally, a wide event is a structured log with enough columns that metrics become aggregations over it and traces become groupings of it. ClickHouse runs its own internal platform this way: scaling past 100 petabytes and 500 trillion rows by embracing wide events and replacing its OpenTelemetry collection pipeline with a native one, handling a 20x increase in event volume with less than 10% of the collection CPU footprint.
How ClickStack treats the three pillars
ClickStack, the ClickHouse observability stack, ingests all three pillars but stores them in one database instead of three. OpenTelemetry handles instrumentation and collection, ClickHouse stores logs, metrics, and traces as wide columnar rows, and HyperDX provides the UI over that single store, so cross-pillar correlation is a query rather than a tab-switch.
Because every signal lands in the same SQL-queryable store, the join keys the pillars depend on (trace IDs, resource attributes, timestamps) are join keys in the literal sense. A search shaped like the incident workflow above stays in one place:
-- logs and the failing trace, side by side
SELECT Timestamp, ServiceName, Body
FROM otel_logs
WHERE TraceId = '4bf92f35'
ORDER BY TimestampHigh-cardinality fields that would explode a metrics store are ordinary columns in ClickHouse, which is what makes the wide-events pattern practical at production volumes. To see the single-store approach in practice, ClickStack is open source and runs locally; the getting started guide takes one Docker command.
FAQ
Who coined the three pillars of observability?
Peter Bourgon, in a February 2017 essay titled "Metrics, tracing, and logging," which presented the three as overlapping trade-offs on a Venn diagram. The "pillars" label came later, popularized by observability vendors as they built product lines around the three categories.
Are the three pillars of observability outdated?
No, but the model shows its age. Logs, metrics, and traces remain real and useful data types; the weakness is the assumption that they belong in three separate systems, each discarding context through pre-aggregation, sampling, or siloing.
Do you need all three pillars to have observability?
No. What matters is which questions your telemetry can answer during an incident. A single stream of wide, context-rich events can make a system more observable than three thin, siloed pillars.
What is the difference between the three pillars and MELT?
MELT stands for metrics, events, logs, and traces. It extends the three pillars with events: structured records of significant state changes, such as a deploy or a payment. The distinction from logs is intent and structure, not storage.
