> ## Documentation Index
> Fetch the complete documentation index at: https://clickhouse.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Trace sampling

> Configure sample-weighted aggregations for sampled trace data in Self-managed ClickHouse Observability.

> [!NOTE]
>
> This page covers **self-managed ClickHouse Observability**. For the recommended fully managed experience, see the documentation [here](https://clickhouse.com/docs/clickstack/observability/overview.md).

High-throughput services can produce millions of spans per second. Storing every span is expensive, so teams commonly run the OpenTelemetry Collector's [tail-sampling processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/tailsamplingprocessor) to keep only 1-in-N spans. Each kept span carries a `SampleRate` attribute recording N.

Once data is sampled, naive aggregations are wrong: `count()` returns N-times fewer events than actually occurred, `sum()` and `avg()` are biased, and percentiles shift. Dashboards show misleadingly low request counts, throughput, and error rates.

Self-managed ClickHouse Observability solves this with a sampling-aware query engine. When you configure a sample rate expression on a trace source, the query builder rewrites SQL aggregations to weight each span by its sample rate — across dashboards, alerts, and ad-hoc searches.

## How it works

When a trace source has a `sampleRateExpression` configured, the query builder wraps it as:

```sql theme={null}
greatest(toUInt64OrZero(toString(expr)), 1)
```

Spans without a `SampleRate` attribute default to weight 1, so unsampled data produces identical results to the original queries.

The query builder then rewrites aggregations:

| Aggregation | Before | After (sample-corrected) |
| - | - | - |
| count | `count()` | `sum(weight)` |
| count + condition | `countIf(cond)` | `sumIf(weight, cond)` |
| avg | `avg(col)` | `sum(col * weight) / sum(weight)` |
| sum | `sum(col)` | `sum(col * weight)` |
| quantile(p) | `quantile(p)(col)` | `quantileTDigestWeighted(p)(col, weight)` |
| min / max | unchanged | unchanged |
| count\_distinct | unchanged | unchanged |

> [!NOTE]
>
> Percentiles under sampling use `quantileTDigestWeighted`, an approximate T-Digest sketch. Results are close but not exact.

## Configuring the sample rate expression

Open your trace source in **Source Settings** and enter the ClickHouse expression that evaluates to the per-span sample rate in the **Sample Rate Expression** field.

For example, if your OpenTelemetry tail-sampling processor writes the rate into `SpanAttributes['SampleRate']`:

![Sample Rate Expression field in Source Settings](https://mintcdn.com/private-7c7dfe99/i3Gw7HRb0YxPr7dq/images/clickstack/trace-sampling-source-settings.webp?fit=max\&auto=format\&n=i3Gw7HRb0YxPr7dq\&q=85\&s=68d9a71797d5c4d8cdff21da8a89e1ea)

Once configured, all charts, dashboards, alerts, and service dashboard panels automatically apply sample-weighted aggregations. No changes to individual queries are needed.