> ## Documentation Index
> Fetch the complete documentation index at: https://clickhouse.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Prometheus HTTP API and PromQL

> Prometheus HTTP API support in ClickHouse: remote write, remote read, and PromQL queries over a TimeSeries table.

Private preview

ClickHouse implements the Prometheus HTTP API over a [`TimeSeries`](https://clickhouse.com/docs/reference/engines/table-engines/integrations/time-series) table. One handler serves remote write, remote read, instant PromQL queries, and range PromQL queries.

To expose ClickHouse's own metrics for a Prometheus server to scrape, see the [Prometheus metrics endpoint](https://clickhouse.com/docs/concepts/features/connect-to-clickhouse/prometheus/metrics.md).

## Prerequisites

The setup steps differ between ClickHouse Cloud and self-managed ClickHouse. Follow the section for your deployment.

### ClickHouse Cloud

> [!NOTE]
>
> PromQL support in ClickHouse Cloud is in private preview. The services that take part in the private preview already have the `enable_time_series_table` setting and the Prometheus API endpoints configured. Other ClickHouse Cloud services do not have this configuration, and you cannot enable the feature yourself on such a service. The `SET enable_time_series_table` statement and the `http_handlers` configuration in the next sections apply to self-managed deployments.

On a service that takes part in the private preview, continue at [Create a TimeSeries table](#create-a-timeseries-table). The service serves the endpoint paths listed in the [endpoint table](#configure-prometheus-api).

### Self-managed: enable the TimeSeries setting

Enable the `enable_time_series_table` setting for the user that creates and accesses the table:

```sql theme={null}
SET enable_time_series_table = 1;
```

For HTTP API requests, enable `enable_time_series_table` in the profile of the API user.

### Self-managed: configure the Prometheus API endpoints

Configure one prefix-routed handler on the main ClickHouse HTTP port:

```xml theme={null}
<http_handlers>
    <defaults/>
    <rule>
        <url_prefix>/prometheus/api/v1</url_prefix>
        <handler>
            <type>prometheus_api_v1</type>
        </handler>
    </rule>
</http_handlers>
```

`<defaults/>` preserves the built-in handlers for endpoints such as `/ping` and for SQL requests. The prefix above exposes these endpoints through one handler:

| Endpoint | Purpose |
| - | - |
| `/prometheus/api/v1/write` | Prometheus remote write |
| `/prometheus/api/v1/read` | Prometheus remote read |
| `/prometheus/api/v1/query` | Instant PromQL queries |
| `/prometheus/api/v1/query_range` | Range PromQL queries |
| `/prometheus/api/v1/format_query` | PromQL expression formatting |
| `/prometheus/api/v1/series` | Series metadata |
| `/prometheus/api/v1/labels` | Label names |
| `/prometheus/api/v1/label/<name>/values` | Label values |
| `/prometheus/api/v1/metadata` | Metric-family metadata |
| `/prometheus/api/v1/federate` | Federation of the latest samples |

The example omits `database` and `table` from the handler. Each request must provide the `table` query parameter (except for `/format_query`, which only parses the given PromQL expression and doesn't need a table). It can also provide `database`, use a qualified table name such as `prometheus.metrics`, or omit the database to use `default`. This allows one handler to serve multiple `TimeSeries` tables.

To use one fixed table for every request, configure it in the handler:

```xml theme={null}
<handler>
    <type>prometheus_api_v1</type>
    <database>prometheus</database>
    <table>metrics</table>
</handler>
```

A table configured in the handler cannot be overridden by request parameters.

Routing and handler settings:

| Name | Default | Description |
| - | - | - |
| `url_prefix` | none | Rule filter that matches every request path that starts with the configured prefix. |
| `table` | none | The name of a `TimeSeries` table. When omitted, the request must provide the `table` query parameter. The configured name can include a database. |
| `database` | none | The database containing the table. A request can provide it as a query parameter. When omitted, ClickHouse uses a database from a qualified `table` value or falls back to `default`. |

### Create a TimeSeries table

Create a database and a `TimeSeries` table:

```sql theme={null}
CREATE DATABASE prometheus;
CREATE TABLE prometheus.metrics ENGINE = TimeSeries;
```

## Ingest metrics with remote write

ClickHouse supports the [Prometheus remote-write protocol](https://prometheus.io/docs/specs/remote_write_spec/). Configure Prometheus to write to the handler:

```yaml theme={null}
remote_write:
  - url: https://clickhouse.example.com:8443/prometheus/api/v1/write?database=prometheus&table=metrics
    basic_auth:
      username: default
      password: <password>
```

Prometheus sends samples to the `prometheus.metrics` table.

To batch data from many concurrent remote-write requests into fewer parts, ClickHouse uses [asynchronous inserts](https://clickhouse.com/docs/reference/settings/session-settings/async-insert.md#async_insert). They are enabled by default, and the `async_insert` setting in the URL (or in the user profile) turns them on or off.

ClickHouse acknowledges an asynchronous remote-write request only after the data is flushed to all inner tables of the `TimeSeries` table, regardless of the [`wait_for_async_insert`](https://clickhouse.com/docs/reference/settings/session-settings/wait-for#wait_for_async_insert) setting: the remote-write protocol treats an acknowledged write as durable. If the flush fails, the request returns an error and Prometheus retries it.

Because each request waits for its flush, asynchronous inserts limit how fast one sender can write. Prometheus sends one request at a time per remote-write shard. When several shards send at the same time, the adaptive flush timeout grows to [`async_insert_busy_timeout_max_ms`](https://clickhouse.com/docs/reference/settings/session-settings/async-insert#async_insert_busy_timeout_max_ms), unless the pending requests reach [`async_insert_max_data_size`](https://clickhouse.com/docs/reference/settings/session-settings/async-insert#async_insert_max_data_size) first. A request waits until its batch is flushed, which happens up to that long after the batch is started, so under sustained load each shard writes roughly `samples per request / async_insert_busy_timeout_max_ms`, where Prometheus sends up to 2,000 samples per request by default. To raise it:

* Raise `max_shards` (50 by default) or `max_samples_per_send` in the `queue_config` of [`remote_write`](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write). Prometheus adds shards on its own when it falls behind, up to `max_shards`.
* Lower `async_insert_busy_timeout_max_ms` in the URL or in the user profile, for example `&async_insert_busy_timeout_max_ms=50`. Flushes then happen more often, and each flush creates new parts in the inner tables.
* Disable asynchronous inserts with `async_insert=0`. Each request then creates its own parts in the inner tables.

## Query with PromQL

Use the instant-query endpoint to evaluate a PromQL expression at one point in time:

```bash theme={null}
curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/query" \
  --data-urlencode "query=rate(http_requests_total[5m])" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"
```

Use the range-query endpoint to evaluate an expression over a time range:

```bash theme={null}
curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/query_range" \
  --data-urlencode "query=rate(http_requests_total[5m])" \
  --data-urlencode "start=2026-08-15T12:00:00Z" \
  --data-urlencode "end=2026-08-15T13:00:00Z" \
  --data-urlencode "step=60s" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"
```

The query endpoints also accept the parameters in a form body. Without `--get`, curl sends the parameters as `application/x-www-form-urlencoded` over `POST`:

```bash theme={null}
curl --user default:<password> \
  "https://clickhouse.example.com:8443/prometheus/api/v1/query_range" \
  --data-urlencode "query=rate(http_requests_total[5m])" \
  --data-urlencode "start=2026-08-15T12:00:00Z" \
  --data-urlencode "end=2026-08-15T13:00:00Z" \
  --data-urlencode "step=60s" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"
```

Use the format-query endpoint to parse and format a PromQL expression without evaluating it:

```bash theme={null}
curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/format_query" \
  --data-urlencode "query=sum by(job)(http_requests_total{code=\"200\"})/2"
```

The expression is returned serialized from the parsed query, with the whitespace normalized, the comments removed, the redundant parentheses dropped, and the durations printed in the units the parser accepts: `sum by (job) (http_requests_total{code="200"}) / 2`. A duration keeps its units rather than becoming a number of seconds, so `rate(x[5m])` comes back as `rate(x[5m])`, and one that is not a whole number of a single unit is spelled with each unit it needs, so `rate(x[90s])` comes back as `rate(x[1m30s])`. This endpoint doesn't evaluate the expression, so it doesn't need the `database` and `table` parameters.

See the [supported PromQL features](https://clickhouse.com/docs/reference/functions/table-functions/prometheusQueryRange.md#supported-promql-features) for the function and aggregation operator list used by the HTTP API, the `promql` dialect, and the table functions.

### Grafana

Configure a Prometheus data source with the base URL ending before `/api/v1`:

```yaml theme={null}
apiVersion: 1
datasources:
  - name: ClickHouse Prometheus
    type: prometheus
    access: proxy
    url: https://clickhouse.example.com:8443/prometheus
    basicAuth: true
    basicAuthUser: default
    jsonData:
      httpMethod: POST
      customQueryParameters: database=prometheus&table=metrics
    secureJsonData:
      basicAuthPassword: <password>
```

Grafana appends `/api/v1/query` or `/api/v1/query_range` to this base URL and adds `customQueryParameters` to each request.

With `httpMethod: POST`, Grafana sends the query parameters in the request body. ClickHouse reads the request body and the URL query string, so `customQueryParameters` still applies. Use `POST` for long PromQL expressions, because a URL has a length limit.

> [!NOTE]
>
> Only the query endpoints `/api/v1/query`, `/api/v1/query_range`, and `/api/v1/format_query`, the metadata endpoints `/api/v1/series`, `/api/v1/labels`, `/api/v1/label/<name>/values`, and `/api/v1/metadata`, and the [federation](#federation) endpoint `/api/v1/federate` are implemented. `/api/v1/series` requires at least one `match[]` series selector, supports the optional `start`, `end`, and `limit` parameters, and returns the union of the series matched by each selector. `/api/v1/labels` accepts the same parameters, with `match[]` being optional, and returns the sorted label names of the matched series (or of all series when no selectors are given). `/api/v1/label/<name>/values` accepts the same parameters as `/api/v1/labels` and returns the sorted values of one label, with `<name>` optionally using the Prometheus `U__...` escaping of label names that contain characters outside `[a-zA-Z0-9_]`. These endpoints cover what a Grafana Prometheus datasource uses for label browsing, template variables, and query-builder autocomplete.

### SQL entry points

ClickHouse uses the same PromQL converter for the HTTP API, the `promql` dialect, and the [`prometheusQuery`](https://clickhouse.com/docs/reference/functions/table-functions/prometheusQuery) and [`prometheusQueryRange`](https://clickhouse.com/docs/reference/functions/table-functions/prometheusQueryRange) table functions.

Run PromQL directly with `clickhouse-client`:

```bash theme={null}
clickhouse-client \
  --dialect promql \
  --promql_database prometheus \
  --promql_table metrics \
  --query 'rate(http_requests_total[5m])'
```

Use the table functions to embed PromQL in a SQL query:

```sql theme={null}
SELECT *
FROM prometheusQuery(
    prometheus.metrics,
    'rate(http_requests_total[5m])',
    now()
);
```

## Query metric metadata

The `/prometheus/api/v1/metadata` endpoint returns the metric metadata stored in the `Metrics` target table of the `TimeSeries` table: the type, help text, and unit of each metric family. It supports the following Prometheus parameters in the URL query string:

| Parameter | Description |
| - | - |
| `metric` | Return metadata only for this metric family. |
| `limit` | Limit the number of returned metric families. A negative value means no limit; zero returns no metric families. |
| `limit_per_metric` | Limit the number of metadata objects returned for each metric family. Zero and negative values mean no limit. |

The default `Metrics` target table is a `ReplacingMergeTree` ordered by the metric family name: it keeps the most recently written metadata entry for each metric family. Several entries per family are returned only while the target table stores them — before its parts are merged, or when the table is defined with an engine that preserves them.

```bash theme={null}
curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/metadata" \
  --data-urlencode "metric=http_requests_total" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"
```

## Federate metrics

ClickHouse supports the [Prometheus federation](https://prometheus.io/docs/prometheus/latest/federation/) endpoint at `/prometheus/api/v1/federate` (or at any path ending with `/federate` served by a handler of type `query`). For each series matched by the `match[]` selectors, it returns the latest sample of the last 5 minutes in the Prometheus text format, so a Prometheus server can scrape it:

```yaml theme={null}
scrape_configs:
  - job_name: clickhouse-federate
    honor_labels: true
    scheme: https
    metrics_path: /prometheus/api/v1/federate
    params:
      'match[]': ['{job="node"}']
      table: ['prometheus.metrics']
    static_configs:
      - targets: ['clickhouse.example.com:8443']
```

## Read metrics with remote read

ClickHouse supports the [Prometheus remote-read protocol](https://prometheus.io/docs/prometheus/latest/querying/remote_read_api/) at `/prometheus/api/v1/read`.

Configure a Prometheus server to read from the same `TimeSeries` table:

```yaml theme={null}
remote_read:
  - url: https://clickhouse.example.com:8443/prometheus/api/v1/read?database=prometheus&table=metrics
    basic_auth:
      username: default
      password: <password>
```

## Recording rules

ClickHouse doesn't evaluate Prometheus rule files. A [refreshable materialized view](https://clickhouse.com/docs/reference/statements/create/view.md#refreshable-materialized-view) can do the work of a [recording rule](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/): it evaluates a PromQL expression on a schedule and appends the result to a `TimeSeries` table. The following view records `sum by (job) (rate(http_requests_total[5m]))` as `job:http_requests:rate5m` every minute:

```sql theme={null}
CREATE MATERIALIZED VIEW prometheus.job_http_requests_rate5m
REFRESH EVERY 1 MINUTE APPEND TO prometheus.metrics (tags, samples) AS
SELECT
    mapUpdate(CAST(tags, 'Map(String, String)'), map('__name__', 'job:http_requests:rate5m')) AS tags,
    [(timestamp, value)] AS samples
FROM prometheusQuery(prometheus.metrics, 'sum by (job) (rate(http_requests_total[5m]))', toStartOfMinute(now()));
```

Each refresh evaluates the expression at the start of the current minute, sets the metric name with `mapUpdate`, and appends one sample for each series of the result. Extra labels can be added to the same map, for example `map('__name__', 'job:http_requests:rate5m', 'team', 'web')`. The recorded series is queried like any other metric:

```bash theme={null}
curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/query" \
  --data-urlencode "query=job:http_requests:rate5m" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"
```

The time and the error of the last evaluation are shown in [`system.view_refreshes`](https://clickhouse.com/docs/reference/system-tables/view_refreshes).

A view differs from a Prometheus recording rule in these ways:

* The evaluation time is `now()` when the refresh runs. Besides the schedule, a view refreshes right after it's created (unless it's created with `EMPTY`), after a server restart if its refreshes are not coordinated (a view outside a Replicated or Shared database, or one with `all_replicas = 1`, doesn't remember its last refresh), and when `SYSTEM START VIEW` resumes it after it missed a scheduled refresh. Round `now()` down to the interval as the example does, so that these refreshes don't add samples between the scheduled ones. An extra refresh can still write a second sample at the same rounded timestamp. For a five-minute rule, use `toStartOfInterval(now(), INTERVAL 5 MINUTE)`.
* Missed evaluations are not caught up. After the server was down or the view was stopped, the view evaluates only the current interval, and the intervals before it stay empty. The same happens when a refresh takes longer than the interval.
* A series that drops out of the result gets no staleness marker, so queries keep returning its last recorded value for the lookback period, 5 minutes by default.
* There is no rule file: each rule is a separate view. The rules are not listed by `/api/v1/rules`.
