Prometheus HTTP API and PromQL

ClickHouse implements the Prometheus HTTP API over a TimeSeries table. One handler serves remote write, remote read, instant PromQL queries, and range PromQL queries. To expose ClickHouse’s own metrics for a Prometheus server to scrape, see the Prometheus metrics endpoint.

Prerequisites

The setup steps differ between ClickHouse Cloud and self-managed ClickHouse. Follow the section for your deployment.

ClickHouse Cloud

PromQL support in ClickHouse Cloud is in private preview. The services that take part in the private preview already have the enable_time_series_table setting and the Prometheus API endpoints configured. Other ClickHouse Cloud services do not have this configuration, and you cannot enable the feature yourself on such a service. The SET enable_time_series_table statement and the http_handlers configuration in the next sections apply to self-managed deployments.

On a service that takes part in the private preview, continue at Create a TimeSeries table. The service serves the endpoint paths listed in the endpoint table.

Self-managed: enable the TimeSeries setting

Enable the enable_time_series_table setting for the user that creates and accesses the table:

SET enable_time_series_table = 1;

For HTTP API requests, enable enable_time_series_table in the profile of the API user.

Self-managed: configure the Prometheus API endpoints

Configure one prefix-routed handler on the main ClickHouse HTTP port:

<http_handlers>
    <defaults/>
    <rule>
        <url_prefix>/prometheus/api/v1</url_prefix>
        <handler>
            <type>prometheus_api_v1</type>
        </handler>
    </rule>
</http_handlers>

<defaults/> preserves the built-in handlers for endpoints such as /ping and for SQL requests. The prefix above exposes these endpoints through one handler:

Endpoint	Purpose
`/prometheus/api/v1/write`	Prometheus remote write
`/prometheus/api/v1/read`	Prometheus remote read
`/prometheus/api/v1/query`	Instant PromQL queries
`/prometheus/api/v1/query_range`	Range PromQL queries
`/prometheus/api/v1/format_query`	PromQL expression formatting
`/prometheus/api/v1/series`	Series metadata
`/prometheus/api/v1/metadata`	Metric-family metadata

The example omits database and table from the handler. Each request must provide the table query parameter (except for /format_query, which only parses the given PromQL expression and doesn’t need a table). It can also provide database, use a qualified table name such as prometheus.metrics, or omit the database to use default. This allows one handler to serve multiple TimeSeries tables. To use one fixed table for every request, configure it in the handler:

<handler>
    <type>prometheus_api_v1</type>
    <database>prometheus</database>
    <table>metrics</table>
</handler>

A table configured in the handler cannot be overridden by request parameters. Routing and handler settings:

Name	Default	Description
`url_prefix`	none	Rule filter that matches every request path that starts with the configured prefix.
`table`	none	The name of a `TimeSeries` table. When omitted, the request must provide the `table` query parameter. The configured name can include a database.
`database`	none	The database containing the table. A request can provide it as a query parameter. When omitted, ClickHouse uses a database from a qualified `table` value or falls back to `default`.

Create a TimeSeries table

Create a database and a TimeSeries table:

CREATE DATABASE prometheus;
CREATE TABLE prometheus.metrics ENGINE = TimeSeries;

Ingest metrics with remote write

ClickHouse supports the Prometheus remote-write protocol. Configure Prometheus to write to the handler:

remote_write:
  - url: https://clickhouse.example.com:8443/prometheus/api/v1/write?database=prometheus&table=metrics
    basic_auth:
      username: default
      password: <password>

Prometheus sends samples to the prometheus.metrics table. To batch data from many concurrent remote-write requests into fewer parts, ClickHouse uses asynchronous inserts. They are enabled by default, and the async_insert setting in the URL (or in the user profile) turns them on or off. ClickHouse acknowledges an asynchronous remote-write request only after the data is flushed to all inner tables of the TimeSeries table, regardless of the wait_for_async_insert setting: the remote-write protocol treats an acknowledged write as durable. If the flush fails, the request returns an error and Prometheus retries it. Because each request waits for its flush, asynchronous inserts limit how fast one sender can write. Prometheus sends one request at a time per remote-write shard. When several shards send at the same time, the adaptive flush timeout grows to async_insert_busy_timeout_max_ms, unless the pending requests reach async_insert_max_data_size first. A request waits until its batch is flushed, which happens up to that long after the batch is started, so under sustained load each shard writes roughly samples per request / async_insert_busy_timeout_max_ms, where Prometheus sends up to 2,000 samples per request by default. To raise it:

Raise max_shards (50 by default) or max_samples_per_send in the queue_config of remote_write. Prometheus adds shards on its own when it falls behind, up to max_shards.
Lower async_insert_busy_timeout_max_ms in the URL or in the user profile, for example &async_insert_busy_timeout_max_ms=50. Flushes then happen more often, and each flush creates new parts in the inner tables.
Disable asynchronous inserts with async_insert=0. Each request then creates its own parts in the inner tables.

Query with PromQL

Use the instant-query endpoint to evaluate a PromQL expression at one point in time:

curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/query" \
  --data-urlencode "query=rate(http_requests_total[5m])" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"

Use the range-query endpoint to evaluate an expression over a time range:

curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/query_range" \
  --data-urlencode "query=rate(http_requests_total[5m])" \
  --data-urlencode "start=2026-08-15T12:00:00Z" \
  --data-urlencode "end=2026-08-15T13:00:00Z" \
  --data-urlencode "step=60s" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"

The query endpoints also accept the parameters in a form body. Without --get, curl sends the parameters as application/x-www-form-urlencoded over POST:

curl --user default:<password> \
  "https://clickhouse.example.com:8443/prometheus/api/v1/query_range" \
  --data-urlencode "query=rate(http_requests_total[5m])" \
  --data-urlencode "start=2026-08-15T12:00:00Z" \
  --data-urlencode "end=2026-08-15T13:00:00Z" \
  --data-urlencode "step=60s" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"

Use the format-query endpoint to parse and format a PromQL expression without evaluating it:

curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/format_query" \
  --data-urlencode "query=sum by(job)(http_requests_total{code=\"200\"})/2"

The expression is returned serialized from the parsed query, with the whitespace normalized, the comments removed, the redundant parentheses dropped, and the durations printed in the units the parser accepts: sum by (job) (http_requests_total{code="200"}) / 2. A duration keeps its units rather than becoming a number of seconds, so rate(x[5m]) comes back as rate(x[5m]), and one that is not a whole number of a single unit is spelled with each unit it needs, so rate(x[90s]) comes back as rate(x[1m30s]). This endpoint doesn’t evaluate the expression, so it doesn’t need the database and table parameters. See the supported PromQL features for the function and aggregation operator list used by the HTTP API, the promql dialect, and the table functions.

Grafana

Configure a Prometheus data source with the base URL ending before /api/v1:

apiVersion: 1
datasources:
  - name: ClickHouse Prometheus
    type: prometheus
    access: proxy
    url: https://clickhouse.example.com:8443/prometheus
    basicAuth: true
    basicAuthUser: default
    jsonData:
      httpMethod: POST
      customQueryParameters: database=prometheus&table=metrics
    secureJsonData:
      basicAuthPassword: <password>

Grafana appends /api/v1/query or /api/v1/query_range to this base URL and adds customQueryParameters to each request. With httpMethod: POST, Grafana sends the query parameters in the request body. ClickHouse reads the request body and the URL query string, so customQueryParameters still applies. Use POST for long PromQL expressions, because a URL has a length limit.

Only the query endpoints /api/v1/query, /api/v1/query_range, and /api/v1/format_query and the metadata endpoints /api/v1/series, /api/v1/labels, /api/v1/label/<name>/values, and /api/v1/metadata are implemented. /api/v1/series requires at least one match[] series selector, supports the optional start, end, and limit parameters, and returns the union of the series matched by each selector. /api/v1/labels accepts the same parameters, with match[] being optional, and returns the sorted label names of the matched series (or of all series when no selectors are given). /api/v1/label/<name>/values accepts the same parameters as /api/v1/labels and returns the sorted values of one label, with <name> optionally using the Prometheus U__... escaping of label names that contain characters outside [a-zA-Z0-9_]. These endpoints cover what a Grafana Prometheus datasource uses for label browsing, template variables, and query-builder autocomplete.

SQL entry points

ClickHouse uses the same PromQL converter for the HTTP API, the promql dialect, and the prometheusQuery and prometheusQueryRange table functions. Run PromQL directly with clickhouse-client:

clickhouse-client \
  --dialect promql \
  --promql_database prometheus \
  --promql_table metrics \
  --query 'rate(http_requests_total[5m])'

Use the table functions to embed PromQL in a SQL query:

SELECT *
FROM prometheusQuery(
    prometheus.metrics,
    'rate(http_requests_total[5m])',
    now()
);

Query metric metadata

The /prometheus/api/v1/metadata endpoint returns the metric metadata stored in the Metrics target table of the TimeSeries table: the type, help text, and unit of each metric family. It supports the following Prometheus parameters in the URL query string:

Parameter	Description
`metric`	Return metadata only for this metric family.
`limit`	Limit the number of returned metric families. A negative value means no limit; zero returns no metric families.
`limit_per_metric`	Limit the number of metadata objects returned for each metric family. Zero and negative values mean no limit.

The default Metrics target table is a ReplacingMergeTree ordered by the metric family name: it keeps the most recently written metadata entry for each metric family. Several entries per family are returned only while the target table stores them — before its parts are merged, or when the table is defined with an engine that preserves them.

curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/metadata" \
  --data-urlencode "metric=http_requests_total" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"

Read metrics with remote read

ClickHouse supports the Prometheus remote-read protocol at /prometheus/api/v1/read. Configure a Prometheus server to read from the same TimeSeries table:

remote_read:
  - url: https://clickhouse.example.com:8443/prometheus/api/v1/read?database=prometheus&table=metrics
    basic_auth:
      username: default
      password: <password>

Recording rules

ClickHouse doesn’t evaluate Prometheus rule files. A refreshable materialized view can do the work of a recording rule: it evaluates a PromQL expression on a schedule and appends the result to a TimeSeries table. The following view records sum by (job) (rate(http_requests_total[5m])) as job:http_requests:rate5m every minute:

CREATE MATERIALIZED VIEW prometheus.job_http_requests_rate5m
REFRESH EVERY 1 MINUTE APPEND TO prometheus.metrics (tags, samples) AS
SELECT
    mapUpdate(CAST(tags, 'Map(String, String)'), map('__name__', 'job:http_requests:rate5m')) AS tags,
    [(timestamp, value)] AS samples
FROM prometheusQuery(prometheus.metrics, 'sum by (job) (rate(http_requests_total[5m]))', toStartOfMinute(now()));

Each refresh evaluates the expression at the start of the current minute, sets the metric name with mapUpdate, and appends one sample for each series of the result. Extra labels can be added to the same map, for example map('__name__', 'job:http_requests:rate5m', 'team', 'web'). The recorded series is queried like any other metric:

curl --user default:<password> --get \
  "https://clickhouse.example.com:8443/prometheus/api/v1/query" \
  --data-urlencode "query=job:http_requests:rate5m" \
  --data-urlencode "database=prometheus" \
  --data-urlencode "table=metrics"

The time and the error of the last evaluation are shown in system.view_refreshes. A view differs from a Prometheus recording rule in these ways:

The evaluation time is now() when the refresh runs. Besides the schedule, a view refreshes right after it’s created (unless it’s created with EMPTY), after a server restart if its refreshes are not coordinated (a view outside a Replicated or Shared database, or one with all_replicas = 1, doesn’t remember its last refresh), and when SYSTEM START VIEW resumes it after it missed a scheduled refresh. Round now() down to the interval as the example does, so that these refreshes don’t add samples between the scheduled ones. An extra refresh can still write a second sample at the same rounded timestamp. For a five-minute rule, use toStartOfInterval(now(), INTERVAL 5 MINUTE).
Missed evaluations are not caught up. After the server was down or the view was stopped, the view evaluates only the current interval, and the intervals before it stay empty. The same happens when a refresh takes longer than the interval.
A series that drops out of the result gets no staleness marker, so queries keep returning its last recorded value for the lookback period, 5 minutes by default.
There is no rule file: each rule is a separate view. The rules are not listed by /api/v1/rules.

Fundamentals

ClickHouse concepts

Best practices

Prerequisites

ClickHouse Cloud

Self-managed: enable the TimeSeries setting

Self-managed: configure the Prometheus API endpoints

Create a TimeSeries table

Ingest metrics with remote write

Query with PromQL

Grafana

SQL entry points

Query metric metadata

Read metrics with remote read

Recording rules

​Prerequisites

​ClickHouse Cloud

​Self-managed: enable the TimeSeries setting

​Self-managed: configure the Prometheus API endpoints

​Create a TimeSeries table

​Ingest metrics with remote write

​Query with PromQL

​Grafana

​SQL entry points

​Query metric metadata

​Read metrics with remote read

​Recording rules

Prerequisites

ClickHouse Cloud

Self-managed: enable the TimeSeries setting

Self-managed: configure the Prometheus API endpoints

Create a TimeSeries table

Ingest metrics with remote write

Query with PromQL

Grafana

SQL entry points

Query metric metadata

Read metrics with remote read

Recording rules