Skip to main content
This page covers self-managed ClickHouse Observability. For the recommended fully managed experience, see the documentation here.
When using Self-managed ClickHouse Observability in production, consider the following security, stability, and configuration practices. Self-managed ClickHouse Observability applies industry-standard security practices by default - including enhanced encryption, authentication and connectivity, and managed access controls, as well as providing the following benefits: Follow these best practices for ClickHouse Cloud when using Self-managed ClickHouse Observability.

​
Secure ingestion

By default, the ClickStack OpenTelemetry Collector isn’t secured when deployed outside of the Open Source distributions and doesn’t require authentication on its OTLP ports. To secure ingestion, specify an authentication token when deploying the collector using the OTLP_AUTH_TOKEN environment variable. See “Securing the collector” for further details.

​
Create an ingestion user

It’s recommended to create a dedicated user for the OTel collector for ingestion into ClickHouse Cloud and ensuring ingestion is sent to a specific database e.g. otel. See “Creating an ingestion user” for further details.

​
Configure Time To Live (TTL)

Ensure the Time To Live (TTL) has been appropriately configured for your Self-managed ClickHouse Observability deployment. This controls how long data is retained for - the default of 3 days often needs to be modified.

​
Estimating Resources

The following provides a model for estimating the compute and storage resources required for an observability deployment based on your expected ingest volume. The values produced are estimates only and should be used as an initial baseline - they are not a prescriptive answer. Actual requirements depend on query complexity, concurrency, retention policies, and variance in ingestion throughput. Always monitor resource usage and scale as needed.
All figures are based on uncompressed raw ingestEvery number on this page - throughput (MB/s, TB/month), CPU sizing, and storage - is expressed in terms of uncompressed raw ingest volume, i.e. the size of the data as produced by your applications and sent to the OpenTelemetry collector before any compression is applied.This is the figure you should estimate from your existing logs, traces, and metrics pipelines. Storage figures in the table below already have the assumed 10x compression ratio applied to this raw volume.
Provision compute to cover two independent workloads: ingest and query.
Isolation of Queries vs IngestIn most self-managed deployments, ingest and query share the same nodes. In this case, use the Total CPUs as your baseline. Isolated scaling - where ingest and query compute are provisioned independently - is supported in ClickHouse Cloud through separate compute pools aka Warehouses.
  • A 10x compression ratio for storage - typically conservative for logs and traces.
  • Query SLAs of a P50 of 1.5 seconds and a P99 of 5 seconds.
  • We assume most queries occur over recent data, following a log-normal distribution that peaks at around one hour and tails out to around six hours. Users may wish to provision dedicated compute to query older data. In ClickHouse Cloud this can be idle (thus not incurring costs) when not in use.
  • While query compute can be scaled independently of ingest compute, it remains intrinsically linked to ingest volume. We assume as ingest increases, data density grows, resulting in larger scan volumes at query time and consequently higher query compute requirements.
The following table provides example sizings based on increasing ingest throughput in megabytes per second, alongside the corresponding data volumes in terabytes per month. This assumes a sustained average of 1 QPS across all query types (search, dashboards, alerting). For more details on refining sizing assumptions for your environment, see “Refining sizing assumptions for your environment”.

​
Isolating observability workloads

If you’re adding Self-managed ClickHouse Observability to an existing ClickHouse Cloud service that already supports other workloads, such as real-time application analytics, isolating observability traffic is strongly recommended. Use Managed Warehouses to create a child service dedicated to observability. This allows you to:
  • Isolate ingest and query load from existing applications
  • Scale observability workloads independently
  • Prevent observability queries from impacting production analytics
  • Share the same underlying datasets across services when needed
This approach ensures your existing workloads remain unaffected while allowing observability workloads to scale independently as observability data grows.

​
Isolating read and write workloads

Warehouses also allow ingest and query workloads to be separated onto dedicated compute, with ingestion targeting a read-write service and the observability UI running on a read-only service. This is the recommended production topology for larger deployments. See “Isolating read and write workloads”. For larger deployments or custom sizing guidance, please contact support for a more precise estimate.

​
Query-user permissions

A ClickHouse user used by an external query client only needs to be a readonly user with access to change the following settings:
  • max_rows_to_read (at least up to 1 million)
  • read_overflow_mode
  • cancel_http_readonly_queries_on_client_close
  • wait_end_of_query
By default, the default user in ClickHouse Cloud will have these permissions available however you’re recommended to create a new user with these permissions.