Skip to main content
Skip to main content

Setting up your OpenTelemetry Collector

This guide walks you through deploying an OpenTelemetry (OTel) Collector against an existing Managed ClickStack service, then verifying that data is flowing through it.

The collector runs as a gateway: a single OTLP endpoint that your applications, SDKs, and agent collectors send to. The gateway batches events, applies any processing you've configured, and writes them to ClickHouse via the ClickHouse exporter. This pattern keeps collection logic out of your application code and lets you scale ingestion independently of the workloads producing data. For background on gateway versus agent roles, see Collector roles.

This guide assumes you've completed the Getting started with Managed ClickStack guide and have your connection credentials to hand.

Gather your credentials​

You'll need:

  • The HTTPS endpoint of your ClickHouse Cloud service, including protocol and port, for example https://abc123xyz.us-central1.gcp.clickhouse.cloud:8443.
  • A ClickHouse username and password for ingestion.

If you don't have these recorded, open your service in the ClickHouse Cloud console and select Connect. Record the url from the subsequent dialog. We will create a dedicated user for ingestion below.

Create an ingestion user​

We recommend creating a dedicated user for the collector rather than reusing default. Connect to your service via the SQL console and run:

CREATE USER hyperdx_ingest IDENTIFIED WITH sha256_password BY 'ClickH0u3eRocks123!';
GRANT SELECT, INSERT, CREATE DATABASE, CREATE TABLE, CREATE VIEW ON otel.* TO hyperdx_ingest;
Tip

Replace the password in the snippet above with a strong value

The collector creates the schema for logs, traces, and metrics inside the otel database on first use. For more guidance on production user setup, see Going to production.

Deploy the collector​

Deploy the collector somewhere that's accessible to the applications and infrastructure sending OpenTelemetry data. In the example below, we run the collector locally and generate artificial telemetry from the same machine for simplicity.

info

In production, you would typically deploy the collector in a Kubernetes cluster, or on a virtual machine that can be reached by your OpenTelemetry SDKs, agents, and other collectors. This allows telemetry from across your environment to be centrally collected and forwarded to ClickStack.

Pick a shared secret to authenticate clients sending data to the collector, then export it alongside your connection details and chosen password for the hyperdx_ingest user:

export CLICKHOUSE_ENDPOINT=<HTTPS_ENDPOINT>
export CLICKHOUSE_USER=hyperdx_ingest
export CLICKHOUSE_PASSWORD=ClickH0u3eRocks123!
export OTLP_AUTH_TOKEN="a-strong-shared-secret"

Run the ClickStack OTel collector:

docker run -d \
  -e OTLP_AUTH_TOKEN=${OTLP_AUTH_TOKEN} \
  -e CLICKHOUSE_ENDPOINT=${CLICKHOUSE_ENDPOINT} \
  -e CLICKHOUSE_USER=${CLICKHOUSE_USER} \
  -e CLICKHOUSE_PASSWORD=${CLICKHOUSE_PASSWORD} \
  -e HYPERDX_OTEL_EXPORTER_CLICKHOUSE_DATABASE=otel \
  -p 4317:4317 \
  -p 4318:4318 \
  clickhouse/clickstack-otel-collector:latest

The collector now exposes OTLP gRPC on 4317 and OTLP HTTP on 4318. Applications, SDKs, and agent collectors should send to these ports with authorization: $OTLP_AUTH_TOKEN in the request headers.

Production deployments

For production, we recommend enabling TLS on the OTLP endpoint. See Securing the collector.

Verify the endpoint​

Generate some synthetic traffic against the collector to confirm the full pipeline works. We use otelgen, a small CLI that emits OTLP logs, traces, and metrics.

Install otelgen with Homebrew:

brew install krzko/tap/otelgen

Or with Go:

go install github.com/krzko/otelgen@latest

Send a short burst of logs to the collector:

 otelgen \
  --otel-exporter-otlp-endpoint localhost:4317 \
  --insecure \
  --protocol grpc \
  --header "authorization=${OTLP_AUTH_TOKEN}" \
  --rate 5 \
  --duration 60 \
  logs multi

For the equivalent trace and metrics commands, and a walkthrough of the other otelgen subcommands, see Synthetic data with otelgen.

Confirm in the ClickStack UI​

Open your service in the ClickHouse Cloud console and select ClickStack from the left menu and then Start Ingestion.

The next step can be skipped, as you’ve already configured your collector. Click Launch ClickStack to continue.

ClickStack will open in a new tab and you should be automatically directed to the Getting Started page. If not, select Getting Started from the left-hand menu, then click Start Ingestion followed by Next.

ClickStack should automatically detect your tables and telemetry data, allowing you to proceed. Select Start Exploring to begin exploring your trace data.

Switch the source to Logs and set the time range to Last 15 minutes. The synthetic logs from otelgen should appear within a few seconds.

If nothing shows up:

  • Confirm the OTLP_AUTH_TOKEN value passed to otelgen matches the one set on the collector.
  • Tail the collector logs with docker logs -f <container-id> and look for export errors.
  • Verify the CLICKHOUSE_ENDPOINT includes both the protocol and port (https://...:8443).

Further reading​

This guide covers a single collector instance in its simplest form. The OpenTelemetry collector reference covers what to do next: