Announcing support for ClickStack in the ClickHouse Terraform provider

A dashboard built for one service rarely stays in one environment. It gets recreated in staging and production, along with its filters, saved searches, and alerts. Once those copies exist, keeping them aligned means repeating changes through the UI and checking each environment by hand.

ClickStack configuration can now follow the same workflow as the rest of your infrastructure. The official ClickHouse Terraform provider now supports ClickStack resources for self-hosted deployments and ClickHouse Cloud. Dashboards, alerts, sources, saved searches, connections, and webhooks can live in version control, pass through code review, and be applied with `terraform plan` and `terraform apply`.

Support for self-hosted ClickStack and Managed ClickStack resources are available in the regular provider release as beta from v3.25. Their behavior and configuration may thus change as we get more production feedback.

The ClickHouse Terraform provider #

ClickHouse maintains two official Terraform providers with different jobs:

• ClickHouse/clickhouse manages resources in the ClickHouse Cloud control plane, including services, private endpoints, ClickPipes, organization access, and Managed Postgres.
• ClickHouse/clickhousedbops connects to a ClickHouse instance to manage database-level users, roles, grants, and databases.

ClickStack support belongs in ClickHouse/clickhouse. In feedback from ClickStack users, Terraform was the most common request across both self-hosted and Cloud deployments. We initially developed that support as a separate effort, but realised that publishing another provider would have duplicated release work, testing, documentation, and authentication code.

The ClickStack provider was therefore added as a dedicated service module within the existing ClickHouse/clickhouse provider, keeping its resources and data sources separate from the ClickHouse Cloud and Postgres implementations. Users continue to install a single provider and follow the same release path.

This also keeps authentication consistent in ClickHouse Cloud. ClickStack resources use the same organization ID and Cloud API credentials as the rest of the provider, with a ClickStack service ID identifying the target deployment. Self-hosted, open-source ClickStack uses its own endpoint and personal API access key. We will show both configurations later in the post.

Building on the ClickStack API #

A Terraform resource needs a predictable API contract. Resource identifiers must remain stable across reads and updates. Create, update, delete, and import behavior must be explicit. Errors also need enough structure to point users back to the field that failed during planning.

The ClickStack API work earlier this year, exposing observability resources through the existing ClickHouse Cloud service path:

/v1/organizations/{organizationId}/services/{serviceId}/clickstack/…

Supporting infrastructure tooling also required changes to the API definition. Inline schemas were replaced with named types, numeric fields were defined consistently as integers, and validation failures gained structured error details. Those changes make the OpenAPI contract usable by generated clients and give Terraform enough information to report configuration errors clearly.

Managing dashboards introduced an additional requirement: dashboard definitions are supplied as JSON and need to be validated before they are applied. During terraform plan, the provider sends the definition to the ClickStack validation API when the endpoint is available. Invalid configurations are therefore caught before terraform apply makes any changes.

Note: If the validation endpoint is unavailable e.g. due to earlier versions of ClickStack, the provider emits a warning instead of blocking terraform plan. Validation is then deferred until terraform apply.

With these changes in place, users can manage ClickStack resources through the ClickHouse Terraform provider across both open-source and Managed ClickStack deployments.

Using ClickStack resources #

While the following example uses the provider for configuring Managed ClickStack on ClickHouse Cloud, the same clickhouse_clickstack_dashboard resource also works with open-source self-hosted ClickStack. The resource definition stays the same but authentication and team scoping differ between the two deployments.

You will need Terraform 1.5 or later, at least version 3.25 of the ClickHouse provider, and an existing Managed ClickStack service. The dashboard below also assumes you have an OpenTelemetry logs source and ClickHouse connection configured.

Create a ClickHouse Cloud API key #

The provider calls the ClickHouse Cloud API on your behalf. In the ClickHouse Cloud console, open Organization → API keys, select New API key, and create a key with Service Admin or Org Admin permissions. Save the key ID and secret somewhere secure.

You also need two non-secret identifiers:

• The organization ID for the ClickHouse Cloud organization.
• The service ID of the Managed ClickStack service you want Terraform to manage.

Copy both from the Cloud console. Make sure the service ID belongs to the service where you open ClickStack, rather than another ClickHouse service in the same organization.

> Cloud and self-hosted credentials - ClickHouse Cloud uses the organization ID, Cloud API key ID, Cloud API secret, and ClickStack service ID. A Managed ClickStack service maps to one team, so do not set the team on Cloud resources. Conversely, self-hosted ClickStack uses CLICKSTACK_ENDPOINT and CLICKSTACK_API_KEY instead. The key must be a personal API access key created in the ClickStack UI. For a non-default team, set the resource's team attribute to the team ID. Do not configure Cloud and self-hosted credentials in the same unaliased provider block.

Export the Cloud credentials #

The provider reads credentials from environment variables. This keeps them out of the Terraform files and avoids committing a secret in `terraform.tfvars`.

1export CLICKHOUSE_ORG_ID="<organization-id>"
2export CLICKSTACK_SERVICE_ID="<managed-clickstack-service-id>"
3export CLICKHOUSE_CLOUD_API_KEY="<api-key-id>"
4export CLICKHOUSE_CLOUD_API_SECRET="<api-key-secret>"

> For a self-hosted deployment, replace those Cloud environment variables with:

1export CLICKSTACK_ENDPOINT="https://clickstack.example.com"
2export CLICKSTACK_API_KEY="<personal-api-access-key>"

Use your CI system's secret store when running Terraform in a deployment pipeline. The variable names can remain the same.

In our example, we’ll load dashboard objects. These refer to ClickStack sources and connections by ID. You can retrieve the available values from the ClickStack API using the credentials you exported above.

1curl --silent \
2  --user "${CLICKHOUSE_CLOUD_API_KEY}:${CLICKHOUSE_CLOUD_API_SECRET}" \
3"https://api.clickhouse.cloud/v1/organizations/${CLICKHOUSE_ORG_ID}/services/${CLICKSTACK_SERVICE_ID}/clickstack/sources" \
4| jq -r '["SOURCE_ID","KIND","CONNECTION_ID","NAME"], (.result[] | [.id, .kind, .connection, .name]) | @tsv' \
5| column -t -s $'\t'

Copy the ID of a log source and its corresponding connection that you want the dashboard to query. Pass them to Terraform as input variables:

1export TF_VAR_logs_source_id="68d20d409bc8769c8984585f"
2export TF_VAR_connection_id="68b6b6dd5d2cada7d1c593ac"

Configure the provider #

Create an empty directory for the example, then add this provider configuration to [main.tf](http://main.tf):

1terraform {
2  required_version = ">= 1.5.0"
3
4  required_providers {
5    clickhouse = {
6      source  = "ClickHouse/clickhouse"
7      version = "~> 3.24.0"
8    }
9  }
10}
11
12provider "clickhouse" {}
13
14variable "logs_source_id" {
15  type = string
16}
17
18variable "connection_id" {
19  type = string
20}

> The provider block is empty here because the four environment variables above supply its Cloud configuration. Add team = var.team_id to the dashboard resource only when targeting a non-default team in a self-hosted deployment.

Define the dashboard #

Add the dashboard resource to main.tf. This example creates one log chart counting events per service over time:

1variable "logs_source_id" {
2  description = "ID of the ClickStack logs source used by the dashboard"
3  type        = string
4}
5
6resource "clickhouse_clickstack_dashboard" "simple_logs" {
7  dashboard_json = jsonencode({
8    name    = "simple logs dashboard"
9
10    tiles = [
11      {
12        name = "Log count over time by service",
13        id = "9kcn995dbkdxjlw3jj28k"
14        x  = 0
15        y  = 0
16        w  = 24
17        h  = 11
18
19        config = {
20          name                        = "Logs over time"
21          sourceId                    = var.logs_source_id
22          displayType                 = "line"
23          granularity                 = "auto"
24          alignDateRangeToGranularity = true
25
26          select = [
27            {
28              aggFn                = "count"
29              aggCondition         = ""
30              aggConditionLanguage = "lucene"
31              valueExpression      = ""
32            }
33          ]
34
35          where         = ""
36          whereLanguage = "lucene"
37          groupBy       = "ServiceName"
38        }
39      }
40    ]
41
42    filters    = []
43    containers = []
44  })
45}
46
47output "dashboard_id" {
48  value = clickhouse_clickstack_dashboard.simple_logs.id
49}

The dashboard_json value maps to the ClickStack v2 dashboard API. Keeping it inside jsonencode lets you use Terraform variables without constructing JSON strings by hand.

Plan and apply the change #

Initialize the directory, format and validate the configuration, then inspect the plan:

1terraform init
2terraform fmt
3terraform validate
4terraform plan

ClickStack resources are beta. Terraform will print a beta warning during validation and planning; that warning is expected. Check that the plan contains one clickhouse_clickstack_dashboard resource to add, then apply it:

1dalemcdiarmid@Mac clickstack_terraform % terraform apply
2
3Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
4  + create
5
6Terraform will perform the following actions:
7
8  # clickhouse_clickstack_dashboard.simple_logs will be created
9  + resource "clickhouse_clickstack_dashboard" "simple_logs" {
10      + dashboard_json  = jsonencode(
11            {
12              + containers = []
13              + filters    = []
14              + name       = "simple logs dashboard"
15              + tiles      = [
16                  + {
17                      + config = {
18                          + alignDateRangeToGranularity = true
19                          + displayType                 = "line"
20                          + granularity                 = "auto"
21                          + groupBy                     = "ServiceName"
22                          + name                        = "Logs over time"
23                          + select                      = [
24                              + {
25                                  + aggCondition         = ""
26                                  + aggConditionLanguage = "lucene"
27                                  + aggFn                = "count"
28                                  + valueExpression      = ""
29                                },
30                            ]
31                          + sourceId                    = "68d20d409bc8769c8984585f"
32                          + where                       = ""
33                          + whereLanguage               = "lucene"
34                        }
35                      + h      = 11
36                      + id     = "9kcn995dbkdxjlw3jj28k"
37                      + name   = "Log count over time by service"
38                      + w      = 24
39                      + x      = 0
40                      + y      = 0
41                    },
42                ]
43            }
44        )
45      + id              = (known after apply)
46      + normalized_json = (known after apply)
47    }
48
49Plan: 1 to add, 0 to change, 0 to destroy.
50
51Do you want to perform these actions?
52  Terraform will perform the actions described above.
53  Only 'yes' will be accepted to approve.
54
55  Enter a value: yes
56
57Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
58
59Outputs:
60
61dashboard_id = "6a7daffaecf5dee21ed130c6"

After the apply completes, open ClickStack → Dashboards for the service. The new dashboard should contain both tiles, and Terraform will print its dashboard ID as an output.

> Keep one source of truth. Manage a dashboard either in Terraform or in the ClickStack UI. The dashboard resource does not report UI edits as drift. Those edits remain until dashboard_json changes, at which point Terraform replaces the dashboard definition and can overwrite them.

Change the dashboard name, tags, or tile configuration and run terraform plan again to see the update before applying it. For a temporary test deployment, remove the dashboard when you are finished:

1terraform destroy

Managing existing dashboards #

Existing dashboards can be brought under Terraform management with terraform import. Add a matching resource block first, then import the dashboard by ID:

1terraform import clickhouse_clickstack_dashboard.collectors <dashboard-id>

For a self-hosted dashboard in a non-default team, use <team-id>/<dashboard-id> as the import ID. The complete schema and import behavior are documented in the ClickStack dashboard resource.

Because you’ve been creating resources manually, we’ve also made it much easier to import existing ClickHouse resources into your Terraform configuration. Simply open the resource you want to bring into Terraform management in ClickStack, such as the dashboard shown below, select the Terraform icon, copy the generated import block into a .tf file, and run the terraform plan -generate-config-out=generated-dashboard.tf command. This workflow requires Terraform 1.5 or later.

You can also bulk-export all your existing resources and generate resource configurations.

The exported file can then be used to bring the entire ClickStack deployment under Terraform management. For example:

1cp ~/Downloads/hyperdx-import.tf . 
2terraform init
3terraform plan -generate-config-out=generated-dashboard.tf

Conclusion #

ClickStack resources can now follow the same review and deployment process as the rest of your infrastructure. Teams can reproduce observability configuration across environments, inspect changes before applying them, and bring existing resources under Terraform management through import.

Folding this work into ClickHouse/clickhouse also means one provider, one release path, and consistent authentication for Managed ClickStack. The same resources work with open-source, self-hosted ClickStack using its endpoint and a personal API access key.

ClickStack support is available in version 3.25.0 of the provider. Start with the provider documentation, try it with an existing dashboard, and share any issues or unexpected behavior through the provider’s GitHub repository.