Operations

This page covers day-2 operation of the connector on both install targets. For install and enrollment, see onboarding. Operating the services themselves (create, scale, delete, and platform updates) is covered in managed services and platform updates.

Upgrades

Kubernetes

Day-2 Kubernetes operations use the helm CLI on your workstation. Only init carries a built-in Helm client, so install helm before your first upgrade.

Upgrade the release from the public chart repository with the values overlay that init staged:

CONNECTOR_NAMESPACE='clicklink'   # the connector namespace you chose at init
helm upgrade --install clicklink-connector clicklink-connector \
  --repo https://releases.clicklink.clickhouse.com/charts \
  --version <version> \
  -n "${CONNECTOR_NAMESPACE}" \
  -f clicklink-values.yaml

Chart versions are the release tags without the leading v (chart X.Y.Z corresponds to tag vX.Y.Z). The published chart already points at the public container image, so plain installs and upgrades need no image values. To inspect chart defaults, run helm show values clicklink-connector --repo https://releases.clicklink.clickhouse.com/charts. The executor runs as a single replica with the Recreate strategy, so an upgrade disconnects it briefly. See when the connector is offline for what happens to commands in that window. An install from a direct chart reference (oci://, a URL, or a local archive or directory) has no repository to resolve against. Rerun helm upgrade clicklink-connector <same-chart-reference> at the new version instead. Re-running init with a newer CLI also converges, but needs an entry point: --handoff if you kept the bundle, or a fresh enrollment token with --force after the documented cleanup. The helm upgrade above is the routine path (see re-runs and recovery).

Linux VM

Re-run the installer on the host. It downloads and verifies the newer release as it did during onboarding. It backs up the previous binary, installs the current systemd units, and preserves your live redaction patterns and environment file. Then restart the daemons:

curl -fsSL https://releases.clicklink.clickhouse.com/install.sh | sudo bash -s -- --host
sudo systemctl restart clicklink-scraper clicklink-troubleshooter clicklink-executor

To move to a specific release instead of the latest, append --version vX.Y.Z to the installer command.

Health

Each daemon serves a /livez endpoint on its health port. The health signal is the JSON status field in the response body, not the HTTP status code. Prometheus metrics are served at /metrics on the same port; there is no separate metrics port. Default ports on both targets:

Component	Health port
Scraper	8082
Troubleshooter	8084
Executor	8086

The executor also serves its local command API on 127.0.0.1:9999. It carries no authentication of its own and is never exposed outside the host or pod; the clicklink clctl service commands reach it locally. When support sessions are enabled, the gateway also listens on port 8443: self-signed TLS on a VM, pod-local HTTP behind kubectl port-forward or a TLS-terminating Ingress on Kubernetes. On a VM, run the full check suite at any time:

sudo clicklink clctl preflight

It checks config, files, port conflicts, network reachability, ClickHouse connectivity, systemd unit state, per-component access, disk, and redaction patterns. It exits 2 if any check fails.

Certificates

The connector renews its own client certificate. The scraper, the troubleshooter, and the executor each check the leaf every 12 hours and renew it once 10 days remain. The renewal returns a 30-day leaf over the existing mTLS and HMAC-authenticated channel, so no operator action is needed. On Kubernetes the daemon writes the renewed leaf back to the clicklink-mtls Secret; on a VM it writes it under /etc/clicklink/tls/. To inspect the current expiry:

# Linux VM
sudo openssl x509 -in /etc/clicklink/tls/client.crt -noout -enddate

# Kubernetes
CONNECTOR_NAMESPACE='clicklink'   # the connector namespace you chose at init
kubectl get secret clicklink-mtls -n "${CONNECTOR_NAMESPACE}" -o jsonpath='{.data.tls\.crt}' \
  | base64 -d | openssl x509 -noout -enddate

Credential rotation

API (HMAC) credentials

Request a fresh enrollment token from your account team, then re-run your original init command with --enroll and --force. Keep every target-specific flag from the first install (--target-namespace, --values, and any --chart, --chart-repo, or --chart-version mirror flags), because --force restages the kept configuration. Managed mode carries over from the configuration being replaced, so --managed is optional here. On a default install:

# Kubernetes, from your workstation
clicklink clctl init --enroll https://<subdomain>.<connector-domain> --target helm --force

# Linux VM, on the host
sudo clicklink clctl init --enroll https://<subdomain>.<connector-domain> --force

On a VM, init only enables and starts units that are not yet running, so restart the daemons after the rotation:

sudo systemctl restart clicklink-scraper clicklink-troubleshooter clicklink-executor

On Kubernetes, init pre-creates the rotated clicklink-hmac and clicklink-mtls Secrets; the chart does not render them. Its checksum annotation does not cover them, so running pods keep the credentials they started with. Restart every component after the rotation:

kubectl rollout restart deployment,statefulset -n "${CONNECTOR_NAMESPACE}" \
  -l app.kubernetes.io/part-of=clicklink-connector

For unattended rotation where SQL provisioning needs a password, stdin carries both secrets in order: the token on the first line, the password on the second. The token read consumes exactly one line.

printf '%s\n' "$ENROLLMENT_TOKEN" "$CH_ADMIN_PASSWORD" | \
  clicklink clctl init --enroll https://<subdomain>.<connector-domain> --force --ch-admin-password-stdin

ClickHouse users

Re-provision the connector’s read-only users for each instance you registered yourself. On Kubernetes, run the commands from your workstation. --apply-ch-grants re-applies the regenerated grants in-pod so the new credentials reach ClickHouse; when the admin user has a password, add --ch-admin-password-stdin and pipe it in:

CONNECTOR_NAMESPACE='clicklink'   # the connector namespace you chose at init
clicklink clctl scraper access provision --target helm \
  --target-namespace "${CONNECTOR_NAMESPACE}" \
  --instance <instance-name> --instance-namespace <clickhouse-namespace> \
  --server <kubernetes-api-server-url> \
  --apply-ch-grants --ch-pod <clickhouse-pod-or-label-selector> --ch-pod-namespace <clickhouse-namespace> \
  --force
clicklink clctl troubleshoot access provision --target helm \
  --target-namespace "${CONNECTOR_NAMESPACE}" \
  --instance <instance-name> --instance-namespace <clickhouse-namespace> \
  --server <kubernetes-api-server-url> \
  --apply-ch-grants --ch-pod <clickhouse-pod-or-label-selector> --ch-pod-namespace <clickhouse-namespace> \
  --force

On a VM, run them on the host with the admin password piped in; there the password is mandatory unless you pass --ch-user-via cr:

printf '%s\n' "$CH_ADMIN_PASSWORD" | sudo clicklink clctl scraper access provision --provider local \
  --instance <instance-name> --server <kubernetes-api-server-url> --ch-admin-password-stdin --force
printf '%s\n' "$CH_ADMIN_PASSWORD" | sudo clicklink clctl troubleshoot access provision --provider local \
  --instance <instance-name> --server <kubernetes-api-server-url> --ch-admin-password-stdin --force

For operator-managed instances, add --ch-user-via cr and the pod-selection flags to either form; see the CLI reference.

Client certificate

Renewal is automatic (see certificates). To supersede an unexpired certificate immediately, re-run init with --force. Then restart every component so the daemons present the new leaf: on Kubernetes as described under API credentials, on a VM the three units.

Re-runs and recovery

init re-runs converge, so re-running the same command is always the first move. Without --force, an existing /etc/clicklink/config.yaml (VM) or clicklink-values.yaml overlay (Kubernetes) is kept, and an existing client key is reused; init overwrites credentials and the CA chain atomically. The recovery commands the CLI prints after a partial failure are safe to repeat. --force overwrites the kept config or overlay, regenerates the client key, and supersedes an unexpired client certificate. It never mints a new cluster UUID, so the connector’s identity is preserved. The managed-mode posture of the replaced configuration is kept on unattended re-runs and offered as the default on a terminal, unless you pass --managed or --no-managed. If certificate signing failed after staging, or the signing endpoint returned 409 because an unexpired certificate already exists, you do not need a new token or a second issuance. Complete the install with the signed materials already on disk:

sudo clicklink clctl init --signed-cert client.crt --chain ca-chain.crt

That is the VM form (root rewrites /etc/clicklink and manages the services). On Kubernetes the CLI prints the full form, including --target helm, --target-namespace, and --values; use the printed command as is. Connector offline. While the connector is down or unreachable, the services it manages keep running. The ClickHouse operator in your cluster keeps them up, and the executor is not in the data path. Lifecycle changes need the executor connected, so none are applied until it reconnects. On restart the executor recovers in-flight commands from its command database. A command ClickHouse Cloud delivers again is answered from the recorded result; only a failed command runs again. For lifecycle commands while no executor is connected, see when the connector is offline.

Uninstall

Delete every managed service before uninstalling, following delete a service, and wait for each teardown to finish. Removing the connector first leaves services with no component able to apply their lifecycle.

Linux VM

uninstall.sh ships in the release tarball. If no extracted tarball remains on the host, fetch and extract one (see manual download and verification). Run uninstall.sh from the extracted directory:

sudo ./uninstall.sh

This stops and disables the clicklink-scraper, clicklink-troubleshooter, and clicklink-executor units and removes the systemd units and the binary. It preserves the following, so a later reinstall picks up the existing configuration:

/etc/clicklink, including the executor’s access bundles under /etc/clicklink/access/executor
/var/lib/clicklink, including the executor’s command database executor.db and the service registry
/var/log/clicklink
the clicklink user

To remove those as well:

sudo ./uninstall.sh --purge

The executor’s cluster-wide grant lives in your cluster, not on the host. Remove it as cluster admin once no managed services remain:

kubectl delete validatingadmissionpolicybinding pcm-executor-prefix-guard
kubectl delete validatingadmissionpolicy pcm-executor-prefix-guard
kubectl delete clusterrolebinding pcm-executor-cluster
kubectl delete clusterrole pcm-executor-cluster
kubectl delete serviceaccount pcm-executor -n clicklink-system

The grant also created the clicklink-system namespace; delete it when nothing else uses it.

Kubernetes

CONNECTOR_NAMESPACE='clicklink'   # the connector namespace you chose at init
helm uninstall clicklink-connector -n "${CONNECTOR_NAMESPACE}"

The executor’s PersistentVolumeClaim is annotated helm.sh/resource-policy: keep, so the uninstall leaves it behind with the command database and service registry on it. The Secrets init created, the platform bundle Secret that platform approve wrote, and the service registry ConfigMap the executor maintains are not chart-owned either and survive the uninstall. Delete them explicitly. Include the access Secret and ServiceAccount pair of every instance you registered yourself, and of every managed service you provisioned the troubleshooter for (named after the service). --ignore-not-found covers objects that were never created, such as the platform bundle Secret before a first approval:

kubectl delete pvc -n "${CONNECTOR_NAMESPACE}" -l app.kubernetes.io/instance=clicklink-connector --ignore-not-found
kubectl delete secret clicklink-hmac clicklink-mtls clicklink-platform-bundle -n "${CONNECTOR_NAMESPACE}" --ignore-not-found
kubectl delete configmap clicklink-instance-registry -n "${CONNECTOR_NAMESPACE}" --ignore-not-found
kubectl delete secret -n "${CONNECTOR_NAMESPACE}" --ignore-not-found \
  clicklink-connector-scraper-access-<instance> \
  clicklink-connector-troubleshooter-access-<instance>
kubectl delete serviceaccount -n "${CONNECTOR_NAMESPACE}" --ignore-not-found \
  pcm-scraper-<instance> pcm-troubleshooter-<instance>
# Repeat for every ClickHouse namespace that holds an instance you registered yourself;
# a managed service's namespace went away with the service.
for ns in <clickhouse-namespace-1> <clickhouse-namespace-2>; do
  kubectl delete serviceaccount,role,rolebinding -n "${ns}" --ignore-not-found \
    pcm-scraper pcm-troubleshooter
done

init applied the executor’s cluster-wide grant, not the chart. Remove it as cluster admin once no managed services remain:

kubectl delete validatingadmissionpolicybinding pcm-executor-prefix-guard
kubectl delete validatingadmissionpolicy pcm-executor-prefix-guard
kubectl delete clusterrolebinding pcm-executor-cluster
kubectl delete clusterrole pcm-executor-cluster
kubectl delete serviceaccount pcm-executor -n "${CONNECTOR_NAMESPACE}"

The platform layer

Uninstalling the connector does not remove the platform layer. The platform namespaces and workloads, the pcm-platform RBAC, and the admission objects your approvals installed stay in the cluster, and so does every managed service. Delete the services first; see delete a service. On a test cluster with no services, clicklink clctl platform reset uninstalls the platform releases. The CustomResourceDefinitions, RBAC, and StorageClass it leaves need deleting by hand; see resetting a test cluster. Outside a test cluster, agree a removal plan with your account team before touching the platform layer.

ClickHouse users

Uninstalling on either target leaves the provisioned read-only users in place on instances you registered yourself. Drop them as an admin on each instance (append your --ch-user-suffix to the names if you set one):

DROP USER IF EXISTS pcm_scraper, pcm_troubleshooter;

Get started

Services and infrastructure

Bring Your Own Cloud

Data and integrations

Connect and query

Access management

Security and compliance

Monitoring and backups

AI tools and agents

Cloud API

Billing

Support

ClickHouse Private

Upgrades

Kubernetes

Linux VM

Health

Certificates

Credential rotation

API (HMAC) credentials

ClickHouse users

Client certificate

Re-runs and recovery

Uninstall

Linux VM

Kubernetes

The platform layer

ClickHouse users

​Upgrades

​Kubernetes

​Linux VM

​Health

​Certificates

​Credential rotation

​API (HMAC) credentials

​ClickHouse users

​Client certificate

​Re-runs and recovery

​Uninstall

​Linux VM

​Kubernetes

​The platform layer

​ClickHouse users

Upgrades

Kubernetes

Linux VM

Health

Certificates

Credential rotation

API (HMAC) credentials

ClickHouse users

Client certificate

Re-runs and recovery

Uninstall

Linux VM

Kubernetes

The platform layer

ClickHouse users