Prerequisites
Before you begin, ensure you have the following tools installed:- kubectl — compatible with your target Kubernetes version
- Helm v3.x
- skopeo — for copying container images between registries
- A Kubernetes cluster that meets the requirements in Step 1
- A private, OCI-compatible container registry reachable from the cluster, to hold the mirrored ClickHouse images and Helm charts
- Access to the ClickHouse Private ECR repository (
<<SOURCE_ECR_ACCOUNT_ID>>.dkr.ecr.us-east-1.amazonaws.com). The source ECR account ID and pull credentials are provided by ClickHouse during onboarding. - The version tags for your deployment (provided by ClickHouse):
<<SERVER_TAG>>— ClickHouse server image tag<<KEEPER_TAG>>— ClickHouse keeper image tag<<OPERATOR_TAG>>— Operator image and Helm chart tag<<CR_HELM_TAG>>— ClickHouse Cluster Helm Chart tag
Step 1: Kubernetes Cluster Requirements
ClickHouse Private runs on any conformant Kubernetes cluster that meets the requirements below. How you provision that cluster (bare-metal installer, kubeadm, RKE2, k3s, an existing on-premises platform, etc.) is up to you.Cluster
Failure domains (availability zones)
The operator spreads ClickHouse Keeper across at least three failure domains using thetopology.kubernetes.io/zone node label (Keeper’s pod topology spread requires a minimum of three domains with a max skew of one). Server replicas are likewise balanced across the zones you configure on the operator.
On bare metal you assign these zones yourself by labeling nodes. Map each zone to a real fault-isolation boundary where possible — a separate rack, chassis, power feed, or room — so that losing one domain never takes down a Keeper quorum.
- Use three distinct
topology.kubernetes.io/zonevalues (for examplezone-a,zone-b,zone-c). - Place at least one Keeper node in each zone.
- Distribute Server nodes across the same three zones.
Keeper runs as a quorumClickHouse Keeper requires a quorum to operate, so deploy at least one Keeper node in each of the three zones. Spreading Keeper across three failure domains keeps the quorum available even if a single zone is lost.
operator.availabilityZones.
Node roles
Dedicate nodes to ClickHouse Server and Keeper, and keep at least one additional node available for the operator and cluster add-ons. Label and taint the ClickHouse nodes so the operator schedules pods correctly and nothing else lands on them.
Apply the labels and taints with
kubectl label / kubectl taint, for example:
zone-b and zone-c.
SizingKeeper nodes are light (a few cores and ~16 GiB RAM is typical). Server nodes should be sized for your workload and must have local NVMe SSD for the ClickHouse disk cache — see Step 3. Recommended starting points are in reference/infrastructure-requirements.md.
Step 2: Mirror Container Images
Copy the ClickHouse images and Helm charts from the ClickHouse ECR into your private registry using skopeo. The--all flag preserves all architectures (amd64, arm64).
The commands below refer to your registry as $REGISTRY_HOST (for example registry.internal:5000 or a project path on a hosted registry). Adjust the login step to match how your registry authenticates.
$REGISTRY_HOST, so the registry must be reachable from every ClickHouse node. If your registry requires authentication for pulls, create the operator namespace and its image pull secret now; the same secret is referenced by the operator install in Step 6, and the equivalent is created in the cluster namespace in Step 7.
Step 3: Provision Local Storage
Two kinds of local storage are needed, and they are provisioned differently:- Server filesystem cache — backed by local NVMe SSD on the Server nodes. This cache sits in front of object storage and is critical to query performance; see ClickHouse Filesystem Cache for Shared Merge Tree. Server pods are stateless by default (data lives in object storage), so the NVMe cache is the only local disk a Server node needs.
- Keeper data — a small persistent volume for Keeper’s Raft logs and snapshots, served by a StorageClass. Keeper does not use NVMe.
Durability of local storageClickHouse’s source of truth lives in the S3-compatible object store, so the local disks provisioned here do not need to be highly durable. The filesystem cache repopulates from object storage after a loss, and a Keeper node restores its state from the rest of the quorum.
Prepare the NVMe filesystem cache (Server nodes only)
On each Server node, mount the NVMe SSD at a consistent path — this guide uses/nvme/disk. If a node has more than one NVMe device, combine them into a single RAID-0 array for the best throughput. Format as ext4 and mount with noatime,nobarrier, then make the mount persistent in /etc/fstab.
For example, with two NVMe devices:
- HostPath — maps the host’s
/nvme/diskdirectly into the pod. This is the default on AWS and the simplest fit for bare metal, since you have already mounted the NVMe at a host path. - EmptyDir — backed by the node’s local ephemeral storage; requires the kubelet’s ephemeral storage to live on the NVMe.
/nvme/disk; you configure it on the cluster in Step 7 via server.ssdCacheConfiguration. For a production setup we strongly recommend using **EmptyDir•• mode,
as Local Ephemeral Storage reduces the attack surface and long term is operationally simpler.
For the trade-offs between the two modes, see EmptyDir vs HostPath.
Provide a StorageClass for Keeper
Keeper stores its data on a persistent volume provisioned by a StorageClass. Pick any StorageClass whose PVs include a zone label (topology.kubernetes.io/zone, or a label matching topology.*zone / topology.*node) in their node affinity. The ClickHouse operator uses that zone information from Keeper PVs to enforce Keeper’s cross-zone topology spread; a StorageClass whose PVs only carry kubernetes.io/hostname affinity (as stock Rancher local-path-provisioner does) is not enough on its own.
Zone-aware options on bare metal include:
- A cloud CSI driver, if the cluster is running in a cloud (for example the AWS EBS CSI driver with a
gp3StorageClass) - A zone-aware storage layer such as OpenEBS, Rook, or Longhorn
Step 4: Deploy Object Storage
ClickHouse Private stores all of its data in S3-compatible object storage. On bare metal you provide this yourself. The recommended on-premises option is MinIO AIStor, MinIO’s commercial object store; any other S3-compatible object store (on-prem or external) works as well, as long as it is reachable from the cluster.Deploy MinIO AIStor
Install AIStor following MinIO’s documentation. A few points specific to this deployment:- Provision AIStor for performance. ClickHouse performance depends directly on the object store: if AIStor cannot sustain the load, neither can ClickHouse. Give it high network throughput and performant disks.
- Expose an S3 API endpoint reachable from the ClickHouse pods (in-cluster service DNS or an ingress/load balancer). If you terminate TLS on the endpoint, the CA must be trusted by the ClickHouse pods.
- Ensure it is configured for strict read-after-write consistency.
- Avoid object versioning as this can leave data on MinIO AIStor even if it is deleted in ClickHouse.
- Size it for your dataset plus growth; ClickHouse retains all primary data here.
Create a bucket and credentials
ClickHouse authenticates to the object store with a static access key / secret key pair. Provision the following and record the values for Step 7:Step 5: Install Kubernetes Prerequisites
Install VolumeSnapshot CRDs
These CRDs are required by the ClickHouse operator. If your cluster does not already provide them, install them from the external-snapshotter project:Step 6: Install the Operator
Log into your registry from Helm if it requires authentication, then install the operator. Set the availability zones to thetopology.kubernetes.io/zone labels you assigned in Step 1.
imagePullSecrets flag references the pull secret created in Step 2. Omit it if your registry allows unauthenticated pulls.
Step 7: Deploy a ClickHouse Cluster
Naming Your Cluster
Each ClickHouse cluster needs a unique name within the Kubernetes cluster. Use the convention$DESCRIPTOR-$LETTERS-$ORDINAL:
$DESCRIPTOR— descriptive name using letters only$LETTERS— reserved, usexxfor simplicity$ORDINAL— incrementing ordinal starting with01- Example:
default-xx-01
Create the Namespace and Secrets
Create the cluster namespace and, inside it, the object storage credentials from Step 4 and the registry pull secret from Step 2:Generate Password Hash and Deploy
Guaranteed QoS (recommended)ClickHouse workloads should run with matching
requests and limits for
both CPU and memory. The single SERVER_CPU/SERVER_MEMORY/KEEPER_CPU/
KEEPER_MEMORY values below are applied to both resources.requests
and resources.limits in the helm invocation that follows, which places
the pods in the Guaranteed
QoS class. If you need to run with a different QoS class, review
Pod QoS: Guaranteed (recommended)
first for the trade-offs.Step 8: Verify Installation
Port-forward the ClickHouse Service
9000 to your local machine.
Connect and Run a Query
Confirm data reaches object storage
SELECT 1 only proves the server is running. Because the object store is something you provide on bare metal, verify that ClickHouse can actually write to and read from it: create a table, insert data, then drop the local filesystem cache so the next read must come from object storage.
keyPrefix with your object store’s client or console to see the parts ClickHouse wrote.
Next Steps
- FIPS / government compliance: See tutorials/deploy-government.md to apply FIPS 140-3 certificates and TLS configuration on top of this infrastructure.
- Compute-Compute separation: See how-to/configure-compute-compute-separation.md to set up multiple compute groups with separate endpoints sharing a single dataset.
- Management API: See tutorials/install-api.md to install the optional Private API for backups and scaling operations.
- Monitoring and alerting: See how-to/configure-alerting.md to set up alerting for your deployment.
- Troubleshooting: See troubleshooting.md for common issues and solutions.