Infrastructure Requirements Reference - ClickHouse Documentation

This document consolidates infrastructure prerequisites, node specifications, networking, storage, and IAM requirements across all supported ClickHouse Private deployment platforms.

Prerequisites by Platform

AWS (EKS)

Requirement	Details
AWS CLI	Installed and configured
`kubectl`	Installed
`helm`	v3.x installed
`skopeo`	Recommended for copying container images (retains all architectures)
EKS cluster IAM role	Created via AWS console “Create recommended role” with default permissions
Node IAM roles	Created via AWS console “Create recommended role” with default permissions
OIDC provider	Created for the EKS cluster

GCP (GKE)

Requirement	Details
GCP Project	Billing enabled
`gcloud` CLI	Installed and authenticated
`kubectl`	Installed
`helm`	v3.x installed
`skopeo`	Installed (for copying container images)
AWS credentials	Read access to ClickHouse ECR (for image copying)
Bastion host	In the VPC for accessing the private cluster
Deployer service account	With appropriate permissions (see IAM / Service Account Requirements)
(FIPS only) Red Hat UBI8 image	Pulled via GKE for certificate generation

Azure (AKS)

Requirement	Details
Azure subscription	With permissions to create AKS clusters, ACR, Storage Accounts, VNets, and Managed Identities
`az` CLI	Installed and authenticated
`kubectl`	Installed
`helm`	v3.x installed
`skopeo`	Installed (for copying container images)
AWS credentials	Read access to ClickHouse ECR (for image copying)
OIDC issuer	Enabled on the AKS cluster (`--enable-oidc-issuer`)
Workload Identity	Enabled on the AKS cluster (`--enable-workload-identity`)

Bare Metal

Requirement	Details
Kubernetes	Any conformant distribution, v1.33+. RKE2, kubeadm, and k3s are all suitable.
`kubectl`	Installed
`helm`	v3.x installed
`skopeo`	For copying container images between registries
Private container registry	OCI-compatible and reachable from the cluster, to hold the mirrored images and charts
Object storage	MinIO AIStor recommended on-prem
Object storage credentials	Static access key / secret key

Node Group Specifications

AWS (EKS)

Keeper Node Group

Property	Value
AMI (x86)	`AL2023_x86_64`
AMI (arm64)	`AL2023_ARM_64`
Disk size	20 GB
Recommended instance type	`m7g.2xlarge`
Min nodes	3 per ClickHouse cluster
Kubernetes label (x86)	`clickhouseGroup: keeper`
Kubernetes label (arm64)	`clickhouseGroup: keeper-arm64`
Taint	`clickhouse.com/do-not-schedule: true, NoSchedule`
Taint (arm64 only)	`clickhouse.com/arch: arm64, NoSchedule`
Recommended resources	CPU: 4, Memory: 16Gi

Server Node Group

Property	Value
AMI (x86)	`AL2023_x86_64`
AMI (arm64)	`AL2023_ARM_64`
Disk size	20 GB
Recommended instance type	`m7gd.16xlarge` (strongly recommended “d” series with NVMe SSD)
Min nodes	Equal to desired number of ClickHouse replicas
Kubernetes label (x86)	`clickhouseGroup: server`
Kubernetes label (arm64)	`clickhouseGroup: server-arm64`
Taint	`clickhouse.com/do-not-schedule: true, NoSchedule`
Taint (arm64 only)	`clickhouse.com/arch: arm64, NoSchedule`
Recommended resources	CPU: 64, Memory: 256Gi
Launch template	Custom template required for NVMe SSD RAID setup (see source doc)

Operator Node Group (x86)

Property	Value
AMI	`AL2023_x86_64`
Disk size	20 GB
Minimum instance size	xlarge
Instance type	Any x86-compatible type
Notes	Can be an existing node group if already x86-compatible with minimum xlarge

All AWS nodes require IMDS for authentication. For keeper and server node groups, create one node group per AZ if you wish to support cluster autoscaler across AZs. Autoscaler tags (if using cluster-autoscaler):

k8s.io/cluster-autoscaler/enabled: true
k8s.io/cluster-autoscaler/$CLUSTER_NAME: owned

GCP (GKE)

Keeper Node Pool

Property	Value
Machine type (dev)	`n2-standard-4`
Machine type (prod)	`n2-standard-8`
Disk type	`pd-ssd`
Disk size	20 GB
Node label	`clickhouseGroup=keeper`
Taint	`clickhouse.com/do-not-schedule=true:NoSchedule`
Autoscaling	min 1, max 3
Workload metadata	`GKE_METADATA` (for Workload Identity)
Recommended resources	CPU: 2, Memory: 8Gi

Server Node Pool

Property	Value
Machine type (dev)	`n2-standard-8`
Machine type (prod)	`n2-standard-64`
Disk type	`pd-standard` + ephemeral local SSD
Disk size	20 GB
Local SSD	Attached as ephemeral storage (`--ephemeral-storage-local-ssd count=N`); 375 GiB each, striped into RAID-0 when `count > 1`. Scale `count` so the array holds the cache (`bytesPerGiRAM * server_memory`) plus headroom for images/logs
Node label	`clickhouseGroup=server`
Taint	`clickhouse.com/do-not-schedule=true:NoSchedule`
Autoscaling	min 1, max 10
Workload metadata	`GKE_METADATA` (for Workload Identity)
Recommended resources	CPU: 4, Memory: 16Gi (dev); CPU: 64, Memory: 256Gi (prod)
SSD cache	`emptyDir` backed by ephemeral local SSD; enabled by `isOnEmptyDir=true` (GCP base config default)

Default Node Pool

Property	Value
Machine type	`n2-standard-4`
Disk type	`pd-standard`
Disk size	50 GB
Nodes	1

Azure (AKS)

Keeper Node Pool

Property	Value
VM size	`Standard_D8ds_v5`
OS	AzureLinux
OS disk	32 GB Premium SSD (Managed)
Min nodes	3 per ClickHouse cluster
Kubernetes label	`clickhouseGroup=keeper`
Taint	`clickhouse.com/do-not-schedule=true:NoSchedule`
Recommended resources	CPU: 4, Memory: 16Gi

Server Node Pool

Property	Value
VM size (dev)	`Standard_D16ds_v5`
VM size (prod)	`Standard_D48ds_v5`
OS	AzureLinux
OS disk type	`Ephemeral` (no managed disk cost; lands on VM cache disk)
Min nodes	Equal to desired number of ClickHouse replicas
Kubernetes label	`clickhouseGroup=server`
Taint	`clickhouse.com/do-not-schedule=true:NoSchedule`
Recommended resources	CPU: 16, Memory: 64Gi (dev); CPU: 48, Memory: 192Gi (prod)
SSD cache	`hostPath` pointing at `/mnt` (Azure mounts local SSD there automatically); `isOnEmptyDir=false`, `hostPath=/mnt`; no DaemonSet or RAID required

Temp storage by instance size (Standard_D*ds_v5, Intel Icelake):

VM Size	vCPUs	RAM	Temp Storage	Cache headroom (~75%)
`Standard_D16ds_v5`	16	64 GiB	600 GiB	~450 GiB
`Standard_D32ds_v5`	32	128 GiB	1200 GiB	~900 GiB
`Standard_D48ds_v5`	48	192 GiB	1800 GiB	~1350 GiB

System / Operator Node Pool

Property	Value
VM size	`Standard_D4s_v3` or larger
Notes	Default system pool created with the cluster; no ClickHouse labels or taints required

For keeper and server node pools, create one pool per availability zone if you wish to support cluster autoscaler across zones.

Bare Metal

Assign nodes to distinct roles and label and taint them so the operator schedules ClickHouse pods correctly and nothing else lands on them. At least three failure domains are recommended so the operator can spread Keeper across them (topology spread with max skew 1 and a minimum of 3 domains on topology.kubernetes.io/zone). Map each domain to a real fault-isolation boundary such as a rack, chassis, or power feed where possible.

Keeper Nodes

Property	Value
Count	3 (one per failure domain)
Node label	`clickhouseGroup=keeper`
Taint	`clickhouse.com/do-not-schedule=true:NoSchedule`
Zone label	a distinct `topology.kubernetes.io/zone` per node, see the Kubernetes documentation on topology constraints for more context
Storage	small persistent volume via a StorageClass (not NVMe)
Recommended resources	CPU: 4, Memory: 16Gi

Server Nodes

Property	Value
Count	3+ (spread across the failure domains)
Node label	`clickhouseGroup=server`
Taint	`clickhouse.com/do-not-schedule=true:NoSchedule`
Zone label	spread across the failure domains
Local NVMe SSD	Required for the filesystem cache; mounted at `/nvme/disk` (RAID-0 if multiple devices)
Recommended resources	Sized to the workload

Operator Nodes

At least one additional node for the operator and cluster add-ons. No ClickHouse labels or taints are required.

Pod QoS: Guaranteed (recommended)

ClickHouse server, keeper, and child compute group pods should run with the Kubernetes Guaranteed QoS class. In practice, this means that for every container in the pod, resources.requests equals resources.limits for both CPU and memory, and both fields are set. Every deployment example in this documentation follows this pattern.

Why

ClickHouse’s memory tracker relies on a stable memory ceiling. When requests != limits, the effective ceiling and the OOM-kill boundary diverge and behavior becomes hard to reason about:

If the memory tracker uses the request value, queries are killed while the node still has headroom (over-conservative).
If it uses the cgroup limit, the pod can exceed its request and be OOM-killed by the kernel with no ClickHouse-side warning.

Guaranteed QoS also gives the pod the highest priority under node pressure — see the Kubernetes Pod QoS docs for the eviction and scheduling implications.

Choosing a different QoS class

The operator does not enforce Guaranteed QoS — you can deliberately run ClickHouse under Burstable (or another class) if, for example, you are trading predictability for pack density. Before you do, make sure you understand the OOM trade-offs above and the wider Kubernetes resource model:

For the applied examples, see any of the deployment tutorials or the compute-compute separation guide.

Networking Requirements

AWS

Component	Specification
VPC CIDR	`10.20.0.0/16` (example)
IPv6	Not used
Tenancy	Default
Minimum AZs	3 (for HA)
Public subnets	3 (one per AZ, `/20` each)
Private subnets	3 (one per AZ, `/18` each)
NAT gateways	1 per AZ
VPC endpoints	S3 gateway
DNS hostnames	Enabled
DNS resolution	Enabled
CNI	IPv4 required (e.g., Amazon VPC CNI with IRSA)

Example subnet layout (us-west-2):

Type	AZ	CIDR
Public	us-west-2a	`10.20.192.0/20`
Public	us-west-2b	`10.20.208.0/20`
Public	us-west-2c	`10.20.224.0/20`
Private	us-west-2a	`10.20.0.0/18`
Private	us-west-2b	`10.20.64.0/18`
Private	us-west-2c	`10.20.128.0/18`

GCP

Component	Specification
VPC mode	Custom subnet
GKE subnet range	`10.1.0.0/24` (example)
Pod secondary range	`10.244.0.0/14` (must be on `/14` boundary; 3rd octet divisible by 4)
Service secondary range	`10.252.0.0/20` (must be on `/20` boundary)
Master IPv4 CIDR	`172.16.0.0/28`
Private nodes	Enabled (internal IPs only)
Private endpoint	Enabled (no public API; FedRAMP)
Master authorized networks	GKE subnet + bastion subnet
Cloud NAT	Required for outbound internet access from private nodes
Private Google Access	Enabled on subnet

Azure

Component	Specification
VNet address space	`10.20.0.0/16` (example)
Node subnet	`10.20.0.0/22`
Network plugin	Azure CNI with overlay mode
Pod CIDR	`192.168.0.0/16`
Service CIDR	`10.0.0.0/16`
DNS service IP	`10.0.0.10`
Availability zones	3 (for HA)

Bare Metal

Component	Specification
CNI	Any CNI providing standard IPv4 pod networking (Cilium, Calico, Canal, etc.)
Failure domains	At least 3, expressed via `topology.kubernetes.io/zone` node labels
Object storage reachability	S3 endpoint reachable from the ClickHouse pods
Client access	Via port-forward or a load balancer / ingress

Storage Requirements

AWS

Resource	Details
S3 bucket	Standard class, encryption enabled, same region as EKS cluster. Do not enable S3 Lifecycle policies — see Manage data lifecycle
S3 bucket scope	One per ClickHouse cluster, or shared with unique `keyPrefix` per cluster
EBS CSI driver	Required; install via Helm with IRSA
StorageClass	`gp3-encrypted` (provided by Helm chart) or custom
NVMe cache	”d” series instances (e.g., `m7gd.*`); mounted via EC2 launch template RAID script at `/nvme/disk`
Cache sizing	`bytesPerGiRAM * pod_memory_limit` (e.g., 11.3Gi * 250Gi = 2825Gi)
VolumeSnapshot CRDs	Required by operator

GCP

Resource	Details
GCS bucket	Standard class, uniform bucket-level access, same region as GKE cluster. Do not enable Object Lifecycle Management — see Manage data lifecycle
GCS S3-compatible endpoint	`https://storage.googleapis.com`
GCS region setting	`auto`
StorageClass	`premium-rwo` (GKE built-in) or custom via Helm chart with `pd.csi.storage.gke.io` provisioner
Topology key	`topology.gke.io/zone` (not `topology.kubernetes.io/zone`)
Local SSD cache	`emptyDir` backed by ephemeral local SSD (`--ephemeral-storage-local-ssd`); GCP base config sets `isOnEmptyDir=true`
VolumeSnapshot CRDs	Required by operator

Azure

Resource	Details
Storage Account	Standard LRS, StorageV2, public blob access disabled, same region as AKS cluster. Do not enable Blob Lifecycle Management rules — see Manage data lifecycle
Blob container scope	One per ClickHouse cluster, or shared with unique `keyPrefix` per cluster
Azure Disk CSI driver	Pre-installed in AKS; provisioner: `disk.csi.azure.com`
StorageClass	`managed-csi` (server, Standard SSD) and `managed-csi-premium` (keeper, Premium SSD); both pre-installed
SSD cache	`emptyDir` backed by temp disk; AKS bind-mounts `/mnt/aks/kubelet/` to `/var/lib/kubelet` when `kubelet_disk_type=Temporary`; `isOnEmptyDir=true`; no DaemonSet or RAID required
Cache sizing	`bytesPerGiRAM * pod_memory_limit` targeting ~75% of temp storage (e.g., 7Gi * 64Gi = 448Gi for `Standard_D16ds_v5`)
VolumeSnapshot CRDs	Required by operator

Bare Metal

Resource	Details
Object storage	MinIO AIStor recommended on-prem. Do not enable object lifecycle / expiration rules — see Manage data lifecycle
Object storage credentials	Static access key / secret key, supplied to server and keeper via `additionalEnvVars` from a Kubernetes Secret
Server filesystem cache	Local NVMe SSD mounted at `/nvme/disk` (RAID-0 if multiple devices); mounted into Server pods by the operator as a HostPath (default) or EmptyDir volume, not through a StorageClass
Keeper storage	Persistent volume via a StorageClass (any storage class with a zone-aware label); non-NVMe
Cache sizing	`cacheDiskSize` set to 70-80% of usable NVMe, or `bytesPerGiRAM * pod_memory_limit`
S3 region	Set `isOnPremiseInstance=true`; a region is not required for on-prem object stores
VolumeSnapshot CRDs	Required by operator

IAM / Service Account Requirements

AWS

ECR Pull Role

Role name: ClickHouseAirgapECRPullRole
Purpose: Pull images from ClickHouse private ECR
Permissions: ecr:GetDownloadUrlForLayer, ecr:BatchGetImage, ecr:BatchCheckLayerAvailability, ecr:DescribeImages, ecr:ListImages, ecr:GetAuthorizationToken
Action: Provide the role ARN to the ClickHouse team

ClickHouse Server/Keeper S3 Role

One role per provisioned ClickHouse cluster
Naming convention: CH-S3-$NAME-$REGION-$ORDINAL-Role
- $NAME: cluster name (e.g., default-xx-01)
- $REGION: shortened region (e.g., uw2 for us-west-2)
- $ORDINAL: reserved, set to 00
- Example: CH-S3-default-xx-01-uw2-00-Role
Permissions: s3:*, s3:ListBucket on the bucket resource
Trust policy: IRSA via sts:AssumeRoleWithWebIdentity with EKS cluster’s OIDC provider

EBS CSI Driver Role

Example name: ClickHouse_EksEbsCsiDriverRole
Managed policy: arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy
Trust policy: IRSA with EKS cluster’s OIDC provider

GCP

Deployer Service Account

Role	Purpose
`roles/compute.networkAdmin`	Create/manage VPC, subnets, NAT, routers
`roles/container.admin`	Create/manage GKE clusters and node pools
`roles/storage.admin`	Create/manage GCS buckets
`roles/iam.serviceAccountAdmin`	Create/manage service accounts
`roles/iam.serviceAccountUser`	Use service accounts for workload identity
`roles/iam.workloadIdentityUser`	Configure workload identity bindings
`roles/artifactregistry.admin`	Create/manage Artifact Registry repos
`roles/resourcemanager.projectIamAdmin`	Manage IAM policies on the project

ClickHouse Workload Identity Service Account

GCP SA: clickhouse@$PROJECT.iam.gserviceaccount.com
K8s SA (manual): clickhouse in namespace ns-$CLUSTER_NAME
K8s SA (helm-created): ch-$CLUSTER_NAME-sa in namespace ns-$CLUSTER_NAME
K8s SA annotation: iam.gke.io/gcp-service-account=<GSA_EMAIL>
IAM binding member: serviceAccount:$PROJECT.svc.id.goog[<namespace>/<ksa>]
Bucket permissions: roles/storage.objectAdmin, roles/storage.legacyBucketReader
Artifact Registry permissions: roles/artifactregistry.reader

Azure

Deployer Permissions

Role	Purpose
`Contributor` on resource group	Create/manage VNet, AKS, ACR, Storage Accounts
`User Access Administrator` on resource group	Assign roles to the managed identity
`AcrPush` on ACR	Push images during setup

ClickHouse Managed Identity

Type: User-Assigned Managed Identity (UAMI)
K8s SA (helm-created): ch-$CLUSTER_NAME-sa in namespace ns-$CLUSTER_NAME
K8s SA annotation: azure.workload.identity/client-id=<UAMI_CLIENT_ID>
Federated credential subject: system:serviceaccount:ns-$CLUSTER_NAME:ch-$CLUSTER_NAME-sa
Role assignment: Storage Blob Data Contributor on the Storage Account

Bare Metal

Object storage credentials: Static access key / secret key for MinIO AIStor, held in a Kubernetes Secret (object-storage-credentials) and injected into server and keeper pods via additionalEnvVars.
Registry pull secret: registry-credentials docker-registry secret in the clickhouse-operator-system and cluster namespaces, if your registry requires authentication.
Service account: No cloud identity binding required.

Cluster Naming Conventions

The naming convention for ClickHouse clusters is consistent across all platforms:

$DESCRIPTOR-$LETTERS-$ORDINAL

Component	Description	Example
`$DESCRIPTOR`	Descriptive name (letters only)	`default`
`$LETTERS`	Reserved; any two letters	`xx`
`$ORDINAL`	Incrementing ordinal starting at `01`	`01`

Example name: default-xx-01 The cluster name must be unique within the target Kubernetes cluster and is used in:

Kubernetes namespace: ns-$CLUSTER_NAME
Kubernetes resources: prefixed with c-$CLUSTER_NAME
IAM role names (AWS): CH-S3-$CLUSTER_NAME-$REGION-$ORDINAL-Role
S3/GCS key prefixes: ch-s3-<uuid> (unique per cluster)

Required Kubernetes Components

Component	AWS	GCP	Azure	Bare Metal
CNI (IPv4)	Amazon VPC CNI (recommended)	GKE built-in	Azure CNI overlay (built-in)	Any IPv4 CNI
CSI driver	aws-ebs-csi-driver	GKE built-in (pd.csi.storage.gke.io)	AKS built-in (disk.csi.azure.com)	StorageClass for Keeper (any storage class with a zone-aware label); Server cache uses HostPath/EmptyDir
DNS	CoreDNS	GKE built-in	AKS built-in (CoreDNS)	Any in-cluster DNS (e.g. CoreDNS)
VolumeSnapshot CRDs	Required	Required	Required	Required
Autoscaler	Cluster Autoscaler (recommended)	GKE node pool autoscaling	AKS cluster autoscaler	N/A
Cert manager	Not required (unless FIPS)	Required for FIPS	Not required (unless FIPS)	Not required (unless FIPS)

​Prerequisites by Platform

​AWS (EKS)

​GCP (GKE)

​Azure (AKS)

​Bare Metal

​Node Group Specifications

​AWS (EKS)

​Keeper Node Group

​Server Node Group

​Operator Node Group (x86)

​GCP (GKE)

​Keeper Node Pool

​Server Node Pool

​Default Node Pool

​Azure (AKS)

​Keeper Node Pool

​Server Node Pool

​System / Operator Node Pool

​Bare Metal

​Keeper Nodes

​Server Nodes

​Operator Nodes

​Pod QoS: Guaranteed (recommended)

​Why

​Choosing a different QoS class

​Networking Requirements

​AWS

​GCP

​Azure

​Bare Metal

​Storage Requirements

​AWS

​GCP

​Azure

​Bare Metal

​IAM / Service Account Requirements

​AWS

​ECR Pull Role

​ClickHouse Server/Keeper S3 Role

​EBS CSI Driver Role

​GCP

​Deployer Service Account

​ClickHouse Workload Identity Service Account

​Azure

​Deployer Permissions

​ClickHouse Managed Identity

​Bare Metal

​Cluster Naming Conventions

​Required Kubernetes Components

Prerequisites by Platform

AWS (EKS)

GCP (GKE)

Azure (AKS)

Bare Metal

Node Group Specifications

AWS (EKS)

Keeper Node Group

Server Node Group

Operator Node Group (x86)

GCP (GKE)

Keeper Node Pool

Server Node Pool

Default Node Pool

Azure (AKS)

Keeper Node Pool

Server Node Pool

System / Operator Node Pool

Bare Metal

Keeper Nodes

Server Nodes

Operator Nodes

Pod QoS: Guaranteed (recommended)

Why

Choosing a different QoS class

Networking Requirements

AWS

GCP

Azure

Bare Metal

Storage Requirements

AWS

GCP

Azure

Bare Metal

IAM / Service Account Requirements

AWS

ECR Pull Role

ClickHouse Server/Keeper S3 Role

EBS CSI Driver Role

GCP

Deployer Service Account

ClickHouse Workload Identity Service Account

Azure

Deployer Permissions

ClickHouse Managed Identity

Bare Metal

Cluster Naming Conventions

Required Kubernetes Components