Deploy ClickHouse with FIPS Compliance (Government)

This tutorial covers deploying ClickHouse Private with FIPS 140-3 compliance on AWS. It builds on the standard AWS deployment, with stricter node architecture and AMI requirements, then adds the FIPS-specific Kubernetes configuration.

Deploying on GKE?The FIPS-specific Kubernetes steps below, including certificate generation, certificate secrets, and the openSSL Helm flags, are cloud-agnostic. For GCP, provision x86_64 infrastructure with Deploy on GCP GKE, then apply those steps here, substituting the GCS storage settings and Workload Identity from the GCP guide.

Prerequisites

Before starting this tutorial, you must have completed the following from tutorials/deploy-aws.md:

Step 1: IAM role for ECR access
Step 2: Container images copied to your ECR using the FIPS version tags below
Step 3: VPC created
Step 4: EKS cluster created
Step 5: FIPS node groups created using the x86_64 requirements below instead of the standard ARM64 recommendations
Step 6: S3 bucket and IAM roles created

FIPS node group requirements

Use x86_64 nodes for ClickHouse Server, ClickHouse Keeper, and the ClickHouse Operator. The cryptographic provider validation for the current Server and Keeper FIPS images does not include Linux on ARM64. A FIPS-enabled ARM node image does not extend that validation to the cryptographic module inside the ClickHouse containers. AWS supports two x86_64 node image paths for this deployment:

Use BOTTLEROCKET_x86_64_FIPS, which is available directly as an EKS managed node group AMI type.
Build a custom EKS-optimized Amazon Linux 2023 x86_64 AMI with FIPS mode enabled, then select it through an EC2 launch template. AWS publishes AL2023_x86_64_STANDARD, but does not expose a separate AL2023_x86_64_FIPS managed node group AMI type. See the EKS AMI build scripts.

The Bottlerocket option avoids maintaining a custom AMI. Use the AL2023 option when your organization standardizes on AL2023 and already has a custom AMI build and patching process.

Node group	Architecture	FIPS node AMI options	Instance guidance	Kubernetes label
Server	x86_64	`BOTTLEROCKET_x86_64_FIPS` or custom FIPS-enabled AL2023 x86_64	x86 “d” series with local NVMe and a 1:4 vCPU-to-memory ratio; `m5d.16xlarge` is a compatible GovCloud example	`clickhouseGroup: server`
Keeper	x86_64	`BOTTLEROCKET_x86_64_FIPS` or custom FIPS-enabled AL2023 x86_64	`m8i.2xlarge` when available; `m7i.2xlarge`, `m6i.2xlarge`, or `m5.2xlarge` are compatible alternatives	`clickhouseGroup: keeper`
Operator	x86_64	`BOTTLEROCKET_x86_64_FIPS` or custom FIPS-enabled AL2023 x86_64	Minimum xlarge, using an x86-compatible instance type	No dedicated label required

Configure instance-store NVMe using the matching AL2023 or Bottlerocket tab in Step 5 of the standard AWS deployment. In the Helm command below, set SSD_CACHE_IS_ON_EMPTY_DIR=true for Bottlerocket, which binds instance-store NVMe to /var/lib/kubelet. Set it to false for AL2023, which mounts instance-store NVMe at /nvme/disk.

FIPS Version Tags

Use the following FIPS-specific image tags when copying artifacts:

Artifact	Tag
clickhouse-server	`26.2.1.525-fips`
clickhouse-keeper	`26.2.1.258-fips`
clickhouse-operator	`main-1.20441.1-fips`
helm/clickhouse-operator-helm	`1.20441.1`
helm/onprem-clickhouse-cluster	`1.8.7`

S3 Bucket Naming Restriction

The S3 bucket name cannot contain any periods (.) when using FIPS. The S3 FIPS endpoints require virtual-host-style addressing, which does not work when the bucket name contains periods.

Step 1: Create FIPS Certificates

FIPS compliance requires TLS certificates generated using FIPS-approved algorithms in a FIPS-enabled environment. You need three sets of certificates:

Certificate	Private Key	Public Key	SAN Required
Server	`server.key`	`server.crt`	`*.c-${CLUSTER_NAME}-server-headless.ns-${CLUSTER_NAME}.svc.cluster.local`, `c-${CLUSTER_NAME}-server-any.ns-${CLUSTER_NAME}.svc.cluster.local`
Client	`client.key`	`client.crt`	(none)
Keeper	`keeper.key`	`keeper.crt`	`*.c-${CLUSTER_NAME}-keeper-headless.ns-${CLUSTER_NAME}.svc.cluster.local`

You also need a CA certificate (ca.crt and ca.key), either self-signed or from your organization’s certificate authority. Requirements:

Minimum RSA 3072-bit key size
SHA-256 or higher signature algorithm
Generated in a FIPS-enabled environment
A unique set of certificates (excluding CA) must be created for each ClickHouse cluster

For the full certificate generation procedure using a Red Hat UBI8 container with FIPS crypto policy, see how-to/configure-fips-certificates.md.

Step 2: Install Kubernetes Prerequisites

Install VolumeSnapshot CRDs

kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml

Install StorageClass (Optional)

Only if you do not have a custom or existing StorageClass:

ECR_HOST=0000000000.dkr-ecr-fips.us-west-2.on.aws
CHART_VERSION=1.8.7

helm install clickhouse-prerequisites \
    oci://$ECR_HOST/helm/onprem-clickhouse-cluster \
    --version=$CHART_VERSION \
    -n default \
    --set-json="storageClass.create=true" \
    --set-json="createCluster=false" \
    --set-json="serviceAccount.create=false"

Step 3: Install Operator

Note the FIPS ECR endpoint format: 0000000000.dkr-ecr-fips.us-west-2.on.aws (not the standard .amazonaws.com endpoint).

# update ECR_HOST as needed
ECR_HOST=0000000000.dkr-ecr-fips.us-west-2.on.aws

# should use the version of the operator's helm chart, not of the operator itself (eg 1.20441.1, not main-1.20441.1)
OPERATOR_VERSION=1.20441.1

# image tag: the FIPS operator build (note the main- prefix and the -fips suffix).
# Always set this explicitly: the FIPS variant is selected by the tag, so an
# unset tag will not give you a FIPS build.
OPERATOR_IMAGE_TAG=main-1.20441.1-fips

# set AZs as determined by VPC subnets
AZ_LIST='["us-west-2a","us-west-2b","us-west-2c"]'

helm install clickhouse-operator \
   oci://$ECR_HOST/helm/clickhouse-operator-helm \
   --version=$OPERATOR_VERSION \
   --create-namespace \
   -n clickhouse-operator-system \
   --set-json="image.repository=\"$ECR_HOST/clickhouse-operator\"" \
   --set-json="image.tag=\"$OPERATOR_IMAGE_TAG\"" \
   --set-json='cilium.enabled=false' \
   --set-json='idleScalerEnabled=false' \
   --set-json='webhooks.enabled=false' \
   --set-json='operator.debug=true' \
   --set-json='operator.metricsScraper.enabled=false' \
   --set-json="operator.availabilityZones=$AZ_LIST" \
   --set-json='operator.featureFlags.backupOnASeparatePod=true' \
   --set-json='operator.featureFlags.serverCMEKEnabled=true'

Step 4: Create Certificate Secrets

Secrets must be installed in the same Kubernetes namespace where the ClickHouse cluster will run. If your cluster name is default-xx-01, the namespace will be ns-default-xx-01. Ensure the namespace exists, then create the secrets. Do not change the secret names or key names.

CERT_DIR=.
CLUSTER_NAME=default-xx-01

# server
kubectl create secret generic -n ns-${CLUSTER_NAME} ${CLUSTER_NAME}-server-cert-secret \
  --from-file=ca.crt="$CERT_DIR/ca/ca.crt" \
  --from-file=server.crt="$CERT_DIR/server/server.crt" \
  --from-file=server.key="$CERT_DIR/server/server.key" \
  --from-file=client.crt="$CERT_DIR/client/client.crt" \
  --from-file=client.key="$CERT_DIR/client/client.key"

# keeper
kubectl create secret generic -n ns-${CLUSTER_NAME} ${CLUSTER_NAME}-keeper-cert-secret \
  --from-file=ca.crt="$CERT_DIR/ca/ca.crt" \
  --from-file=keeper.crt="$CERT_DIR/keeper/keeper.crt" \
  --from-file=keeper.key="$CERT_DIR/keeper/keeper.key"

Step 5: Deploy ClickHouse Cluster

Use the correct FIPS version of the endpoints for ECR and S3. FIPS endpoints can be found at https://aws.amazon.com/compliance/fips/.

Guaranteed QoS (recommended)ClickHouse workloads should run with matching requests and limits for both CPU and memory. The single SERVER_CPU/SERVER_MEMORY/KEEPER_CPU/ KEEPER_MEMORY values below are applied to both resources.requests and resources.limits in the helm invocation that follows, which places the pods in the Guaranteed QoS class. If you need to run with a different QoS class, review Pod QoS: Guaranteed (recommended) first for the trade-offs.

FIPS deployments set the image tags explicitlyThe standard tutorials leave server.image.tag / keeper.image.tag unset and take the versions pinned in the cluster Helm chart. FIPS is the exception: the FIPS variant is selected by the image tag rather than by a separate chart, so the -fips tags below must be set explicitly. Keep them aligned with the versions in the Component Versions reference.

# this will be the `default` user's password
PASSWORD='My super secret p@$$w0rd'
HASHED_PASSWORD=`echo -n "$PASSWORD" | shasum -a 256 | awk '{printf $1}' | base64`

# update values below as needed,
# the ecr host should be the one in your account
# the s3 bucket should be the one created earlier
CLUSTER_NAME=default-xx-01
ECR_HOST=0000000000.dkr-ecr-fips.us-west-2.on.aws
SA_IAM_ROLE_ARN=arn:aws:iam::0000000000:role/CH-S3-${CLUSTER_NAME}-uw2-00-Role

S3_BUCKET_REGION=us-gov-west-1
S3_BUCKET_URL=https://my-clickhouse-data.us-gov-west-1.amazonaws.com
S3_BUCKET_NAME=my-clickhouse-data

# s3 key prefix can use any UUID value, but must be unique for all clusters storing data in the bucket specified above
S3_KEY_PREFIX=ch-s3-$(uuidgen | tr '[:upper:]' '[:lower:]')

# these must match the x86_64 clickhouseGroup labels added to the node groups
SERVER_NG_GROUP=server
KEEPER_NG_GROUP=keeper

# these values should change depending on selected instance sizes
# be sure to take daemon set requirements into account when setting CPU and MEMORY values
SERVER_CPU=64
SERVER_MEMORY=256Gi
# Keeper: high-load profile; for moderate load 2-4 CPU / 8Gi also works (see the infrastructure reference)
KEEPER_CPU=4
KEEPER_MEMORY=16Gi

# Update the `verificationMode` field to control how TLS certificates are verified.
# Defaults to `none`. See Context.h in the ClickHouse source code for more details.
SERVER_VERIFICATION_MODE=none
KEEPER_VERIFICATION_MODE=none

# Currently, usage of `strict` is not recommended and you might encounter issues with the Operator.
# If you require mTLS cert auth, use `relaxed` (Teleport integration requires it)
# SERVER_VERIFICATION_MODE=relaxed
# KEEPER_VERIFICATION_MODE=relaxed

CHART_VERSION=1.8.7

# As a general rule, set cache size to 70-80% of the allocatable instance-store SSD capacity,
# accounting for DaemonSets and cloud-provider reserved disk space.
# The example below sizes the cache for the four 600 GB NVMe devices on m5d.16xlarge.
CACHE_DISK_SIZE_GI=1700Gi

# Select according to the Server node AMI:
# - Bottlerocket with instance-store NVMe bound to /var/lib/kubelet: true
# - AL2023 with instance-store NVMe mounted at /nvme/disk: false
SSD_CACHE_IS_ON_EMPTY_DIR=true

helm install $CLUSTER_NAME \
    oci://$ECR_HOST/helm/onprem-clickhouse-cluster \
    --version=$CHART_VERSION \
    -n ns-$CLUSTER_NAME \
    --create-namespace \
    --set-json="account.hashedPassword=\"$HASHED_PASSWORD\"" \
    --set-json="server.image.repository=\"$ECR_HOST/clickhouse-server\"" \
    --set-json="server.storage.s3.endpoint=\"$S3_BUCKET_URL\"" \
    --set-json="server.storage.s3.region=\"$S3_BUCKET_REGION\"" \
    --set-json="server.storage.s3.bucketName=\"$S3_BUCKET_NAME\"" \
    --set-json="server.storage.s3.keyPrefix=\"$S3_KEY_PREFIX\"" \
    --set-json='server.arm64=false' \
    --set-json="server.image.tag=\"26.2.1.525-fips\"" \
    --set-json="server.podPolicy.nodeSelector.clickhouseGroup=\"$SERVER_NG_GROUP\"" \
    --set-json="server.openSSL.enabled=true" \
    --set-json="server.openSSL.required=true" \
    --set-json="server.openSSL.verificationMode=$SERVER_VERIFICATION_MODE" \
    --set-json="keeper.image.repository=\"$ECR_HOST/clickhouse-keeper\"" \
    --set-json='keeper.arm64=false' \
    --set-json="keeper.image.tag=\"26.2.1.258-fips\"" \
    --set-json="keeper.podPolicy.nodeSelector.clickhouseGroup=\"$KEEPER_NG_GROUP\"" \
    --set-json="keeper.openSSL.enabled=true" \
    --set-json="keeper.openSSL.required=true" \
    --set-json="keeper.openSSL.verificationMode=$KEEPER_VERIFICATION_MODE" \
    --set-json="serviceAccount.annotations={\"eks.amazonaws.com/role-arn\":\"$SA_IAM_ROLE_ARN\"}" \
    --set-json='server.tolerations=[{"effect":"NoSchedule","key":"clickhouse.com/do-not-schedule","operator":"Exists"}]' \
    --set-json='keeper.tolerations=[{"effect":"NoSchedule","key":"clickhouse.com/do-not-schedule","operator":"Exists"}]' \
    --set-json="server.ssdCacheConfiguration.cacheDiskSize=\"$CACHE_DISK_SIZE_GI\"" \
    --set-json="server.ssdCacheConfiguration.isOnEmptyDir=$SSD_CACHE_IS_ON_EMPTY_DIR" \
    --set-json="server.podPolicy.resources.limits.cpu=\"$SERVER_CPU\"" \
    --set-json="server.podPolicy.resources.limits.memory=\"$SERVER_MEMORY\"" \
    --set-json="server.podPolicy.resources.requests.cpu=\"$SERVER_CPU\"" \
    --set-json="server.podPolicy.resources.requests.memory=\"$SERVER_MEMORY\"" \
    --set-json="keeper.podPolicy.resources.limits.cpu=\"$KEEPER_CPU\"" \
    --set-json="keeper.podPolicy.resources.limits.memory=\"$KEEPER_MEMORY\"" \
    --set-json="keeper.podPolicy.resources.requests.cpu=\"$KEEPER_CPU\"" \
    --set-json="keeper.podPolicy.resources.requests.memory=\"$KEEPER_MEMORY\""

Key differences from the standard AWS deployment:

Setting	Standard	FIPS
Node architecture	ARM64 recommended on AWS	x86_64 required
Node AMI	Standard EKS-optimized AMI	`BOTTLEROCKET_x86_64_FIPS` or a custom FIPS-enabled x86_64 AMI
ECR endpoint	`0000000000.dkr.ecr.us-west-2.amazonaws.com`	`0000000000.dkr-ecr-fips.us-west-2.on.aws`
Server image tag	`26.2.1.525`	`26.2.1.525-fips`
Keeper image tag	`26.2.1.258`	`26.2.1.258-fips`
Operator image tag	`main-1.20441.1`	`main-1.20441.1-fips`
S3 endpoint	(default)	Explicit FIPS endpoint URL
`server.openSSL.enabled`	not set	`true`
`server.openSSL.required`	not set	`true`
`keeper.openSSL.enabled`	not set	`true`
`keeper.openSSL.required`	not set	`true`
Architecture flags	`server.arm64=true`, `keeper.arm64=true`	`server.arm64=false`, `keeper.arm64=false`
Cache configuration	`bytesPerGiRAM`	`cacheDiskSize` (explicit size); `isOnEmptyDir=true` for Bottlerocket or `false` for AL2023

Step 6: Verify Installation

Port-forward on TLS Port

With FIPS mode, ClickHouse uses TLS on port 9440 instead of the standard port 9000.

kubectl port-forward svc/c-default-xx-01-server-any 9440:9440 -n ns-default-xx-01

Create Client Configuration

Create a client-config.xml file to tell the ClickHouse client where to find the CA certificate. Update the path to your ca.crt file:

<config>
    <openSSL>
        <client>
            <caConfig>/path/to/certs/ca.crt</caConfig>
        </client>
    </openSSL>
</config>

Connect and Run a Query

clickhouse client --host localhost --port 9440 --password $PASSWORD --secure --config=client-config.xml

Run a test query:

clickhouse-cloud :) select 1;

Expected output:

SELECT 1
Query id: 825591bf-a8e5-4995-ac9f-afb864854ba2
   ┌─1─┐
1. │ 1  │
   └───┘
1 row in set. Elapsed: 0.001 sec.

Important: Port 9000 (plaintext native) and port 8123 (plaintext HTTP) will refuse connections when openSSL.required=true. Use port 9440 (native TLS) or port 8443 (HTTPS) instead.

Next Steps

Teleport integration (optional): For secure database access via Teleport Enterprise, see Integrate Teleport with ClickHouse Private. This requires SERVER_VERIFICATION_MODE=relaxed and KEEPER_VERIFICATION_MODE=relaxed.

Get started

Services and infrastructure

Bring Your Own Cloud

Data and integrations

Connect and query

Access management

Security and compliance

Monitoring and backups

AI tools and agents

Cloud API

Billing

Support

ClickHouse Private

Deploy ClickHouse with FIPS Compliance (Government)

Prerequisites

FIPS node group requirements

FIPS Version Tags

S3 Bucket Naming Restriction

Step 1: Create FIPS Certificates

Step 2: Install Kubernetes Prerequisites

Install VolumeSnapshot CRDs

Install StorageClass (Optional)

Step 3: Install Operator

Step 4: Create Certificate Secrets

Step 5: Deploy ClickHouse Cluster

Step 6: Verify Installation

Port-forward on TLS Port

Create Client Configuration

Connect and Run a Query

Next Steps

​Prerequisites

​FIPS node group requirements

​FIPS Version Tags

​S3 Bucket Naming Restriction

​Step 1: Create FIPS Certificates

​Step 2: Install Kubernetes Prerequisites

​Install VolumeSnapshot CRDs

​Install StorageClass (Optional)

​Step 3: Install Operator

​Step 4: Create Certificate Secrets

​Step 5: Deploy ClickHouse Cluster

​Step 6: Verify Installation

​Port-forward on TLS Port

​Create Client Configuration

​Connect and Run a Query

​Next Steps

Prerequisites

FIPS node group requirements

FIPS Version Tags

S3 Bucket Naming Restriction

Step 1: Create FIPS Certificates

Step 2: Install Kubernetes Prerequisites

Install VolumeSnapshot CRDs

Install StorageClass (Optional)

Step 3: Install Operator

Step 4: Create Certificate Secrets

Step 5: Deploy ClickHouse Cluster

Step 6: Verify Installation

Port-forward on TLS Port

Create Client Configuration

Connect and Run a Query

Next Steps