Requires ClickHouse operator version
1.23234.1 or later. The operator image and the clickhouse-operator-helm chart share the same version.Why the extra step is needed
A StorageClass-backed cache comes from the StatefulSet’svolumeClaimTemplates, and that field is
immutable.
The Operator cannot add it to, or remove it from, a StatefulSet that already
exists. The StatefulSet has to be replaced instead.
Move onto a StorageClass
1
Set the StorageClass in your Helm values
accessModes and volumeMode default to ReadWriteOnce and Filesystem. The
cache is per-replica and never shared, so these suit most drivers.Size the claim at or above the cache budget you give ClickHouse. ClickHouse
fills to that budget whatever the volume’s actual size, so an undersized claim
fails writes with ENOSPC instead of evicting from the cache.Upgrade the release. The replicas will start failing as described above.2
Recreate each Server StatefulSet, one replica at a time
Each Server replica runs in its own StatefulSet with a single pod. List them:Delete one and wait for the Operator to rebuild it:The Operator recreates it from scratch, this time with the cache claim template
in place, which takes a couple of minutes. Confirm the replica is healthy
before moving on to the next one.
Deleting the StatefulSet does not delete your data. All data is stored in
object storage, so this migration to a StorageClass-backed filesystem cache is
safe to perform.
3
Verify
The recreated StatefulSet should list both claim templates:And each replica should have a bound cache claim:Once every replica has been recreated the instance returns to
Running:Move back off a StorageClass
The same constraint applies in reverse. Once a StatefulSet carries a cache claim template the Operator cannot remove it, so clearingstorage.storageClassName and choosing another backing is not complete until
each StatefulSet has been recreated. Follow the same procedure.
The old ch-cache-volume claims stay bound afterwards, because the Operator
only collects cache volumes whose replica has gone away. Delete them once every
replica is healthy to release the capacity: