memory_worker_correct_memory_tracker
Whether the background memory worker corrects the global memory tracker, on every tick, from an external measurement of the memory the process really uses: the cgroup memory usage when cgroups are available (seememory_worker_use_cgroup), otherwise jemalloc’s stats.resident.
The global memory tracker is a counter: allocations add to it and deallocations subtract from it. Any accounting asymmetry stays in it for the lifetime of the process, because nothing else lowers it, and an upward drift is never worked off — the memory it describes has already been freed. Once the drift alone exceeds max_server_memory_usage, every allocation fails, down to the zero-byte check at the start of a connection, and the server rejects all queries while using a fraction of its limit. Correcting from a measurement bounds the lifetime of such a drift to one tick of the worker (memory_worker_period_ms, by default 50 ms when reading from cgroups and 100 ms when reading from jemalloc).
The correction does not hide the drift. MemoryTrackingUncorrected keeps the value the tracker would have had with no corrections applied (a snapshot of the plain counter, refreshed on every tick of the worker), so MemoryTrackingUncorrected - MemoryTracking is the drift accumulated so far.
Setting this to 0 restores the behavior of previous versions: the tracker is corrected only on the first tick of the worker and whenever it goes negative.
memory_worker_decay_adjustment_period_ms
Duration in milliseconds that memory pressure must persist before dynamically adjusting jemalloc’sdirty_decay_ms. When memory usage remains above the purge threshold for this period, automatic dirty page decay is disabled (dirty_decay_ms=0) to aggressively reclaim memory. When usage stays below the threshold for this period, the default decay behavior is restored. Set to 0 to disable dynamic adjustment and use jemalloc’s default decay settings.
memory_worker_dynamic_hard_limit
Whether the background memory worker periodically recomputes the server’s hard memory limit at runtime as(resident memory + system available memory) * max_server_memory_usage_to_ram_ratio, so the server leaves headroom for other processes running on the same host.
When disabled, max_server_memory_usage_to_ram_ratio only caps the hard memory limit statically, at startup and on configuration reload, as in previous versions.
Has no effect when max_server_memory_usage_to_ram_ratio is 0.
memory_worker_period_ms
Tick period of background memory worker which corrects memory tracker memory usages and cleans up unused pages during higher memory usage. If set to 0, default value will be used depending on the memory usage sourcememory_worker_purge_dirty_pages_threshold_ratio
The threshold ratio for jemalloc dirty pages relative to the memory available to ClickHouse server. When dirty pages size exceeds this ratio, the background memory worker forces purging of dirty pages. If set to 0, forced purging based on dirty pages ratio is disabled.memory_worker_purge_total_memory_threshold_ratio
The threshold ratio for purging jemalloc relative to the memory available to ClickHouse server. When total memory usage exceeds this ratio, the background memory worker forces purging of dirty pages. If set to 0, forced purging based on total memory is disabled.memory_worker_rss_speculative_reserve_ratio
On eachMemoryWorker tick, reserve an additional
ratio * min(resident - previous_resident, resident - tracked) on top of
the observed RSS, on the assumption that the next tick may grow by the same
amount as the last one (resident - previous_resident is the RSS growth
over the last tick). The growth is capped by resident - tracked, the part
of RSS not visible to the global memory tracker, because growth that is
already tracked is handled by the ordinary hard-limit check. The reservation
is applied to the rss counter that the global hard-limit check consults
via MemoryTracker::allocImpl, so when the extrapolated value crosses
max_server_memory_usage, subsequent allocations throw
MEMORY_LIMIT_EXCEEDED before the kernel OOM-killer fires. A value of 0
disables speculation (falling back to rss = resident); the default 1
reserves one full growth delta of headroom for the next interval. Under
sanitizers (ASan, UBSan, MSan, TSan) the default is 0, because the
resident - tracked gap is dominated by sanitizer shadow / runtime overhead
rather than tracker bookkeeping lag.