merge_memory_estimate_per_source_part_column
Estimated memory, in bytes, that a merge needs per (source part, column) regardless of how much data the parts hold, used to lowermax_parts_to_merge_at_once so that one merge takes at most a sixteenth of the
server’s memory limit (max_server_memory_usage).
A merge keeps one block from every source part alive at the same time, and every column of every one of
those blocks costs at least a minimum allocation even when the block holds a handful of rows. For a table
with thousands of columns - system.metric_log has over two thousand - merging the default 100 parts at
once takes hundreds of megabytes before a single row is read. On a server with little memory the merge
then fails with MEMORY_LIMIT_EXCEEDED, its source parts stay where they were, and the next selection
round picks the same parts again, so the table stops compacting altogether.
A merge that runs through the vertical algorithm (see enable_vertical_merge_algorithm) holds only the
key columns of every source part at once and gathers the other columns one at a time, so it is priced by
its key columns alone and keeps its full width on a small server. Whether a merge will be vertical is
predicted from the table definition and the rows and bytes of the candidate parts the same way the merge
decides it; where the prediction is unsure it prices the merge as a horizontal one. OPTIMIZE ... DEDUPLICATE
always merges horizontally and is priced as such. The projection parts that a merge keeps are merged by
child merges after it, and the merge is narrowed to fit each of them as well.
Set to 0 to disable this and select merges by max_parts_to_merge_at_once alone.
merge_total_max_bytes_to_prewarm_cache
Only available in ClickHouse Cloud. Maximal size of parts in total to prewarm cache during merge.merge_use_batch_sorting_queue
Use the batch sorting queue to reduce per-row queue overhead when merging sorted streams. Only applies to merges that do not change rows, i.e. plainMergeTree
(Ordinary merge mode). It has no effect on engines with merge-time
semantics such as ReplacingMergeTree, CollapsingMergeTree,
SummingMergeTree, AggregatingMergeTree, CoalescingMergeTree,
GraphiteMergeTree and VersionedCollapsingMergeTree, which keep using the
default queue regardless of this setting.
merge_workload
Used to regulate how resources are utilized and shared between merges and other workloads. Specified value is used asworkload setting value for
background merges of this table. If not specified (empty string), then
server setting merge_workload is used instead.
See Also