XML-based settings profiles and configuration files are not supported for ClickHouse Cloud. Therefore, in ClickHouse Cloud, you won’t find a config.xml file. Instead, you should use SQL commands to manage settings through settings profiles.For further details, see “Configuring Settings”
/etc/clickhouse-server/config.xml as the default configuration file, but it is also possible to specify the location of the configuration file manually at server startup using command line option --config-file or -C.
Additional configuration files may be placed into directory config.d/ relative to the main configuration file, for example into directory /etc/clickhouse-server/config.d/.
Files in this directory and the main configuration are merged in a preprocessing step before the configuration is applied in ClickHouse server.
Configuration fragments are merged in lexicographical order by their full paths. For files in the standard config.d/ directory, this is equivalent to ordering by file name. The legacy conf.d/ directory is also merged; because conf.d sorts before config.d, all its fragments are processed first when both directories exist.
To simplify updates and improve modularization, it is a best practice to keep the default config.xml file unmodified and place additional customization into config.d/.
The ClickHouse keeper configuration lives in /etc/clickhouse-keeper/keeper_config.xml.
Similarly, additional configuration files for Keeper need to be placed in /etc/clickhouse-keeper/keeper_config.d/.
It is possible to mix XML and YAML configuration files, for example you could have a main configuration file config.xml and additional configuration files config.d/network.xml, config.d/timezone.yaml and config.d/keeper.yaml.
Mixing XML and YAML within a single configuration file is not supported.
XML configuration files should use <clickhouse>...</clickhouse> as the top-level tag.
In YAML configuration files, clickhouse: is optional, if absent the parser inserts it automatically.
Merging configuration
Two configuration files (usually the main configuration file and another configuration file fromconfig.d/) are merged as follows:
- If a node (i.e. a path leading to an element) appears in both files and does not have attributes
replaceorremove, it is included in the merged configuration file and children from both nodes are included and merged recursively. - If one of the two nodes contains the
replaceattribute, it is included in the merged configuration file but only children from the node with attributereplaceare included. - If one of the two nodes contains the
removeattribute, the node is not included in the merged configuration file (if it exists already, it is deleted).
config.xml
config.d/other_config.xml
Substitution by environment variables and ZooKeeper nodes
To specify that a value of an element should be replaced by the value of an environment variable, you can use the attributefrom_env.
For example, with environment variable $MAX_QUERY_SIZE = 150000:
from_zk (ZooKeeper node):
How values are interpreted
from_env and from_zk differ in how the substituted value is interpreted:
- A direct value substituted via
from_envis taken as literal text. XML special characters such as&,<and>are escaped automatically, so an environment variable can contain any XML-representable value, for examplea&b<c>d, without breaking the configuration (see the note on XML-illegal control characters below). A direct value is never interpreted as XML, even if it happens to look like an XML fragment:<a>1</a>substitutes the literal text<a>1</a>. This also applies when the direct attribute is placed on an ordinary container such as<profiles from_env="…"/>: an existing XML-subtree substitution must be migrated to<profiles><include from_env="…"/></profiles>. For backward compatibility,<include from_env="…"/>is different: it parses the environment variable as an XML fragment and splices its children into the parent element. - A value substituted via
from_zkdepends on whether it begins with<. A value that begins with<is always parsed as an XML fragment: this is how a subtree is spliced into any element, both a structural<include from_zk="…"/>and an ordinary container such as<profiles from_zk="…"/>; it is kept for backward compatibility, because earlier versions always interpreted afrom_zkvalue as XML. A non-<value is parsed as a YAML subtree only with the explicit form<include from_zk="…" yaml="true"/>; malformed YAML then rejects the configuration. Theyaml="true"attribute enables YAML parsing wherever the generic<include>form is used, including inside a leaf element, so do not use it for a literal setting or secret. Withoutyaml="true", a non-<value is literal text, including when it is used through the generic leaf-replacement form<password><include from_zk="/secret"/></password>. This preserves values that happen to contain YAML syntax or are malformed YAML, such asabc # rotatedor[1. For that form, exact bytes are preserved only when the<include>is inline with no surrounding whitespace text nodes; for secrets and leaf settings where exact bytes matter, prefer the direct<password from_zk="…"/>form. On any other element — whether a leaf substitution such as<some_setting from_zk="…"/>or<password from_zk="…"/>, or an ordinary container such as<profiles from_zk="…"/>— a non-<value is also kept as literal text using its exact original bytes (XML special characters are escaped automatically, the same as forfrom_env), so a value such asabc: defstays the literal textabc: definstead of becoming an<abc>def</abc>sub-element. To splice a subtree into an ordinary element (rather than a structural<include>), provide it as an XML fragment (a value beginning with<).
Literal values must be valid XML 1.0 textA literal substitution (
from_env, or a non-< leaf from_zk value) preserves the exact bytes of the value for every character that XML 1.0 can represent as text. Control characters that are illegal in XML 1.0 — 0x00–0x08, 0x0B, 0x0C and 0x0E–0x1F — cannot be carried through an XML configuration at all (the XML specification forbids them even as numeric character references such as ), and a value containing one is rejected when the substitution is processed. Tab (\t), line feed (\n) and carriage return (\r) are legal and round-trip byte-for-byte.Upgrade behaviour for entity-encoded leaf valuesEarlier versions reparsed every direct
from_env value as XML, so a raw &, < or > was rejected as not well-formed and the only way to embed one was to XML-entity-encode it: an environment variable set to a&b decoded to a&b, <a>1</a> decoded to <a>1</a>, and decoded to a carriage return. A direct from_env value is now literal text, so those same environment values resolve to the literal a&b, <a>1</a> and . This means only plain-text values keep their previous meaning unchanged: an environment value that was entity-encoded for the old XML path must be set raw instead (for example set a&b, which the old path rejected but the new one accepts as the literal text a&b).Earlier versions reparsed every leaf from_zk value as XML, so a raw &, < or > was rejected as not well-formed and the only way to embed one was to XML-entity-encode it: a value stored as a&b decoded to a&b, <a>1</a> decoded to <a>1</a>, and decoded to a carriage return. A non-< leaf value is now kept as its exact original bytes, so those same nodes now resolve to the literal a&b, <a>1</a> and . This means only plain-text scalars keep their previous meaning unchanged: a leaf value that was entity-encoded for the old XML path must be stored raw instead (for example store a&b, which the old path rejected but the new one accepts as the literal text a&b).The same change applies to a plain scalar substituted through an inline <include from_zk="…"/> under a leaf setting or a secret, for example <password><include from_zk="/secret"/></password>: a scalar is now kept as its exact original bytes there too, so /secret = a&b (which used to decode to a&b) now resolves to the literal a&b and must be stored raw as a&b instead. Do not use surrounding whitespace for this form, because that whitespace is part of the resulting setting value; prefer the direct <password from_zk="…"/> form when exact bytes matter. (A from_zk value that begins with <, or a YAML subtree referenced with <include from_zk="…" yaml="true"/>, is unaffected — only unmarked non-< values are kept literal.)Default values
An element with thefrom_env or from_zk attributes may additionally have the attribute replace="1" (the latter must appear before from_env/from_zk).
In this case, the element may define a default value.
The element takes on the value of the environment variable or ZooKeeper node if set, otherwise it takes on the default value.
The previous example is repeated, but assuming MAX_QUERY_SIZE is not set:
Substitution with file content
It is also possible to replace parts of the configuration by file contents. This can be done in two ways:- Substituting Values: If an element has the attribute
incl, its value will be replaced by the content of the referenced file. The path to the file with substitutions is set by theinclude_fromelement in the server config; there is no default path, so substitutions are only read when it is specified. The substitution values are specified in/clickhouse/substitution_nameelements in this file. If a substitution specified inincldoes not exist, it is recorded in the log. To prevent ClickHouse from logging missing substitutions, specify attributeoptional="true"(for example, settings for macros). - Substituting elements: If you want to replace the entire element with a substitution, use
includeas the element name. The element nameincludecan be combined with the attributefrom_zk = "/path/to/node". In this case, the element value is replaced by the contents of the ZooKeeper node at/path/to/node. This also works with you store an entire XML subtree as a Zookeeper node, it will be fully inserted into the source element.
Changed in version 26.8: before that version, the file
/etc/metrika.xml was used implicitly whenever it existed, even when include_from was not specified. If you rely on that file, specify its path in include_from explicitly. Note that include_from is read from each configuration file individually: files that are loaded separately from the main server configuration — the users’ configuration (e.g. users.xml when it is not included in the main file) and XML dictionary configurations — each need their own include_from element; specifying it only in the main server configuration does not apply to them.merge="true". For example: <include from_zk="/some_path" merge="true">. In this case, the existing configuration will be merged with the content from the substitution and the existing configuration settings will be replaced with values from the substitution.
Encrypting and hiding configuration
You can use symmetric encryption to encrypt a configuration element, for example, a plaintext password or private key. To do so, first configure the encryption codec, then add the attributeencrypted_by with the name of the encryption codec as the value to the element to encrypt.
Unlike attributes from_zk, from_env and incl, or element include, no substitution (i.e. decryption of the encrypted value) is performed in the preprocessed file.
Decryption happens only at runtime in the server process.
For example:
from_env and from_zk can also be applied to encryption_codecs:
config.xml is given as:
users.xml is given as:
encrypt_decrypt:
hide_in_preprocessed.
For example:
User settings
Theconfig.xml file can specify a separate config with user settings, profiles, and quotas. The relative path to this config is set in the users_config element. By default, it is users.xml. If users_config is omitted, the user settings, profiles, and quotas are specified directly in config.xml.
User configuration can be split into separate files similar to config.xml and config.d/.
The directory name is defined as users_config setting without .xml postfix concatenated with .d.
The directory users.d is used by default, as users_config defaults to users.xml.
User configuration fragments are merged in lexicographical order by their full paths. For files in the standard users.d/ directory, this is equivalent to ordering by file name. The legacy conf.d/ directory is also merged; because conf.d sorts before users.d, all its fragments are processed first when both directories exist.
Note that configuration files are first merged taking into account settings, and includes are processed after that.
XML example
For example, you can have a separate config file for each user like this:YAML examples
Here you can see the default config written in YAML:config.yaml.example.
There are some differences between YAML and XML formats in terms of ClickHouse configurations.
Tips for writing configuration in YAML format are presented below.
An XML tag with a text value is represented by a YAML key-value pair
@ prefix. Note that @ is reserved by YAML standard, so must be wrapped in double quotes:
#text attribute key:
Implementation details
For each config file, the server also generatesfile-preprocessed.xml files when starting. These files contain all the completed substitutions and overrides, and they are intended for informational use. If ZooKeeper substitutions were used in the config files but ZooKeeper is not available on the server start, the server loads the configuration from the preprocessed file.
The server tracks changes in config files, as well as files and ZooKeeper nodes that were used when performing substitutions and overrides, and reloads the settings for users and clusters on the fly. This means that you can modify the cluster, users, and their settings without restarting the server.