When clickhouse-server is stuck in startup or in shutdown, the regular ports do not accept connections and the server cannot be asked what it is doing.
An introspection port is a native protocol TCP listener that starts before the server begins attaching tables and stops only after the tables’ detach completes. During these windows an operator can connect to it with a stock clickhouse-client and run queries such as SHOW PROCESSLIST, SELECT * FROM system.stack_trace, or SYSTEM INSTRUMENT ADD 'QueryMetricLog::startQuery' SLEEP ENTRY 0.5.
Configuration
An introspection port is a composable protocol endpoint marked with <introspection>true</introspection>. No introspection port is configured by default:
Because it is a regular composable protocol endpoint, it supports everything an endpoint supports: host, port, description, a networks allow list, default_database, and wrapping into tls and proxy1 layers.
The introspection flag is a property of the endpoint - the entry that carries the port - and is not inherited through impl references, so a layer can be shared between an introspection endpoint and a regular one.
The stack must end in a tcp layer, because the semantics below are implemented by the native protocol handler. Marking an endpoint whose stack contains an http, mysql, postgres, prometheus or interserver layer is rejected with INVALID_CONFIG_PARAMETER at startup, rather than producing a port that stops answering exactly when it is needed.
Behavior
- Accepted queries are
SELECT, SHOW, DESCRIBE, EXPLAIN, EXISTS, KILL QUERY, SYSTEM, SET and USE. Everything else, including DDL and data-modifying queries, is rejected with QUERY_IS_PROHIBITED, because the port is open while the server state is either not fully constructed or being torn down.
SYSTEM RELOAD CONFIG and SYSTEM RELOAD USERS are rejected with QUERY_IS_PROHIBITED until the server is completely started, because reloading the configuration during startup may break the initialization order.- Queries on these ports bypass
max_concurrent_queries and the workload scheduler, like SHOW PROCESSLIST does.
- Connections are served by a dedicated thread pool, so exhaustion of the regular connection pool does not affect these ports. The pool is sized by
max_connections.
- Connections are accepted even when the server refuses regular connections because of CPU overload.
- Full normal authentication and authorization apply: connecting to this port grants no extra access rights. It does lift the concurrency limits above, so bind it to an address that only operators can reach.
- These ports are not subject to
SYSTEM START LISTEN / SYSTEM STOP LISTEN (including ALL and CUSTOM) and are not reconfigured on config reload.
Last modified on August 15, 2026
