Skip to main content
Skip to main content
Edit this page

Self-managed monitoring

This guide provides enterprise teams evaluating ClickHouse open-source with comprehensive information on monitoring and observability capabilities for production deployments. Enterprise customers frequently ask about out-of-the-box monitoring features, integration with existing observability stacks including tools like Datadog and AWS CloudWatch, and how ClickHouse’ss monitoring compares to self-hosted deployments.

Prometheus-based integration architecture​

ClickHouse exposes Prometheus-compatible metrics through different endpoints depending on your deployment model, each with distinct operational characteristics:

Self-Managed/OSS ClickHouse

Direct server Prometheus endpoint accessible via the standard /metrics endpoint on your ClickHouse server. This approach provides:

  • Complete metric exposure: Full range of available ClickHouse metrics without built-in filtering
  • Real-time metrics: Generated directly from system tables when scraped

Direct system access

Queries production system tables, which adds monitoring load and prevents cost-saving idle states

Integration examples​

External integration allows organizations to maintain established monitoring workflows, leverage existing team expertise with familiar tools, and integrate ClickHouse monitoring with broader infrastructure observability without disrupting current processes or requiring significant retraining investments. Teams can apply existing alerting rules and escalation procedures to ClickHouse metrics, while correlating database performance with application and infrastructure health within a unified observability platform. This approach maximizes ROI on current monitoring setups and enables faster troubleshooting through consolidated dashboards and familiar tooling interfaces.

Grafana Cloud monitoring​

Grafana provides ClickHouse monitoring through both direct plugin integration and Prometheus-based approaches. The Prometheus endpoint integration maintains operational separation between monitoring and production workloads while enabling visualization within existing Grafana Cloud infrastructure. See Grafana's ClickHouse documentation for configuration guidance.

Datadog monitoring​

Datadog is developing a dedicated API integration that will provide proper cloud service monitoring while respecting service idling behavior. In the interim, teams can use the OpenMetrics integration approach via ClickHouse Prometheus endpoints for operational separation and cost-efficient monitoring. For configuration guidance, see Datadog's Prometheus and OpenMetrics integration documentation.

ClickStack​

ClickStack is ClickHouse's recommended observability solution for deep system analysis and debugging, providing a unified platform for logs, metrics, and traces using ClickHouse as the storage engine. This approach relies on HyperDX, the ClickStack UI, connecting directly to the system tables inside your ClickHouse instance. HyperDX ships with a ClickHouse focused dashboard with tabs for Selects, Inserts, and Infrastructure. Teams can also use Lucene or SQL syntax to search system tables and logs, as well as create custom visualizations via Chart Explorer for detailed system analysis. This approach is ideal for debugging complex issues, performance analysis, and deep system introspection rather than real-time production alerting.

Note

Note that this approach will wake idle services as HyperDX queries the system tables directly.

ClickStack deployment options​

  • Helm: Recommended for Kubernetes-based debugging environments. Allows for environment-specific configuration, resource limits, and scaling via values.yaml.
  • Docker Compose: Deploys each component (ClickHouse, HyperDX, OTel collector, MongoDB) individually.
  • HyperDX Only: Standalone HyperDX container.

For complete deployment options and architecture details, see the ClickStack documentation and data ingestion guide.

Direct Grafana plugin integration​

The ClickHouse data source plugin for Grafana enables visualization and exploration of data directly from ClickHouse using system tables. This approach works well for monitoring performance and creating custom dashboards for detailed system analysis. For plugin installation and configuration details, see the ClickHouse data source plugin. For a complete monitoring setup using the Prometheus-Grafana mix-in with pre-built dashboards and alerting rules, see Monitor ClickHouse with the new Prometheus-Grafana mix-in.

Direct Datadog Integration​

Datadog offers a Clickhouse Monitoring plugin for its agent which queries system tables directly. This integration provides comprehensive database monitoring with cluster awareness through clusterAllReplicas functionality.

Note

This integration is not recommended for ClickHouse Cloud deployments due to incompatibility with cost-optimizing idle behavior and operational limitations of the cloud proxy layer.

Using system tables directly​

Users can perform deep query performance analysis by connecting to ClickHouse system tables, particularly system.query_log and querying directly. Using either the SQL console or clickhouse client, teams can identify slow queries, analyze resource usage, and track usage patterns across the organization.

Query Performance Analysis

Users can use the system table query logs to perform Query Performance Analysis.

Example query: Find the top 5 long-running queries across all cluster replicas:

SELECT
    type,
    event_time, 
    query_duration_ms,
    query,
    read_rows,
    tables
FROM clusterAllReplicas(default, system.query_log)
WHERE event_time >= (now() - toIntervalMinute(60)) AND type='QueryFinish'
ORDER BY query_duration_ms DESC
LIMIT 5
FORMAT VERTICAL

Community monitoring solutions​

The ClickHouse community has developed comprehensive monitoring solutions that integrate with popular observability stacks. ClickHouse Monitoring provides a complete monitoring setup with pre-built dashboards. This open source project offers a quick-start approach for teams looking to implement ClickHouse monitoring with established best practices and proven dashboard configurations.

Note

Like other direct database monitoring approaches, this solution queries ClickHouse system tables directly, which prevents instances from idling and impacts cost optimization.