Monitoring Redis logs with ClickHouse Observability

Collect Redis server logs with an OpenTelemetry collector and send them to ClickHouse Observability. Use the demo dataset to try the integration without installing Redis.

Prerequisites

A ClickHouse Observability service
Docker to run the collector examples, or an existing OpenTelemetry Collector Contrib deployment
For your own logs, an existing Redis installation and access to its log files

Connection details

Fully managed
Self-managed

Open ClickHouse Observability and go to Team Settings → Data → Managed Telemetry. Copy your OTLP HTTP endpoint and API key. You can use Copy connection details to copy the configuration, or Reveal key to view the masked key.For the OTLP exporter examples below, set these variables in the terminal where you will run the collector:

export OTEL_EXPORTER_OTLP_ENDPOINT='https://YOUR_OTEL_ENDPOINT:4318'
export OTEL_EXPORTER_OTLP_AUTHORIZATION='Bearer YOUR_API_KEY'

Replace the placeholders with the exact endpoint, including its port, and API key from your service. Use the base HTTP endpoint without a /v1/logs suffix; the exporter appends the signal path.

Use your collector’s reachable OTLP HTTP endpoint (for example, http://YOUR_COLLECTOR_HOST:4318) and authentication settings. If your collector does not require authentication, omit the exporter’s headers block from the configurations below. See the self-managed getting-started guide for setup guidance.

export OTEL_EXPORTER_OTLP_ENDPOINT='http://YOUR_COLLECTOR_HOST:4318'
export OTEL_EXPORTER_OTLP_AUTHORIZATION='YOUR_INGESTION_KEY'

The endpoint must be reachable from the collector container. For a destination collector running on this computer, use http://host.docker.internal:4318. The Docker commands below include the host mapping.

Integration with existing Redis

Verify Redis logging configuration

Check where Redis writes its logs:

redis-cli CONFIG GET logfile

Common Redis log locations:

Linux (apt/yum): /var/log/redis/redis-server.log
macOS (Homebrew): /usr/local/var/log/redis.log or /opt/homebrew/var/log/redis.log
Docker: Often logged to stdout, but can be configured to write to /data/redis.log

If Redis is logging to stdout, configure it to write to a file by updating redis.conf:

# Log to file instead of stdout
logfile /var/log/redis/redis-server.log

# Set log level (options: debug, verbose, notice, warning)
loglevel notice

Ensure the log directory exists and Redis can write to it. After changing the configuration, restart Redis using your installation’s service name:

# For systemd
sudo systemctl restart redis

# For Docker
docker restart <redis-container>

Configure the collector

Save this as redis-monitoring.yaml:

receivers:
  filelog/redis:
    include:
      - /var/log/redis/redis-server.log
    start_at: end
    operators:
      - type: regex_parser
        regex: '^(?P<pid>\d+):(?P<role>\w+) (?P<timestamp>\d{2} \w+ \d{4} \d{2}:\d{2}:\d{2}\.\d+) (?P<log_level>[.\-*#]) (?P<message>.*)$'
        parse_from: body
        parse_to: attributes
      - type: time_parser
        parse_from: attributes.timestamp
        layout_type: gotime
        layout: '02 Jan 2006 15:04:05.000'
        location: UTC
      - type: add
        field: attributes.source
        value: redis
      - type: add
        field: resource["service.name"]
        value: redis-service

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 256
    spike_limit_mib: 64
  transform/redis:
    error_mode: propagate
    log_statements:
      - set(log.severity_text, "debug") where log.attributes["log_level"] == "." or log.attributes["log_level"] == "-"
      - set(log.severity_number, SEVERITY_NUMBER_DEBUG) where log.severity_text == "debug"
      - set(log.severity_text, "info") where log.attributes["log_level"] == "*"
      - set(log.severity_number, SEVERITY_NUMBER_INFO) where log.severity_text == "info"
      - set(log.severity_text, "warn") where log.attributes["log_level"] == "#"
      - set(log.severity_number, SEVERITY_NUMBER_WARN) where log.severity_text == "warn"
  batch:
    timeout: 5s
    send_batch_size: 1024

exporters:
  otlp_http/observability:
    endpoint: ${env:OTEL_EXPORTER_OTLP_ENDPOINT}
    headers:
      authorization: "${env:OTEL_EXPORTER_OTLP_AUTHORIZATION}"

service:
  pipelines:
    logs/redis:
      receivers: [filelog/redis]
      processors: [memory_limiter, transform/redis, batch]
      exporters: [otlp_http/observability]

This configuration parses the process ID, role, timestamp, log level, and message. It sets service.name to redis-service and maps Redis log levels to SeverityText for the dashboard. Redis notice messages (*) map to info, warnings (#) to warn, and debug/verbose messages (. and -) to debug.Redis log timestamps do not include a timezone. Set location to your Redis server’s timezone, such as America/Chicago; the example uses UTC. Original timestamps and the log body are preserved. With start_at: end, only new entries are read when the collector first opens the file.

Run the collector

Set the connection variables, then run from the directory containing redis-monitoring.yaml. Adjust the host log-directory path and the receiver’s include filename to match your installation:

docker run --rm --name redis-monitoring-collector \
  --add-host=host.docker.internal:host-gateway \
  --memory=512m \
  -e OTEL_EXPORTER_OTLP_ENDPOINT \
  -e OTEL_EXPORTER_OTLP_AUTHORIZATION \
  -v "$(pwd)/redis-monitoring.yaml:/etc/otelcol-contrib/config.yaml:ro" \
  -v /var/log/redis:/var/log/redis:ro \
  otel/opentelemetry-collector-contrib:0.161.0 \
  --config=/etc/otelcol-contrib/config.yaml

Keep this terminal open and generate Redis activity that writes new log entries. The collector must be able to read the mounted files.

Verify logs

Open Search, select Logs, and filter for ServiceName:redis-service. Select a time range covering your new logs. Expand an entry to inspect SeverityText and the parsed fields under LogAttributes.

Demo dataset

The sample contains 10,000 Redis server log entries. Set the connection variables before starting.

Download the sample dataset

curl -fLO https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/redis/redis-server.log

Configure the demo collector

Save this as redis-demo.yaml in the download directory:

receivers:
  filelog/redis:
    include:
      - /var/log/redis-demo/redis-server.log
    start_at: beginning
    operators:
      - type: regex_parser
        regex: '^(?P<pid>\d+):(?P<role>\w+) (?P<timestamp>\d{2} \w+ \d{4} \d{2}:\d{2}:\d{2}\.\d+) (?P<log_level>[.\-*#]) (?P<message>.*)$'
        parse_from: body
        parse_to: attributes
      - type: time_parser
        parse_from: attributes.timestamp
        layout_type: gotime
        layout: '02 Jan 2006 15:04:05.000'
        location: UTC
      - type: add
        field: attributes.source
        value: redis-demo
      - type: add
        field: resource["service.name"]
        value: redis-demo

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 256
    spike_limit_mib: 64
  transform/redis:
    error_mode: propagate
    log_statements:
      - set(log.severity_text, "debug") where log.attributes["log_level"] == "." or log.attributes["log_level"] == "-"
      - set(log.severity_number, SEVERITY_NUMBER_DEBUG) where log.severity_text == "debug"
      - set(log.severity_text, "info") where log.attributes["log_level"] == "*"
      - set(log.severity_number, SEVERITY_NUMBER_INFO) where log.severity_text == "info"
      - set(log.severity_text, "warn") where log.attributes["log_level"] == "#"
      - set(log.severity_number, SEVERITY_NUMBER_WARN) where log.severity_text == "warn"
  transform/demo_timestamps:
    error_mode: propagate
    log_statements:
      - >-
        set(log.time,
          Now() - Duration("2m") -
          (Time("2025-10-28T14:59:22Z", "%Y-%m-%dT%H:%M:%SZ") - log.time)
        )
  batch:
    timeout: 5s
    send_batch_size: 1024

exporters:
  otlp_http/observability:
    endpoint: ${env:OTEL_EXPORTER_OTLP_ENDPOINT}
    headers:
      authorization: "${env:OTEL_EXPORTER_OTLP_AUTHORIZATION}"

service:
  pipelines:
    logs/redis:
      receivers: [filelog/redis]
      processors: [memory_limiter, transform/redis, transform/demo_timestamps, batch]
      exporters: [otlp_http/observability]

The demo-only transform/demo_timestamps processor shifts the sample into roughly the past 24 hours, ending two minutes before ingestion, while retaining its traffic pattern. The original timestamp attribute and log body stay unchanged. Omit this processor for your own Redis logs.

Run the demo collector

From the directory containing redis-server.log and redis-demo.yaml, run:

docker run --rm --name redis-demo-collector \
  --add-host=host.docker.internal:host-gateway \
  --memory=512m \
  -e OTEL_EXPORTER_OTLP_ENDPOINT \
  -e OTEL_EXPORTER_OTLP_AUTHORIZATION \
  -v "$(pwd)/redis-demo.yaml:/etc/otelcol-contrib/config.yaml:ro" \
  -v "$(pwd)/redis-server.log:/var/log/redis-demo/redis-server.log:ro" \
  otel/opentelemetry-collector-contrib:0.161.0 \
  --config=/etc/otelcol-contrib/config.yaml

Leave the terminal open while you verify ingestion. Stop the collector with Ctrl+C when finished. Running the demo again re-ingests the sample with refreshed timestamps because this example does not persist file offsets.

Verify the demo logs

Open Search, select Logs, filter for ServiceName:redis-demo, and select Last 2 days. Expand a log entry to inspect the parsed attributes and severity.

Dashboards and visualization

Import the dashboard

Open Dashboards and click Import in the upper-right corner.
Upload redis-logs-dashboard.json.
Map the charts to your Logs source and click Finish Import.

View the dashboard

Filter for ServiceName:redis-demo for the sample, or ServiceName:redis-service for your own logs. For the demo, select Last 2 days.The dashboard shows log volume and severity. Its Request rate chart counts log entries, not Redis commands.

Troubleshooting

Collector fails to start

Check the collector terminal for configuration errors. Verify the YAML file is mounted at the documented path and the endpoint and any required authentication variables are set in the terminal where you run Docker.

No logs appearing

Check the collector output for authentication or export errors and verify your connection details.
Run redis-cli CONFIG GET logfile to check that Redis writes to a file. Ensure the collector can read it and that its mounted path matches the receiver’s include path.
With start_at: end, generate new log entries after starting the collector.
Check the Logs source, service-name filter, and time range. For the demo, include transform/demo_timestamps in the pipeline and select Last 2 days.

Logs not parsing correctly

The receiver expects Redis’s standard pid:role timestamp level message format. Check your log file and adjust the regex if your format differs. Set the time parser’s location to the Redis server’s timezone.

Next steps

Set up alerts for warnings or unusual log volume.
Create additional dashboards.
See Sending OpenTelemetry data for collector configuration guidance.

Introduction

Features

Administration

Guides

Integration partners

Self-managed

Monitoring Redis logs with ClickHouse Observability

Prerequisites

Connection details

Integration with existing Redis

Demo dataset

Dashboards and visualization

Troubleshooting

Collector fails to start

No logs appearing

Logs not parsing correctly

Next steps

​Prerequisites

​Connection details

​Integration with existing Redis

​Demo dataset

​Dashboards and visualization

​Troubleshooting

​Collector fails to start

​No logs appearing

​Logs not parsing correctly

​Next steps

Prerequisites

Connection details

Integration with existing Redis

Demo dataset

Dashboards and visualization

Troubleshooting

Collector fails to start

No logs appearing

Logs not parsing correctly

Next steps