Monitoring MongoDB logs with ClickHouse Observability

Collect MongoDB server logs with the OpenTelemetry filelog receiver and send them to ClickHouse Observability. Use the demo dataset to try the integration without a MongoDB deployment.

Prerequisites

A ClickHouse Observability service
Docker and curl
For your own logs, a self-managed MongoDB 4.4 or newer deployment with access to its JSON log files

Connection details

Fully managed
Self-managed

Open ClickHouse Observability and go to Team Settings → Data → Managed Telemetry. Copy your OTLP HTTP endpoint and API key. You can use Copy connection details to copy the configuration, or Reveal key to view the masked key.For the OTLP exporter examples below, set these variables in the terminal where you will run the collector:

export OTEL_EXPORTER_OTLP_ENDPOINT='https://YOUR_OTEL_ENDPOINT:4318'
export OTEL_EXPORTER_OTLP_AUTHORIZATION='Bearer YOUR_API_KEY'

Replace the placeholders with the exact endpoint, including its port, and API key from your service. Use the base HTTP endpoint without a /v1/logs suffix; the exporter appends the signal path.

Use your collector’s reachable OTLP HTTP endpoint (for example, http://YOUR_COLLECTOR_HOST:4318) and authentication settings. If your collector does not require authentication, omit the exporter’s headers block from the configurations below. See the self-managed getting-started guide for setup guidance.

export OTEL_EXPORTER_OTLP_ENDPOINT='http://YOUR_COLLECTOR_HOST:4318'
export OTEL_EXPORTER_OTLP_AUTHORIZATION='YOUR_INGESTION_KEY'

The endpoint must be reachable from the collector container. For a destination collector running on this computer, use http://host.docker.internal:4318. The Docker commands below include the host mapping.

Integration with existing MongoDB

Verify MongoDB logging configuration

MongoDB 4.4 and newer use structured JSON server logs. Check the systemLog section in your MongoDB configuration file. A common Linux log path is /var/log/mongodb/mongod.log.If MongoDB logs only to stdout, configure file logging in mongod.conf:

systemLog:
  destination: file
  path: /var/log/mongodb/mongod.log
  logAppend: true

Create the directory with permissions that let MongoDB write and the collector read it. For containerized deployments, share the log directory with the collector and ensure MongoDB loads the configuration file.Restart MongoDB using your deployment process, for example:

sudo systemctl restart mongod
# Or, for Docker:
docker restart <mongodb-container>

Configure the collector

Save this as mongodb-monitoring.yaml:

receivers:
  filelog/mongodb:
    include:
      - /var/log/mongodb/mongod.log
    start_at: end
    operators:
      - type: json_parser
        parse_from: body
        parse_to: attributes
        timestamp:
          parse_from: attributes.t.$$date
          layout: '2006-01-02T15:04:05.000Z07:00'
          layout_type: gotime
        severity:
          parse_from: attributes.s
          overwrite_text: true
          mapping:
            fatal: F
            error: E
            warn: W
            info: I
            debug:
              - D1
              - D2
              - D3
              - D4
              - D5

      - type: move
        from: attributes.msg
        to: body

      - type: add
        field: attributes.source
        value: "mongodb"

      - type: add
        field: resource["service.name"]
        value: "mongodb-service"
processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 400
    spike_limit_mib: 100
  transform/mongodb:
    error_mode: propagate
    log_statements:
      - context: log
        statements:
          - set(log.severity_text, ToLowerCase(log.severity_text))
  batch:
    timeout: 5s
    send_batch_size: 500
    send_batch_max_size: 500
exporters:
  otlp_http/observability:
    endpoint: ${env:OTEL_EXPORTER_OTLP_ENDPOINT}
    headers:
      authorization: ${env:OTEL_EXPORTER_OTLP_AUTHORIZATION}
service:
  pipelines:
    logs/mongodb:
      receivers: [filelog/mongodb]
      processors: [memory_limiter, transform/mongodb, batch]
      exporters: [otlp_http/observability]

The parser reads MongoDB’s timestamp and severity fields, moves msg into the log body, and keeps fields such as c, ctx, id, and attr as log attributes. Keep $$date as written: the collector resolves the escaped dollar sign to MongoDB’s $date field. The live receiver reads new entries with start_at: end.

Run the collector

Set the connection variables, then run from the configuration directory. Adjust the host log path if needed. For group-readable files, add --group-add LOG_GROUP_ID using the numeric group shown by ls -ln /var/log/mongodb.

docker run --rm --name mongodb-logs-collector \
  --add-host=host.docker.internal:host-gateway \
  --memory=512m \
  -e OTEL_EXPORTER_OTLP_ENDPOINT \
  -e OTEL_EXPORTER_OTLP_AUTHORIZATION \
  -v "$(pwd)/mongodb-monitoring.yaml:/etc/otelcol-contrib/config.yaml:ro" \
  -v /var/log/mongodb:/var/log/mongodb:ro \
  otel/opentelemetry-collector-contrib:0.161.0 \
  --config=/etc/otelcol-contrib/config.yaml

Leave the terminal open and generate server activity that writes new log entries. Stop with Ctrl+C when finished testing.

Verify logs

Open Search, select Logs, filter for ServiceName:mongodb-service, and select Last 15 minutes. You should see server messages with parsed timestamps and severity.

Demo dataset

The sample contains 6,956 MongoDB server log entries spanning approximately 24 hours. The collector refreshes their timestamps while preserving messages and structured attributes.

Download the sample

curl -fL -o mongod.log \
  https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/mongodb/mongod.log

Configure the demo collector

Save this as mongodb-demo.yaml:

receivers:
  filelog/mongodb:
    include:
      - /var/log/mongodb/mongod.log
    start_at: beginning
    operators:
      - type: json_parser
        parse_from: body
        parse_to: attributes
        timestamp:
          parse_from: attributes.t.$$date
          layout: '2006-01-02T15:04:05.000Z07:00'
          layout_type: gotime
        severity:
          parse_from: attributes.s
          overwrite_text: true
          mapping:
            fatal: F
            error: E
            warn: W
            info: I
            debug:
              - D1
              - D2
              - D3
              - D4
              - D5

      - type: move
        from: attributes.msg
        to: body

      - type: add
        field: attributes.source
        value: "mongodb-demo"

      - type: add
        field: resource["service.name"]
        value: "mongodb-demo"
processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 400
    spike_limit_mib: 100
  transform/mongodb:
    error_mode: propagate
    log_statements:
      - context: log
        statements:
          - set(log.severity_text, ToLowerCase(log.severity_text))
  transform/demo_timestamps:
    error_mode: propagate
    log_statements:
      - context: log
        statements:
          - set(log.time, Now() - Duration("2m") - (Time("2026-03-09T23:59:52.193Z", "%Y-%m-%dT%H:%M:%S.%LZ") - log.time))
  batch:
    timeout: 5s
    send_batch_size: 500
    send_batch_max_size: 500
exporters:
  otlp_http/observability:
    endpoint: ${env:OTEL_EXPORTER_OTLP_ENDPOINT}
    headers:
      authorization: ${env:OTEL_EXPORTER_OTLP_AUTHORIZATION}
service:
  pipelines:
    logs/mongodb:
      receivers: [filelog/mongodb]
      processors: [memory_limiter, transform/mongodb, transform/demo_timestamps, batch]
      exporters: [otlp_http/observability]

Run the demo collector

Set the connection variables, then run from the directory containing both files:

docker run --rm --name mongodb-logs-demo-collector \
  --add-host=host.docker.internal:host-gateway \
  --memory=512m \
  -e OTEL_EXPORTER_OTLP_ENDPOINT \
  -e OTEL_EXPORTER_OTLP_AUTHORIZATION \
  -v "$(pwd)/mongodb-demo.yaml:/etc/otelcol-contrib/config.yaml:ro" \
  -v "$(pwd)/mongod.log:/var/log/mongodb/mongod.log:ro" \
  otel/opentelemetry-collector-contrib:0.161.0 \
  --config=/etc/otelcol-contrib/config.yaml

Leave the terminal open while verifying ingestion. Stop with Ctrl+C when finished. Running the demo again re-ingests the sample with refreshed timestamps.

Verify demo logs

Open Search, select Logs, filter for ServiceName:mongodb-demo, and select Last 2 days.

Dashboards and visualization

Import the dashboard

Open Dashboards and click Import in the upper-right corner.
Upload mongodb-logs-dashboard.json.
Map the charts to your Logs source and click Finish Import.

View the dashboard

For the sample, optionally filter for ServiceName:mongodb-demo and select Last 2 days. For your own logs, use ServiceName:mongodb-service and a time range covering your records.

Troubleshooting

No logs appearing

Check the collector output for file permission, parsing, authentication, or export errors. Confirm your server writes to the mounted directory and the filenames match the receiver’s include list. The live configuration reads new entries only.

Logs not parsing correctly

Verify the file contains one JSON object per line:

tail -n 1 /var/log/mongodb/mongod.log | python3 -m json.tool

The parser expects MongoDB 4.4+ JSON logs. Legacy text logs require a different parser. Preserve the $$date escaping in the collector configuration.

Next steps

Set up alerts for errors or slow queries.
Create additional dashboards for replica set monitoring or connection tracking.
See Sending OpenTelemetry data for deployment guidance.

Introduction

Features

Administration

Guides

Integration partners

Self-managed

Monitoring MongoDB logs with ClickHouse Observability

Prerequisites

Connection details

Integration with existing MongoDB

Demo dataset

Dashboards and visualization

Troubleshooting

No logs appearing

Logs not parsing correctly

Next steps

​Prerequisites

​Connection details

​Integration with existing MongoDB

​Demo dataset

​Dashboards and visualization

​Troubleshooting

​No logs appearing

​Logs not parsing correctly

​Next steps

Prerequisites

Connection details

Integration with existing MongoDB

Demo dataset

Dashboards and visualization

Troubleshooting

No logs appearing

Logs not parsing correctly

Next steps