ClickStack - Sample logs, traces and metrics
The following example assumes you have started ClickStack using the instructions for the all-in-one image and connected to the local ClickHouse instance or a ClickHouse Cloud instance.
Copy ingestion API key
Navigate to Team Settings and copy the Ingestion API Key from the API Keys section. This API key ensures data ingestion through the OpenTelemetry collector is secure.
Download sample data
In order to populate the UI with sample data, download the following file:
This file contains example logs, metrics, and traces from our public OpenTelemetry demo - a simple e-commerce store with microservices. Copy this file to a directory of your choosing.
Load sample data
To load this data, we simply send it to the HTTP endpoint of the deployed OpenTelemetry (OTel) collector.
First, export the API key copied above.
Run the following command to send the data to the OTel collector:
This simulates OLTP log, trace, and metric sources sending data to the OTel collector. In production, these sources may be language clients or even other OTel collectors.
Returning to the Search view, you should see that data has started to load:
Data loading will take a few minutes. Allow for the load to be completed before progressing to the next steps.
Explore sessions
Suppose we have reports that our users are experiencing issues paying for goods. We can view their experience using HyperDX's session replay capabilities.
Select Client Sessions from the left menu.
This view allows us to see front-end sessions for our e-commerce store. Sessions remain Anonymous until users check out and try to complete a purchase.
Note that some sessions with emails have an associated error, potentially confirming reports of failed transactions.
Select a trace with a failure and associated email. The subsequent view allows us to replay the user's session and review their issue. Press play to watch the session.
The replay shows the user navigating the site, adding items to their cart. Feel free to skip to later in the session where they attempt to complete a payment.
Any errors are annotated on the timeline in red.
The user was unable to place the order, with no obvious error. Scroll to the bottom of the left panel, containing the network and console events from the user's browser. You will notice a 500 error was thrown on making a /api/checkout call.
Select this 500 error. Neither the Overview nor Column Values indicate the source of the issue, other than the fact the error is unexpected, causing an Internal Error.
Explore traces
Navigate to the Trace tab to see the full distributed trace.
Scroll down the trace to see the origin of the error - the checkout service span. Select the Payment service span.
Select the tab Column Values and scroll down. We can see the issue is associated with a cache being full.
Scrolling up and returning to the trace, we can see logs are correlated with the span, thanks to our earlier configuration. These provide further context.
We've established that a cache is getting filled in the payment service, which is preventing payments from completing.
Explore logs
For further details, we can return to the Search view:
Select Logs from the sources and apply a filter to the payment service.
We can see that while the issue is recent, the number of impacted payments is high. Furthermore, a cache related to the visa payments appears to be causing issues.
Chart metrics
While an error has clearly been introduced in the code, we can use metrics to confirm the cache size. Navigate to the Chart Explorer view.
Select Metrics as the data source. Complete the chart builder to plot the Maximum of visa_validation_cache.size (Gauge) and press the play button. The cache was clearly increasing before reaching a maximum size, after which errors were generated.
