48.9k
Synthetic monitoring is an active monitoring technique that runs scripted probes (HTTP checks, API transactions, and full browser journeys) against an application on a fixed schedule from controlled locations. Because the probes run continuously and predictably, they detect outages, broken flows, and latency regressions before real users encounter them.
Google's Site Reliability Engineering book (O'Reilly, 2016) classifies this approach as black-box monitoring: "testing externally visible behavior as a user would see it." The probe doesn't know how the system is built; it only verifies that the system responds correctly from the outside.
Why synthetic monitoring matters
Synthetic monitoring is the only signal that exists when there is no traffic. Passive telemetry such as logs, traces, and real user sessions reports nothing at 3 a.m. if no user is awake to hit the failing endpoint, and nothing at all for a checkout flow that hasn't launched yet. A scripted probe fails on schedule regardless.
That makes synthetic checks the standard basis for availability alerting and SLO verification. A service with a 99.9% monthly availability target has an error budget of 43.2 minutes per 30-day month; a probe that confirms an outage within 2-3 minutes preserves most of that budget, while waiting for user complaints can spend all of it. Synthetic results also feed golden-signal dashboards with a consistent baseline: same request, same location, same interval, so any change in the measurement is a change in the system.
How does synthetic monitoring work?
Synthetic monitoring works by executing a scripted interaction against a target on a timer, asserting on the result, and recording a timestamped measurement. A scheduler triggers probes from one or more locations; each probe performs the interaction, checks assertions such as status code, response content, and latency thresholds, and emits a pass/fail result with timing breakdowns.
The pipeline has four parts:
- Script: the interaction to perform, whether a single HTTP GET, a multi-step API sequence, or a browser flow automated with a tool like Playwright or Selenium.
- Scheduler and probe locations: where and how often the script runs; multiple geographic locations separate regional network issues from application faults.
- Assertions: what "healthy" means, such as HTTP 200, a JSON field present, a page element rendered, or total duration under a threshold.
- Results pipeline: every run produces an event (success flag, duration, DNS/TLS/TTFB timings, location) that flows to storage for alerting and trend analysis.
Open-source implementations follow the same shape: the Prometheus blackbox_exporter, part of the CNCF-graduated Prometheus project, probes endpoints over HTTP, HTTPS, DNS, TCP, and ICMP and exposes the results as metrics.
Types of synthetic monitoring checks
Synthetic checks fall into three tiers, ordered by how much of the user experience they exercise, and by how expensive each run is.
| Check type | What it validates | Typical frequency |
|---|---|---|
| Uptime / ping (ICMP, TCP, single HTTP request) | The endpoint is reachable, TLS is valid, status code and response time are within bounds | Every 30-60 seconds |
| API / multi-step transaction | A sequence of API calls (authentication, data retrieval, writes) works end to end with correct payloads at each step | Every 1-5 minutes |
| Full browser journey (Playwright/Selenium-class) | A real browser completes a user flow (login, search, checkout) including JavaScript execution and page rendering | Every 5-15 minutes |
Uptime checks are cheap and run everywhere; browser journeys spin up a headless browser per run, so they cost one to two orders of magnitude more per execution and are reserved for the flows that earn revenue or block users outright.
How often should synthetic checks run?
Check frequency is a trade-off between detection latency, cost, and noise. The check interval sets a floor on detection time: a 5-minute interval with two confirming failures before alerting means an outage can run 10-15 minutes before anyone is paged. Higher frequency buys faster detection at a linearly higher probe and storage bill.
A reasonable starting point is to run uptime checks on critical endpoints every 30-60 seconds, since they are cheap and detection latency matters most for hard outages; run API transactions every 1-5 minutes; run browser journeys every 5-15 minutes, because rendering accounts for most of the cost and the flows they cover degrade less suddenly. Multiply by locations deliberately: a 60-second check from 5 locations produces 7,200 results per endpoint per day, and every additional location multiplies the volume.
Synthetic monitoring vs real user monitoring
Synthetic monitoring measures what a scripted probe experienced; real user monitoring (RUM) measures what actual users experienced. Synthetic gives control: a fixed request from a known location on a known schedule. That makes it ideal for alerting, SLA verification, and pre-production testing. RUM gives coverage: the long tail of devices, networks, geographies, and navigation paths that no script anticipates.
Each covers what the other cannot. A probe catches the outage no user has hit yet; RUM reveals that users on mid-range Android devices in one region see 6-second page loads that every probe misses. Mature teams run both, using synthetic for availability signals and RUM for experience truth.
What synthetic check results look like at rest
Every probe run emits a small structured event, and at scale those events become a time-series dataset like any other observability signal. A single API-check result looks like this:
{
"timestamp": "2026-07-28T09:15:00Z",
"check_name": "checkout-api-login",
"check_type": "api_transaction",
"probe_location": "eu-west-1",
"url": "https://api.example.com/v1/login",
"status_code": 200,
"success": true,
"duration_ms": 412,
"dns_ms": 11,
"tls_ms": 38,
"ttfb_ms": 301,
"steps_completed": 3
}Questions about synthetic health are aggregations over these events: failure rate by location, p95 duration trend, time-to-recovery per incident. As a SQL shape:
SELECT
probe_location,
quantile(0.95)(duration_ms) AS p95_ms,
countIf(NOT success) / count() AS failure_rate
FROM synthetic_results
WHERE check_name = 'checkout-api-login'
AND timestamp > now() - INTERVAL 24 HOUR
GROUP BY probe_location
ORDER BY p95_ms DESCRetention matters more than volume here: teams keep synthetic history for months to prove SLA compliance and to compare a regression against last quarter's baseline.
Synthetic monitoring with ClickStack
ClickStack, the ClickHouse-based open-source observability stack, treats synthetic results as one more OpenTelemetry signal. Probes that emit results as OTLP logs or metrics land in the same ClickHouse tables as application traces and logs, so a failed check is queryable next to the backend telemetry from the same minute. In HyperDX, ClickStack's UI, a probe failure at 09:15 sits alongside the traces and error logs that explain it. Because storage is columnar ClickHouse, keeping a year of per-run results for SLA evidence is a routine aggregation rather than an archive job.
To try the pattern, point an OTLP-emitting prober at a ClickStack instance and chart success and duration_ms by location.
Frequently asked questions
What is synthetic transaction monitoring?
Synthetic transaction monitoring is the subset of synthetic monitoring that scripts multi-step business flows (log in, add to cart, pay) rather than single requests. Each step carries its own assertions and timings, so a failure report names the exact step that broke, not just the flow.
Synthetic monitoring vs real user monitoring: which do you need?
Most production teams need both, and the order matters: set up synthetic checks first, since they work from day one with zero traffic, then add RUM once real users arrive to capture the device and network variance no script reproduces.
How do I set up synthetic monitoring for an API?
List the endpoints whose failure should page someone, then script each as a request with assertions: expected status code, a required response field, and a latency threshold. Run the scripts every 1-5 minutes from at least two locations, alert on consecutive failures rather than single ones, and store every result for trend analysis.
Is synthetic monitoring the same as uptime monitoring?
No. Uptime monitoring is the simplest tier of synthetic monitoring. An uptime check verifies that an endpoint answers; synthetic monitoring as a category also includes multi-step API transactions and scripted browser journeys that validate whole user flows. Every uptime check is synthetic, but most synthetic monitoring goes beyond reachability.
