Validating webhook body templates before they save
Demo by @brandon-pereira
Previously, you could save a Generic or incident.io webhook with a broken body template and get no warning. Every alert delivery would then fail, with the error only appearing in ClickStack’s traces and server logs. A common cause was JSON-escaping quotes inside a Handlebars expression, such as
{{#if (eq state \"ALERT\")}}: valid JSON, but an invalid template.
The webhook form now checks the template as you edit and shows errors below the editor, including the line and column, such as Template error on line 1, column 29. It waits until you’ve stopped typing for 300 ms, so it won’t complain halfway through a {{ … }} expression. Empty bodies are skipped because they use the service’s default template, as are Slack bodies, which aren’t rendered. Below the editor, you’ll also find the supported variables, including {{alertId}}, {{status}}, {{value}}, and {{groupKey}}, each with a short description.
The same validation now runs server-side across the internal API, external API v2, and MCP clickstack_save_webhook tool. Invalid templates return a 400 on create, update, or test, with the Handlebars error and its line and column. Tests fail before anything is sent. Validation checks that the template compiles; it doesn’t render the result or check whether the output is valid JSON.
For automated workflows, the new POST /api/v2/webhooks/validate endpoint lets Terraform check webhook configurations at plan time. Like POST /api/v2/dashboards/validate, it validates without saving. It runs the same URL and body checks as create and always returns 200 with { valid, errors, normalized }. Name uniqueness isn’t checked.
Related PRs: #3350 Show webhook body template compile errors in the editor, #3365 Reject webhook body templates that don’t compile, add /api/v2/webhooks/validate
Auto-detecting a PromQL source during onboarding
Demo by @karl-power
When a connection has no sources, ClickStack looks for OpenTelemetry tables and automatically creates Logs, Traces, Metrics, and Sessions sources. With PromQL enabled (
NEXT_PUBLIC_ENABLE_PROMQL=true), it now creates a PromQL source too if it finds a ClickHouse TimeSeries table.
Detection looks for tables with engine = 'TimeSeries', ignoring the inner MergeTree tables they create. If there’s more than one candidate, it checks in this order:
prometheus.metrics, the table used in the ClickHouse Prometheus docs.metrics_tsin the same database as the OpenTelemetry tables, created by the collector’s PromQL migration.metrics_tsin any database.
TimeSeries table, ClickStack uses it regardless of its name.
Connections with only a TimeSeries table now get a PromQL source without being sent to manual setup. If detection or PromQL source creation fails, the other sources stay in place.
Related PRs: #3310 Auto-detect a PromQL source during onboarding, #3311 Provision a PromQL source in the dev default sources
Series limits and minimum granularity for PromQL charts
Demo by @karl-power
PromQL line and stacked bar charts now have the same Series Limit display setting as builder and raw SQL charts. Previously, they capped rendered series at 250 in the browser, with an option to load the rest, but the setting was hidden and wasn’t saved with the tile.
The limit is applied in the browser, just as it is for raw SQL charts. Prometheus’s own
limit parameter returns a subset sorted by labels rather than the top series by value, and the ClickHouse table-function fallback ignores it. The hidden-series hint also gives PromQL-specific advice: narrow your label matchers or add sum by (...), rather than using GROUP BY or WHERE.
PromQL sources now have a Minimum auto granularity setting too, matching metric sources. This sets the smallest bucket size auto granularity can choose for line, number, table, pie, and bar charts, including their generated PromQL previews. $__rate_interval also uses this value as the scrape interval, falling back to 15s when it’s unset. Explicit tile granularity settings are unchanged.
Related PRs: #3292 Support minimum auto granularity on PromQL sources, #3294 Support series limit on PromQL time charts
Source-level access control for PromQL sources
Demo by @karl-power
This feature applies to Managed ClickStack only.
Role-based access control now applies to PromQL sources, just as it does to logs, traces, and metrics. A fine-grained source role can grant Read access to one PromQL source while restricting another.
Every
/v1/prometheus/* route now checks Read access to the source, returning 403 RBAC_PERMISSION_DENIED when it’s missing. If the caller’s source access rules hide the source entirely, the response is 404 Source not found, matching the sources API. This change doesn’t add ClickHouse-side role enforcement: in ClickStack Cloud, PromQL queries already use the user’s own token.
Ad hoc filters for dashboards
Demo by @pulpdrew
Dashboards now have an Ad hoc keys and values filter type alongside the existing variables and filters. Each ad hoc filter uses one or more ClickHouse sources, such as logs and traces, or a Prometheus source to suggest available keys and values. By default, conditions apply to every tile using those sources, but you can limit which sources they apply to. The feature is behind
NEXT_PUBLIC_ENABLE_ADHOC_FILTERS, with about half of the planned work complete.
Click the filter’s input in the dashboard filter bar to choose a key, an operator, and a value, with suggestions for both keys and values. SQL sources support equals, not equals, regex, not regex, LIKE, and NOT LIKE. PromQL sources use Prometheus labels rather than column keys. Applied conditions appear as pills you can edit, switch to exclusions, or clear. Selections are stored in the URL and can be saved as part of the dashboard’s default filter state.
How conditions reach a tile depends on its type:
- Builder tiles use the existing
filterconfiguration property, so you don’t need to add a variable to yourWHEREclause. - Raw SQL tiles use
$__filters. Your query needs to include this macro, and you’ll see a warning if it’s missing. With no conditions applied, it expands to1 = 1. - PromQL tiles use the same parser library as Grafana to add conditions as label matchers to every metric selector.
Moving SELECT and ORDER BY into the results table
Demo by @elizabetdev
This is exploratory work, with no commitment to ship it.
The search page currently puts
SELECT and ORDER BY next to WHERE, even though they only affect the results table. Switch to Event Patterns or Event Deltas, and they’re still there, but changing them does nothing. Users we’ve spoken to also didn’t realize SELECT was where they could choose columns. They were looking for column and sort controls.
This prototype moves both into the results table toolbar as Columns and Sort, hiding them in Event Deltas and Event Patterns. Columns lists your current SELECT items, with options to drag them into order or remove them. Pick a suggestion or type an expression such as upper(Body) as message, then click Add. A SQL tab lets you edit the full SELECT directly.
Sort lists each ORDER BY field and its direction, including tuples such as (toDateTime(Timestamp), Timestamp) DESC. You can edit the raw clause under SQL too. Expressions the controls can’t split, such as Timestamp DESC NULLS LAST, open in the SQL tab. Changes in the controls build the SQL for you, and you can still write it yourself.
Apply updates the clause and runs the search, Cancel discards your changes, and Reset to default restores the source default. Table actions for resetting widths, wrapping lines, viewing generated SQL, and exporting CSV move to icon buttons on the right of the toolbar.
That also gives the search form a little more room. The source sits on the left of the first line, with Update and Alerts on the right. The WHERE clause gets its own full-width line alongside Live Tail, the refresh interval, and Run.
The PR is still a draft, and we’d love to hear how this feels to use.
Related PRs: #3387 Edit search columns and sort from the results table (open)
A usable search page on a phone
Demo by @elizabetdev
Opening an alert on your phone should let you get straight to the logs. Previously, the filter sidebar couldn’t shrink below 200px, the query row didn’t wrap, and results were clipped inside a
100vh frame. You couldn’t even scroll to the logs.
The search page now switches to a compact layout below 900px, where the navigation already collapses. Filters start closed and open in a drawer through Show filters, leaving the full width for the log table. SELECT and ORDER BY sit behind a Columns toggle, while the search box stays visible. The histogram stays too, shortened to 72px, with results scrolling in the space below the toolbar. Opening the filter drawer won’t change your desktop preference, and widening the window past 900px restores the desktop layout.
The compact-viewport hook and responsive side panel live in packages/app/src/components/CompactViewport/, ready for other pages to reuse. Search is the first to use them. This landed before the columns-and-sort prototype covered in this post, which would move SELECT and ORDER BY into the results table at every screen width.
Related PRs: #3376 Keep saved search usable on a phone
Streaming filter keys and values into the sidebar
Demo by @knudtty
Previously, the search page’s filter sidebar showed a single spinner until every filter query finished. The queries already streamed their results, but you still had to wait for the slowest one before you could use any filters.
Now, keys appear first with individual loading indicators, and values fill in as each query returns. The queries themselves haven’t changed; the sidebar just shows results as they arrive. It’s particularly noticeable over longer time ranges, such as a two-day search.
We’ve also fixed the sidebar forgetting where you left it. Filter groups are removed from the DOM when they scroll out of view to keep the browser responsive. Previously, scrolling back meant finding expanded groups collapsed and loaded distributions gone. Both now stay as you left them.
Related PRs: #3348 Stream filter keys and value results to UI, #3377 Persist filter expand state upon scrolling