## Summary
Adds the authoring experience for dashboard sections (create, rename, delete, manage tiles) and introduces a polymorphic `DashboardContainer` abstraction that future-proofs the schema for tabs and groups.
Builds on #1900 (core collapsible sections mechanics). Closes#1897.
### Schema: `DashboardSection` → `DashboardContainer`
- Renamed `DashboardSectionSchema` → `DashboardContainerSchema` with a new `type` field (`'section'` for now, extensible to `'group'` / `'tab'` later)
- `sectionId` → `containerId` on tiles
- `sections` → `containers` on dashboards
- Updated across all packages: common-utils types, API Mongoose model, app types, import/export utils
### Authoring UX
| Action | How |
|---|---|
| **Create section** | Dashboard `...` overflow menu → "Add Section" |
| **Rename section** | Click the title text directly (Kibana-style inline editing) |
| **Delete section** | Hover section header → `...` → Delete Section (tiles become ungrouped, not deleted) |
| **Collapse/expand** | Click section header chevron |
| **Toggle default state** | Hover header → `...` → Collapse/Expand by Default |
| **Add tile to section** | Hover section header → `+` button opens tile editor pre-assigned to that section |
| **Move tile to section** | Hover tile → grid icon → pick target section from dropdown |
| **Move tile out** | Same dropdown → "(Ungrouped)" |
### UX polish (informed by best practices research)
- **Click-to-rename** — click section title text to edit inline (no menu navigation needed)
- **Hover-only controls** — `...` menu and `+` button only appear on section header hover, keeping view mode clean
- **"Add Section" demoted** — moved from equal-sized button to dashboard overflow menu (section creation is less frequent than tile creation)
- **"Move to Section" reordered** — placed before delete button for discoverability, uses `IconLayoutList` instead of `IconFolders`
### What's NOT in this PR (follow-up work)
- **Drag tiles between sections** — needs `react-dnd` custom drag layer; data model already supports it (`containerId` update)
- **Reorder sections** — needs sortable list library; data model supports it (array order)
- **Tabs / Groups** — new container types; just add to the `type` enum and build UIs
## Test plan
- [x] 30 unit tests pass (16 existing schema/grouping + 14 new authoring operations)
- [x] All 110 dashboard tests pass unchanged
- [x] ESLint clean
- [x] No TypeScript errors in changed files
- [x] Backward compatible — dashboards without containers render exactly as before
🤖 Generated with [Claude Code](https://claude.com/claude-code)
== Motivation ==
Time histograms on the search page silently drop data past an hour/minute boundary when the source `timestampValueExpression` includes a `toStartOf*` expression for primary key optimization.
== Details ==
When `convertToTimeChartConfig` aligns the date range to granularity it sets `dateRangeEndInclusive: false`, which is correct for the raw timestamp column (end was rounded up, so `<` gives equivalent coverage). But `timeFilterExpr` applies that same `<` uniformly to every expression in a compound `timestampValueExpression`. With a range ending at `04:08`, this yields `toStartOfHour(ts) < toStartOfHour(04:08)` = `< 04:00` — excluding the entire `04:xx` hour.
The coarse filter exists only for index pruning; the raw column already enforces exact bounds. Making it wider by one interval is harmless, making it narrower drops real rows.
== Testing ==
- `yarn jest renderChartConfig.test.ts` — 54 passed, 29 snapshots passed
- Added cases for `toStartOfHour` with exclusive end, compound expr with exclusive end, and exclusive start
## Summary
This PR adds the Spectral linter for linting our OpenAPI spec, with rules preventing fields with missing examples or descriptions, which are often enforced in the Control Plane repo.
This PR also resolves lint errors that were already present.
### Screenshots or video
### How to test locally or on Vercel
Run `make ci-lint` to lint the openapi specs
### References
- Linear Issue: Closes HDX-3768
- Related PRs:
## Summary
Some Distributed tables refer to "local" tables which are not available on the local node. To read metadata (primary key, skip indexes) for such distributed tables, we can read from `cluster(<cluster with local tables>, system, <metadata table>)` instead of the local system tables.
### Screenshots or video
After adding the distributed table as a source, we can see that the order by optimization and the skip index detection are working as intended, indicating that the cluster() queries are working and fetching the required metadata:
<img width="2099" height="624" alt="Screenshot 2026-03-19 at 8 39 48 AM" src="https://github.com/user-attachments/assets/1f8384fb-8ae1-4549-9432-e4359ac72e02" />
### How to test locally or on Vercel
<details>
<summary>First, setup two clusters</summary>
- docker compose: https://pastila.clickhouse.com/?003644f9/c65444330e3601726c00b7cc9e095e71#7W62EjQox6MnTj0vCGL0AA==GCM
- config.xml: https://pastila.clickhouse.com/?002ee55c/d82248e8db633b3fbaf14cee2ee51b0e#royNZZ4snbBpZUd8xulw5w==GCM
- config-2.xml: https://pastila.clickhouse.com/?009f57b4/cf1d51fa36eee025f17beda4da6621fa#KBbHphEhcS+1m7mBqNfY4A==GCM
- config-3.xml: https://pastila.clickhouse.com/?003115c7/e984fc157de834095bedea86bc698dca#1rEmfXnq6H0tiT4qNgayNg==GCM
- keeper.xml: https://pastila.clickhouse.com/?005dc0a8/1599254d15dbac2868f04f5ab33125c2#R90W3HfA3J0yeTNf9hrDNQ==GCM
</details>
<details>
<summary>Then setup the local and distributed tables</summary>
```sql
CREATE TABLE default.otel_logs_toStartOf on cluster hdx_cluster_2
(
`Timestamp` DateTime64(9) CODEC(Delta(8), ZSTD(1)),
`TimestampTime` DateTime DEFAULT toDateTime(Timestamp),
`TraceId` String CODEC(ZSTD(1)),
`SpanId` String CODEC(ZSTD(1)),
`TraceFlags` UInt8,
`SeverityText` LowCardinality(String) CODEC(ZSTD(1)),
`SeverityNumber` UInt8,
`ServiceName` LowCardinality(String) CODEC(ZSTD(1)),
`Body` String CODEC(ZSTD(1)),
`ResourceSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
`ResourceAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
`ScopeSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
`ScopeName` String CODEC(ZSTD(1)),
`ScopeVersion` LowCardinality(String) CODEC(ZSTD(1)),
`ScopeAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
`LogAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
INDEX idx_trace_id TraceId TYPE bloom_filter(0.001) GRANULARITY 1,
INDEX idx_res_attr_key mapKeys(ResourceAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_res_attr_value mapValues(ResourceAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_scope_attr_key mapKeys(ScopeAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_scope_attr_value mapValues(ScopeAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_log_attr_key mapKeys(LogAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_log_attr_value mapValues(LogAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_body Body TYPE tokenbf_v1(32768, 3, 0) GRANULARITY 8
)
ENGINE = MergeTree
PARTITION BY toDate(TimestampTime)
PRIMARY KEY (toStartOfMinute(TimestampTime), ServiceName, TimestampTime)
ORDER BY (toStartOfMinute(TimestampTime), ServiceName, TimestampTime, Timestamp)
TTL TimestampTime + toIntervalDay(30)
SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1;
CREATE TABLE default.otel_logs_toStartOf_distributed on cluster hdx_cluster
(
`Timestamp` DateTime64(9) CODEC(Delta(8), ZSTD(1)),
`TimestampTime` DateTime DEFAULT toDateTime(Timestamp),
`TraceId` String CODEC(ZSTD(1)),
`SpanId` String CODEC(ZSTD(1)),
`TraceFlags` UInt8,
`SeverityText` LowCardinality(String) CODEC(ZSTD(1)),
`SeverityNumber` UInt8,
`ServiceName` LowCardinality(String) CODEC(ZSTD(1)),
`Body` String CODEC(ZSTD(1)),
`ResourceSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
`ResourceAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
`ScopeSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
`ScopeName` String CODEC(ZSTD(1)),
`ScopeVersion` LowCardinality(String) CODEC(ZSTD(1)),
`ScopeAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
`LogAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1))
)
ENGINE = Distributed('hdx_cluster_2', 'default', 'otel_logs_toStartOf', rand());
ALTER TABLE otel_logs_toStartOf ON CLUSTER hdx_cluster_2 ADD INDEX text_idx(Body)
TYPE text(tokenizer=splitByNonAlpha, preprocessor=lower(Body))
SETTINGS enable_full_text_index=1;
ALTER TABLE otel_logs_toStartOf ON CLUSTER hdx_cluster_2 MATERIALIZE INDEX text_idx;
```
</details>
<details>
<summary>To test text index detection, first enable full text indexes locally in your users.xml file</summary>
```xml
<clickhouse>
<profiles>
<default>
...
<enable_full_text_index>1</enable_full_text_index>
</default>
</profiles>
...
<clickhouse>
```
</details>
### References
- Linear Issue:
- Related PRs:
## Summary
Large refactor changing the TSource type to a true discriminated union. This means that the expected fields for `kind: 'log'` will differ from those for `'trace', 'session', 'metrics'`. This avoids the current laissez faire source type that currently exists, and required extensive changes across the api and app packages. Also includes a nice addition to `useSource` - you can now specify a `kind` field, which will properly infer the type of the returned source.
This also makes use of discriminators in mongoose. This does change a bit of the way that we create and update sources. Obvious changes to sources have also been made, namely making `timeValueExpression` required on sources. Care has been taken to avoid requiring a migration.
### How to test locally or on Vercel
1. `yarn dev`
2. Play around with the app, especially around source creation, source edits, and loading existing sources from a previous version
### References
- Linear Issue: References HDX-3352
- Related PRs:
Ref: HDX-3352
## Summary
Fixes an AppNav crash caused by blank or whitespace-only user names. The user menu now normalizes the display name before generating avatar initials and includes a regression test for both whitespace-heavy and blank inputs.
### How to test locally or on Vercel
1. Open the app with a user whose name is blank or contains only whitespace and confirm the AppNav renders instead of crashing.
2. Run `cd packages/app && yarn ci:unit src/components/__tests__/AppNavUserMenu.test.tsx`.
3. Optionally run `make ci-lint` and `make ci-unit` to compare against current repo-wide CI status.
### References
- Linear Issue: https://linear.app/clickhouse/issue/HDX-3732/bug-app-crashed-at-the-appnav-component
- Related PRs: None
## Summary
- escape service name values when generating the Services page SQL filter to prevent malformed queries when names contain quotes
- switch from string interpolation to `SqlString.format` with a raw left-hand expression and escaped right-hand value
## Why
- service names containing apostrophes/single quotes broke ClickHouse query parsing, causing the Services page to error
Linear: https://linear.app/clickhouse/issue/HDX-3277/service-page-quote-escape-bug
## Summary
This PR adds lint rules disallowing Date.now() and new Date(), which can cause unnecessary re-renders.
### Screenshots or video
No behavior changes are expected.
### How to test locally or on Vercel
This can be tested in the preview environment - it is an app-only change
### References
- Linear Issue: Closes HDX-2187
- Related PRs:
## Summary
This change updates the return type of queryTableMetadata (and therefore `getTableMetadata()` and `useTableMetadata()`) to include `undefined`, since the lookup can return undefined when the provided table does not exist. This required adding undefined checks at a few call sites.
### Screenshots or video
There are no behavior changes expected, other than fewer error logs on a few pages where CTEs are used (eg. K8s dashboard).
### How to test locally or on Vercel
This can be tested in the preview environment.
### References
- Linear Issue: n/a
- Related PRs:
## Summary
- Adds a new `variant="link"` for `Button` and `ActionIcon` that renders as a plain text link — no background, no border, no padding. Text is muted by default and brightens on hover (adapts to light/dark mode).
- Adds a **Brand** theme switcher to the Storybook toolbar so stories can be previewed in both HyperDX and ClickStack themes.
- Documents the new variant in `code_style.md` and adds comprehensive Storybook stories.
## Changes
| File | What |
|------|------|
| `themes/hyperdx/mantineTheme.ts` | `link` variant for Button (`vars` + `classNames`) and ActionIcon (`vars` + `classNames`) |
| `themes/clickstack/mantineTheme.ts` | Same for ClickStack theme |
| `styles/variants.module.scss` | Hover color transition and transparent disabled state for link variants |
| `stories/Button.stories.tsx` | Link variant in CustomVariants, DisabledStates, LoadingStates |
| `stories/ActionIcon.stories.tsx` | Link variant in CustomVariants, DisabledStates, LoadingStates |
| `.storybook/preview.tsx` | Brand theme switcher (HyperDX / ClickStack) in toolbar |
| `agent_docs/code_style.md` | Documented link variant usage and guidelines |
## Test plan
- [ ] Verify `variant="link"` renders without background/border/padding in Storybook
- [ ] Verify hover brightens text in both light and dark modes
- [ ] Verify disabled state shows reduced opacity with no background
- [ ] Switch brand theme in Storybook toolbar and confirm both HyperDX and ClickStack render correctly
Made with [Cursor](https://cursor.com)
## Summary
Adds duration labels next to each **span** bar in the trace waterfall timeline so users can read the duration at a glance without hovering the tooltip.
## Changes
### Timeline chart
- **TimelineChartRowEvents**
- Renders a duration label (e.g. `12ms`, `1.2s`) **outside** each bar using `position: absolute`, so it doesn’t affect layout.
- **Placement:** Label is on the **right** of the bar when most of the bar is before the timeline midpoint, and on the **left** when most of the bar is past the midpoint (based on bar center vs. `maxVal / 2`), to keep it in the emptier side.
- Uses existing `renderMs()` from `TimelineChart/utils` for formatting.
- Wraps each bar in a container with `overflow: visible` so the duration label is not clipped by the bar’s `text-truncate` (overflow hidden).
- **TTimelineEvent** now supports optional `showDuration?: boolean`. When `false`, the duration label is hidden (default is to show).
### DBTraceWaterfallChart
- When building timeline events, sets `showDuration: type !== SourceKind.Log` so **log** rows do not show a duration (only spans do).
## Screenshots
<img width="1293" height="1187" alt="Screenshot 2026-03-11 at 18 36 34" src="https://github.com/user-attachments/assets/da04c317-a0bd-45d7-b0cc-f298564fb850" />
## Testing
- Open a trace with multiple spans and at least one correlated log; confirm duration appears beside span bars (left or right by midpoint) and does not appear beside log rows.
### References
- Closes HDX-3671
## Summary
Transforms the Event Deltas heatmap from a hard-to-interpret linear visualization into an actionable tool for latency analysis. Addresses a well-known limitation where log-normal latency distributions compress into the bottom few pixels of a linear-scale heatmap.
## Changes
### `DBHeatmapChart.tsx`
- **Log-spaced y-axis buckets**: Bucket by `log(value)` via ClickHouse `widthBucket(log(val), log(min), log(max), N)` instead of raw linear buckets. Reveals multi-modal distributions and latency bands.
- **Sqrt color mapping**: Replace `log(count)/log(20)` with `sqrt(count)/sqrt(max)` for less aggressive compression and better visual separation between sparse and dense regions.
- **Quantile lower bound + actual max upper bound**: Lower bound uses `quantile(0.01)` (log) / `quantile(0.001)` (linear) to avoid near-zero outliers stretching the axis. Upper bound uses actual `max()` so that latency spikes (typically <1% of spans) remain visible — log scale handles wide ranges naturally.
- **Color legend**: Small gradient bar component showing low-to-high count mapping.
- **Fully opaque viridis palette**: Removed transparency from colors so they work in both dark and light themes.
- **`scaleType` prop**: `HeatmapContainer` accepts `'log' | 'linear'` (default `'log'`).
### `Search/DBSearchHeatmapChart.tsx`
- **Log/Linear toggle**: `SegmentedControl` next to the heatmap form, defaulting to Log. Stored in component state.
- Passes `scaleType` through to `DBHeatmapChart`.
### `__tests__/heatmapBuckets.test.ts` (new)
- 17 unit tests covering bucket boundary computation, effectiveMin capping, widthBucket behavior, and regression tests proving the old p99 max hid latency spikes.
## Test plan
- [x] All unit tests pass with no regressions
- [x] ESLint clean
- [x] Manual: Open Event Deltas — heatmap should show log-spaced y-axis with visible latency bands
- [x] Manual: Toggle to Linear — y-axis should revert to uniform spacing
- [x] Manual: Color legend visible below the heatmap
- [x] Manual: Verify heatmap looks good in both dark and light themes
- [x] Manual: Verify latency spikes >1s are visible on the heatmap (not hidden by range capping)
## Out of scope (follow-up issues)
- #1910 — Symlog scale for zero-inclusive metrics
- #1911 — Hover tooltip with percentile context
- #1912 — Adaptive bucket density
- #1914 — Overflow bucket indicators (would enable smart quantile-based range clamping without hiding spikes)
Closes#1909🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Summary
This PR updates Raw SQL charts with support for dashboard filters, using the $__filters macro.
Lucene global filters require a Source to be included in the ChartConfig, for schema introspection and the `implicitColumnExpression` value. To support Lucene filters, this PR also updates the RawSqlChartConfig type to include optional `source`, `implicitColumnExpression`, and `from` properties. Only `source` is saved in the database. The external API has been updated to accept the `source` field for raw SQL charts as well.
Dashboard import/export has also been updated to support source mapping for raw sql charts with sources.
### Screenshots or video
Both the global filter and the drop-down filters are applied:
<img width="683" height="574" alt="Screenshot 2026-03-17 at 10 57 36 AM" src="https://github.com/user-attachments/assets/280ba0b5-55f7-4107-a55c-eeb1497ac7de" />
To render Lucene conditions, we require Source information (the `from` and `implicitColumnExpression` fields). When a source is not set, filters are therefore not applied:
<img width="782" height="618" alt="Screenshot 2026-03-17 at 10 54 41 AM" src="https://github.com/user-attachments/assets/3ad19ea7-12ee-4334-abe2-8985a0be952c" />
Similarly, if the `$__filters` macro is not used, then the filters are not applied
<img width="704" height="292" alt="Screenshot 2026-03-17 at 10 56 33 AM" src="https://github.com/user-attachments/assets/e1169e4a-2f64-4cd2-bc05-f699fecef8c1" />
Import:
<img width="993" height="669" alt="Screenshot 2026-03-17 at 3 35 16 PM" src="https://github.com/user-attachments/assets/6ebe20c0-19e2-4e90-95d0-8b02c2af0612" />
### How to test locally or on Vercel
This can be tested in the preview environment.
1. Create a saved dashboard and add some filters.
2. Create a raw sql tile, with or without the $__filters macro
3. Try selecting filters, deselecting filters, and applying global SQL and Lucene filters
4. Inspect the generated queries in the console.
### References
- Linear Issue: Closes HDX-3694
- Related PRs:
Add two new OpenCode command specifications for Linear ticket workflows:
- `do-linear.md`: Full workflow for fetching, implementing, testing, and opening a PR for a Linear ticket
- `plan-linear.md`: Research and planning workflow for creating implementation plans from Linear tickets
Ported from EE
## Summary
- The sidenav footer was using `position: absolute` with a `linear-gradient` overlay that visually faded/cut off the right border line of the cloud banner
- Replaced the overlay approach with a normal flex child (`flex-shrink: 0`) so the footer sits below the scroll area without obscuring anything
- Restored the gradient fade as a `::before` pseudo-element positioned above the footer, preserving the visual transition without overlapping content
- Removed the compensating `padding-bottom: 80px` on the onboarding section and the now-unnecessary `pointer-events` overrides
### Before and after
<img width="1112" height="408" alt="image" src="https://github.com/user-attachments/assets/83f6b28e-e42f-4bce-89a7-5ee2e89d9a9f" />
## Test plan
- [ ] Verify the sidenav footer (help button + user menu) renders correctly at the bottom
- [ ] Verify the cloud banner / onboarding checklist is fully visible without any fade effect
- [ ] Verify scrolling the sidenav works correctly when there are many saved searches/dashboards
- [ ] Verify collapsed sidenav still looks correct
- **`getRecentAlertHistories`**: Add interval-based `createdAt` lower bound to `$match` (scans only `limit * interval` instead of the full 30-day TTL window) and insert `$sort: { createdAt: -1 }` before `$group` to enable the streaming group optimization using the existing `{ alert: 1, createdAt: -1 }` index. For a 1-minute alert with limit 20, this reduces the scan from ~43k docs to ~20 docs (~2,160x).
- **`getPreviousAlertHistories`**: Add a new `{ alert: 1, group: 1, createdAt: -1 }` compound index and align the `$sort` stage to match it, so the `$group`/`$first` pattern can leverage the index directly. Also add a 7-day `createdAt` lower bound to the `$match` stage (~4x scan reduction vs. 30-day TTL).
- Update tests to use `Date.now()`-relative timestamps and pass the new `interval` parameter.
## Summary
This PR extends Raw SQL Charts to support [macros that are supported by the Grafana ClickHouse plugin](https://github.com/grafana/clickhouse-datasource?tab=readme-ov-file#macros).
Query Params and Macros are also now included as auto-complete suggestions in the SQL Editor.
### Screenshots or video
<img width="1434" height="1080" alt="Screenshot 2026-03-16 at 12 53 03 PM" src="https://github.com/user-attachments/assets/07f753e4-28f1-43f4-8add-f123dae0b12a" />
### How to test locally or on Vercel
This can be tested in Vercel preview - just reference the supported macros in a raw SQL chart.
### References
- Linear Issue: Closes HDX-3651
- Related PRs:
## Summary
This PR updates the `getTableMetadata` and `getSkipIndices` functions to handle distributed tables by looking up primary keys and indexes (respectively) from the underlying local table (since the distributed table does not have them).
- Source config inference works again
- The default order by optimization (adding `toStartOfXX()` to the search page order by when it's present in the primary key) now correctly applies when querying a distributed table source
- The date range filter now correctly filters on both `toStartOfXX(TimestampTime)` and `TimestampTime` when `toStartOfXX(TimestampTime)` is present in the primary key of the local table.
- Source schema preview now shows both the distributed table and the local table, when the source is defined by a distributed table.
- Text indexes are now detected correctly for distributed tables
### Screenshots or video
https://github.com/user-attachments/assets/d1c60964-99f0-4470-9378-a812f963c692
When text index is present, hasAllTokens is used:
<img width="848" height="139" alt="Screenshot 2026-03-16 at 10 55 24 AM" src="https://github.com/user-attachments/assets/2bd780dc-291d-495f-bd12-c636988648c1" />
### How to test locally or on Vercel
<details>
<summary>Testing locally, you'll need to create a distributed logs table with a local table that has a timestamp optimization:</summary>
```sql
CREATE TABLE default.otel_logs_toStartOf on cluster hdx_cluster
(
`Timestamp` DateTime64(9) CODEC(Delta(8), ZSTD(1)),
`TimestampTime` DateTime DEFAULT toDateTime(Timestamp),
`TraceId` String CODEC(ZSTD(1)),
`SpanId` String CODEC(ZSTD(1)),
`TraceFlags` UInt8,
`SeverityText` LowCardinality(String) CODEC(ZSTD(1)),
`SeverityNumber` UInt8,
`ServiceName` LowCardinality(String) CODEC(ZSTD(1)),
`Body` String CODEC(ZSTD(1)),
`ResourceSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
`ResourceAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
`ScopeSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
`ScopeName` String CODEC(ZSTD(1)),
`ScopeVersion` LowCardinality(String) CODEC(ZSTD(1)),
`ScopeAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
`LogAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
INDEX idx_trace_id TraceId TYPE bloom_filter(0.001) GRANULARITY 1,
INDEX idx_res_attr_key mapKeys(ResourceAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_res_attr_value mapValues(ResourceAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_scope_attr_key mapKeys(ScopeAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_scope_attr_value mapValues(ScopeAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_log_attr_key mapKeys(LogAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_log_attr_value mapValues(LogAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
INDEX idx_body Body TYPE tokenbf_v1(32768, 3, 0) GRANULARITY 8
)
ENGINE = MergeTree
PARTITION BY toDate(TimestampTime)
PRIMARY KEY (toStartOfMinute(TimestampTime), ServiceName, TimestampTime)
ORDER BY (toStartOfMinute(TimestampTime), ServiceName, TimestampTime, Timestamp)
TTL TimestampTime + toIntervalDay(30)
SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1;
CREATE TABLE default.otel_logs_toStartOf_distributed on cluster hdx_cluster
(
`Timestamp` DateTime64(9) CODEC(Delta(8), ZSTD(1)),
`TimestampTime` DateTime DEFAULT toDateTime(Timestamp),
`TraceId` String CODEC(ZSTD(1)),
`SpanId` String CODEC(ZSTD(1)),
`TraceFlags` UInt8,
`SeverityText` LowCardinality(String) CODEC(ZSTD(1)),
`SeverityNumber` UInt8,
`ServiceName` LowCardinality(String) CODEC(ZSTD(1)),
`Body` String CODEC(ZSTD(1)),
`ResourceSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
`ResourceAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
`ScopeSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
`ScopeName` String CODEC(ZSTD(1)),
`ScopeVersion` LowCardinality(String) CODEC(ZSTD(1)),
`ScopeAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
`LogAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1))
)
ENGINE = Distributed('hdx_cluster', 'default', 'otel_logs_toStartOf', rand());
ALTER TABLE otel_logs_toStartOf ON CLUSTER hdx_cluster ADD INDEX text_idx(Body)
TYPE text(tokenizer=splitByNonAlpha, preprocessor=lower(Body))
SETTINGS enable_full_text_index=1;
ALTER TABLE otel_logs_toStartOf ON CLUSTER hdx_cluster MATERIALIZE INDEX text_idx;
```
</details>
<details>
<summary>To test text index detection, first enable full text indexes locally in your users.xml file</summary>
```xml
<clickhouse>
<profiles>
<default>
...
<enable_full_text_index>1</enable_full_text_index>
</default>
</profiles>
...
<clickhouse>
```
</details>
### References
- Linear Issue: Closes HDX-3703
- Related PRs:
## Summary
ClickStack embedded seemed to sometimes show `http://localhost:8123` instead of `window.location.origin` in the onboarding connection form. These changes should harden the behavior to show `window.location.origin`
## Summary
- Adds data model support for collapsible sections: `DashboardSection` schema with `id`, `title`, `collapsed` fields
- Tiles can optionally reference a section via `sectionId`
- Dashboards gain an optional `sections` array
- `SectionHeader` component renders a clickable bar with chevron toggle and tile count when collapsed
- Per-section `ReactGridLayout` grids: collapsed sections don't mount tiles at all, so no queries fire (lazy loading)
- Fully backward compatible — dashboards without sections render exactly as before
## Architecture
Uses a **multi-grid** approach: each section gets its own `ReactGridLayout` instance. This avoids the y-offset shifting complexity of a single-grid approach and gives us lazy loading for free (collapsed sections = unmounted tiles = no API calls).
Key tradeoff: cross-section drag-and-drop is not supported in this PR — that's addressed in the follow-up authoring UX issue (#1897).
## Test plan
- [x] 13 new tests covering schema validation, backward compatibility, tile grouping, and lazy loading filtering
- [x] All 93 existing dashboard tests pass unchanged
- [x] ESLint clean (no errors)
- [x] TypeScript compiles without errors in changed files
Closes#1896🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Closes HDX-2701
Enables horizontal scrolling on the search results table so that column content is no longer clipped when the viewport is narrower than the total column widths, and improves table header styling, resize interactions, and scrollbar aesthetics.
### Changes
- **Dynamic `minWidth` on the table element** — Computes the sum of all column widths from TanStack Table's sizing state (substituting a 200px minimum for the flexible last column) and sets it as an inline `min-width` on the `<table>`. When the container is narrower than this threshold, the table overflows and the wrapper scrolls horizontally. When wider, `width: 100%` ensures the table fills the container normally.
- **Dynamic last column width** — Uses a `ResizeObserver` to track container width and computes the last column size as the remaining space after all other columns, instead of using the `UNDEFINED_WIDTH` sentinel. This ensures the last column fills remaining space responsively while respecting a 200px minimum.
- **Minimum column width** — Added `MIN_COLUMN_WIDTH` (50px) via TanStack Table's `defaultColumn.minSize` to prevent columns from being resized below a usable size. Explicitly set `minSize: 32` on the expand button column to prevent it from inheriting the 50px default.
- **Last column resizing** — Removed the `!isLast` guard so the last column now has a resize handle, making all columns consistently resizable.
- **Resize handle redesign** — Replaced the `IconGripVertical` drag icon with a minimal 1px vertical line using a CSS `::after` pseudo-element, styled with the new `--color-border-emphasis` token.
- **`min-width: 0` on flex containers** — Added `miw={0}` to the `DBSearchPage` results and pattern containers so flex children can shrink below their content size, allowing overflow to trigger scrolling.
- **Consolidate utility classes into SCSS module** — Moved `overflow: auto`, `height: 100%`, and `font-size` from inline Bootstrap utility classes into `.tableWrapper`, and moved `width: 100%` into `.table` in `LogTable.module.scss`.
- **Replace `Button` with `UnstyledButton` for sort headers** — Replaced Mantine `Button` with `UnstyledButton` for column sort headers, with custom SCSS that only darkens text on hover (no background).
- **Consolidate header styles into `TableHeader.module.scss`** — Moved `.cursorColResize` from `Table.module.scss` and `.headerCellWithAction`/`.headerRemoveButton` from `LogTable.module.scss` into a new `TableHeader.module.scss`, co-located with `TableHeader.tsx`.
- **Refactor `CsvExportButton`** — Removed the `UnstyledButton` wrapper around `CSVDownloader` to eliminate invalid `<a>` inside `<button>` markup. The `CSVDownloader` is now the root element with inline flex styling.
- **New `--color-border-emphasis` design token** — Added a slightly more prominent border color token for UI elements like resize handles, defined across HyperDX and ClickStack themes in both light and dark modes.
- **Global thin scrollbar styling** — Added app-wide custom scrollbar styles in `globals.css` for both WebKit and Firefox, providing thin (6px), rounded, semi-transparent scrollbars using theme color tokens.
### Files changed
- `packages/app/src/components/DBRowTable.tsx` — Added `containerWidth` tracking via `ResizeObserver`; added `tableMinWidth` and `lastColumnWidth` computations; set inline `minWidth` on table; added `defaultColumn.minSize`; moved utility classes to SCSS
- `packages/app/src/DBSearchPage.tsx` — Added `miw={0}` to results and pattern container Flex wrappers
- `packages/app/src/components/DBTable/TableHeader.tsx` — Replaced `Button` with `UnstyledButton`; removed `IconGripVertical`; enabled last column resizing; consolidated all style imports to `TableHeader.module.scss`
- `packages/app/src/components/DBTable/TableHeader.module.scss` *(new)* — Styles for `.sortButton`, `.resizer` (with `::after` pseudo-element line), `.headerCellWithAction`, and `.headerRemoveButton`
- `packages/app/src/components/CsvExportButton.tsx` — Removed `UnstyledButton` wrapper; `CSVDownloader` is now the root element with flex layout
- `packages/app/src/components/ExpandableRowTable.tsx` — Added `minSize: 32` to expand button column to prevent inheriting 50px default
- `packages/app/src/components/Table.module.scss` — Removed `.cursorColResize` (moved to `TableHeader.module.scss`)
- `packages/app/src/tableUtils.tsx` — Added `MIN_LAST_COLUMN_WIDTH` and `MIN_COLUMN_WIDTH` constants
- `packages/app/src/theme/themes/hyperdx/_tokens.scss` — Added `--color-border-emphasis` token
- `packages/app/src/theme/themes/clickstack/_tokens.scss` — Added `--color-border-emphasis` token
- `packages/app/src/theme/semanticColorsGrouped.ts` — Registered `color-border-emphasis` in borders group
- `packages/app/styles/LogTable.module.scss` — Added `overflow`, `height`, `font-size` to `.tableWrapper`; added `width: 100%` to `.table`; removed header styles (moved to `TableHeader.module.scss`)
- `packages/app/styles/globals.css` — Added global thin scrollbar styles for WebKit and Firefox
## Test plan
- [x] Open the Search page with multiple columns selected (e.g. Timestamp, ServiceName, SeverityText, Body, ScopeName)
- [x] Narrow the browser window — verify a horizontal scrollbar appears and columns are not cut off
- [x] Scroll horizontally — verify all column content is accessible
- [x] Widen the browser window — verify the table fills the container and no unnecessary scrollbar appears
- [x] Verify the last column still expands to fill remaining space on wide viewports
- [x] Resize columns via drag handles — verify horizontal scroll adjusts dynamically and columns cannot be resized below ~50px
- [x] Resize the **last** column — verify it now has a resize handle and works correctly
- [x] Verify the resize handle appears as a thin vertical line (not the old grip icon)
- [x] Hover over column sort headers — verify text darkens with no background change
- [x] Hover over column headers with remove button — verify remove button appears on hover
- [x] Click "Download table as CSV" — verify it works without layout issues
- [x] Verify scrollbars across the app are thin, rounded, and semi-transparent
- [x] Verify wrap lines toggle still works correctly
- [x] Switch to Event Patterns analysis mode — verify no layout regressions
- [x] Check the table in other contexts (Dashboard tiles, Pattern side panel) to confirm no layout regressions
## Summary
- Consolidate all agent coding instructions into `AGENTS.md`, which is the standard convention supported by multiple agentic coding tools (Claude Code, Cursor, Copilot, OpenCode, etc.)
- Slim down `CLAUDE.md` to a single `@AGENTS.md` reference so Claude Code inherits the shared instructions
- Added missing GitHub Action workflow section and `npx lint-staged` fallback to `AGENTS.md` to ensure full coverage of what was previously in `CLAUDE.md`
## Summary
- Enable multiple agents/developers to run `make dev-int` simultaneously from different git worktrees without Docker port conflicts
- Compute a deterministic port offset (0-99) from the worktree directory name via `cksum`, giving each worktree its own isolated Docker Compose project and port range
- Switch `.env.test` files to use `${HDX_CI_*:-default}` variable expansion (powered by `dotenv-expand`) so test processes connect to the correct dynamic ports
## How it works
Each worktree gets a unique **slot** derived from its directory name. All service ports are offset by that slot:
| Service | Base port | Example (slot 68) |
|-----------------|-----------|-------------------|
| ClickHouse HTTP | 18123 | 18191 |
| MongoDB | 39999 | 40067 |
| API test server | 19000 | 19068 |
| OpAMP | 14320 | 14388 |
Docker Compose project names are also unique (`int-<slot>`), isolating containers and networks.
Backward compatible — when no `HDX_CI_*` env vars are set, all ports fall back to their original defaults.
## Changes
- **Makefile**: Added `HDX_CI_SLOT` computation and dynamic project names/ports for all `dev-int` targets
- **docker-compose.ci.yml**: Ports use `${HDX_CI_*:-default}` env vars; removed unused OTel collector published port; removed hardcoded network name (auto-generated from project name)
- **packages/api/.env.test** / **packages/common-utils/.env.test**: Ports use `${HDX_CI_*:-default}` expansion syntax
- **packages/api/jest.config.js** / **packages/common-utils/jest.int.config.js**: Switched from `dotenv/config` to `dotenv-expand/config` to enable variable expansion
- **packages/api/package.json** / **packages/common-utils/package.json**: Added `dotenv-expand` devDependency
- **agent_docs/development.md**: Documented multi-agent worktree support
## Testing
Ran full Alert integration test suite (`make dev-int FILE=alerts`) — **6 test suites, 150 tests passed** on slot 68 with dynamic ports.
## Summary
- Fixes flaky integration test in `charts.test.ts` where 12 of 27 tests intermittently fail
- Root cause: test data timestamps (`now` and `now + 1000`) sometimes straddle a ClickHouse 1-minute time bucket boundary, producing 2 rows instead of the expected 1
- Fix: floor the base timestamp to the start of the minute so both data points always land in the same bucket
## Failing CI run
https://github.com/hyperdxio/hyperdx/actions/runs/23056501733/job/66971286014
## Summary
Fixes an issue with query settings length validation. `maxlength` only works for strings, so the max amount of query settings (10) was never enforced by mongoose.
Adds a small validator to address this.
### How to test locally or on Vercel
The max length is already enforced in the app, so make a HTTP request directly to API - or trust the integration tests :)
## Summary
This PR adds E2E test coverage for Raw SQL Dashboard Tiles.
### Screenshots or video
There are no behavior changes
### How to test locally or on Vercel
### References
- Linear Issue: Closes HDX-3586
- Related PRs:
## Summary
- Thread `filters` from `searchedConfig` through `DBSearchPageAlertModal` → `AlertForm` → `AlertPreviewChart` → `DBTimeChart` config
- The alert preview chart now respects sidebar filtering instead of showing unfiltered data
Fixes HDX-3691
## Test plan
- [x] Apply sidebar filters on the search page
- [x] Open the alert modal and check the preview chart reflects the applied filters
- [x] Verify the chart updates when filters are changed
## Summary
- `randomMongoId()` in `fixtures.ts` was generating decimal number strings (e.g. `"482937164523"`) that usually fail MongoDB ObjectId validation
- This caused the `DELETE /alerts/:id/silenced` route's Zod `objectIdSchema` middleware to reject with 400 before the handler could return 404 for non-existent alerts
- The test `prevents unsilencing an alert that does not exist` was flaky because `Types.ObjectId.isValid()` occasionally accepted the random string (12-byte path), but usually didn't
- Fixed by using `new mongoose.Types.ObjectId().toHexString()` which always produces a valid 24-char hex ObjectId
Likely fixes HDX-3187
## Summary
This PR improves chart validation. Previously, when the chart configuration was incorrect, the save and run buttons would do nothing, or would run an invalid chart config. Now, we surface some known errors to the user via a notification with helpful messages.
Additionally, this PR
1. Removes the last remaining `any` and `as` casts from the `DBEditTimeChartForm` component and its utils
2. Adds error and loading states to the SQL Preview component
### Validations
The following validations are performed:
1. A metric name is selected when a metric source is selected (pre-existing)
3. A source is selected when using the chart builder format (except for markdown charts)
4. A connection is selected when using the raw sql format
5. Each series with a non-`count` aggregation has a non-empty valueExpression
6. Number and Pie charts have only a single series
7. Raw SQL charts have a non-empty sqlTemplate
When errors are found
1. The run and save button do not run or save
2. A notification is shown with user-friendly error messages
<img width="472" height="89" alt="Screenshot 2026-03-13 at 9 04 43 AM" src="https://github.com/user-attachments/assets/18e57a4f-5f95-4984-b2cb-39fe9d441136" />
### SQL Preview Error and Loading States
<img width="1216" height="115" alt="Screenshot 2026-03-13 at 9 06 03 AM" src="https://github.com/user-attachments/assets/8394bea4-c1b6-4661-bb4e-1ef0e3078815" />
<img width="1203" height="123" alt="Screenshot 2026-03-13 at 9 05 53 AM" src="https://github.com/user-attachments/assets/3e0e103b-36a4-488a-bdf8-00e91353c50a" />
## How to test locally or on Vercel
This can be tested in the preview environment.
## References
- Linear Issue: Closes HDX-3598
- Related PRs:
## Summary
- Show attribute value distribution charts in Event Deltas immediately on load, without requiring a heatmap selection first
- Users see blue "All spans" distribution bars right away; selecting an area on the heatmap switches to red/green comparison mode (selection vs background)
- Add legend UI that shows current mode and hints at interaction
## Changes
- **deltaChartUtils.ts**: Add `ALL_SPANS_COLOR` constant (Mantine blue-6 CSS variable)
- **DBDeltaChart.tsx**: Accept nullable `xMin/xMax/yMin/yMax` props; derive `hasSelection` boolean; gate outlier/inlier queries with `enabled: hasSelection`; add `allSpansData` query for distribution mode; add legend UI showing current mode; pass `hasSelection` to `PropertyComparisonChart`
- **PropertyComparisonChart.tsx**: Accept optional `hasSelection` prop (default `true`); show single blue bar labeled "All spans" in distribution mode; hide inlier (background) bar when no selection
- **DBSearchHeatmapChart.tsx**: Always render `DBDeltaChart` (remove null-check conditional and placeholder text); pass `onAddFilter` for click-to-filter support
## Not included (follow-up)
- "X" button to clear heatmap selection and return to distribution mode — see #1906
## Test plan
- [x] Existing deltaChart unit tests pass (51 tests across 3 suites)
- [x] ESLint passes on all changed files
- [x] Manual: Open Event Deltas tab -- attribute distribution charts should appear immediately with blue bars
- [x] Manual: Select an area on the heatmap -- charts should switch to red/green comparison mode with legend update
- [ ] Manual: Clear selection -- should revert to blue distribution mode (currently only via filter action; explicit "X" button in #1906)
Closes#1824
Supersedes #1855 (rebased after #1854 merge, resolved conflicts)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
This PR refactors part of the agent infrastructure to improve internal structure and long-term maintainability.
There are no functional or user-facing changes. The Graph Explorer continues to behave exactly as before.
> Note: This lays groundwork for future extensibility around agent capabilities.
---
### Screenshots or video
No UI changes.
---
### How to test locally or on Vercel
1. Start the app locally
2. Navigate to **Graph Explorer**.
3. Use the **AI generation feature** to generate a query.
4. Verify that:
* Query generation works as expected.
* No regressions or behavioral changes are observed.
---
### References
Fixes HDX-3109
## Summary
Closes HDX-3527
- Allows zooming the trace timeline viewer by holding `cmd` or `ctrl` key and scrolling in or out, adds hint to user that this is possible.
- Allows panning on Y axis with the mouse by clicking and dragging.
- Allows scrolling horizontally as well as vertically.
### Screenshots or video
https://github.com/user-attachments/assets/2a325134-b162-4829-82b2-709495c05428
### How to test locally or on Vercel
1. Open the trace viewer and zoom and pan.
2. Test using normal mouse and trackpad.
## Summary
Define API response Zod schemas in common-utils for alerts, webhooks, team, installation, and me endpoints. Apply Response<T> typing on backend route handlers with explicit Mongoose-to-JSON serialization (ObjectId, Date, Map). Replace all `any` types and `as Promise<T>` casts in frontend TanStack Query hooks with proper generics.
### How to test locally or on Vercel
1. `yarn dev`
2. Interact with app, ensuring nothing is broken
### References
- Linear Issue: HDX-3464
HDX-3625
## Summary
- Saved search alerts were ignoring the `filters` field when building the ClickHouse chart config in `getChartConfigFromAlert`
- This caused alerts on saved searches with SQL filters (e.g., `ServiceName IN ('hdx-oss-dev-api')`, `SeverityText IN ('info')`) to evaluate against unfiltered data, producing incorrect alert results
- Added `filters: savedSearch.filters` to the chart config so `renderChartConfig` includes filter conditions in the WHERE clause
## Summary
Closes#1826
Replaces the basic max-delta sorting for event delta attributes with smarter scoring algorithms:
- **Proportional comparison scoring** (`computeComparisonScore`): Normalizes each group's percentages to sum to 100% before computing max delta, so fields with identical proportional distributions score 0 regardless of coverage rate differences. This makes the sorting resilient to different sample sizes between outlier and inlier groups.
- **Shannon entropy scoring** (`computeEntropyScore`): Returns [0, 1] where 1 = maximally useful (low entropy, dominant value among several) and 0 = not useful (single value, empty, or perfectly uniform). Prepared for future use in distribution mode.
- **Semantic boost** (`semanticBoost`): Provides a tiebreaker boost for well-known OTel attributes (e.g., `service.name`, `http.status_code`, `error`) to surface the most operationally relevant fields.
Currently uses `computeComparisonScore` for sorting. Entropy scoring will be integrated when #1824 (always-on distribution mode) merges.
## Test plan
- [x] 15 unit tests covering all three scoring functions pass
- [ ] Verify attribute order improves in practice: fields with proportionally different distributions should rank higher than fields that differ only due to sample size
- [ ] Confirm no regression in existing event delta chart behavior
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
This PR
1. Refactors the SQLEditor and SQLInlineEditor to share code (and a directory) where possible.
2. Adds auto-complete to SQLEditor, for SQL Keywords, table names, database names, columns, map keys, and functions
The SQL editor autocompletion values are pulled from any tables that are registered in HyperDX as a source (or a materialized view on a source).
The PR is broken into one commit which is just file relocation and a second where the other changes are included.
### Screenshots or video
<img width="777" height="923" alt="Screenshot 2026-03-12 at 2 26 01 PM" src="https://github.com/user-attachments/assets/238bb46d-c459-4722-837c-b1682d971c32" />
### How to test locally or on Vercel
This can be tested in the preview environment.
Things to test include the `SQLEditor` for Raw SQL Charting and the `SQLInlineEditor` (used for WHERE and ORDER BY inputs, as well as in the SourceForm.
### References
- Linear Issue: Closes HDX-3620
- Related PRs:
## Summary
- Bump `@hyperdx/browser` to 0.22.0 and pass `service.version` as an OTel resource attribute to the browser SDK, so all frontend telemetry includes the app version
- Inject `OTEL_RESOURCE_ATTRIBUTES=service.version=$CODE_VERSION` in API and all-in-one Dockerfiles (prod) and `.env.development` (dev) so the Node OTel SDK attaches service version to all backend traces, metrics, and logs
- Remove the standalone Next.js health endpoint — `/api/health` now proxies through to the API server. The original `/api/health` page was redundant since `/api/config` already serves the same purpose
## Summary
- Add `otel-cicd-action` to `release.yml` and `release-nightly.yml` workflows to export OpenTelemetry traces for CI/CD observability
- Add `actions: read` permission to `release.yml` (already present in `release-nightly.yml`)
- Service names: `release-hyperdx-oss-workflow` and `release-nightly-hyperdx-oss-workflow`
Ref: HDX-3670
## Test plan
- [x] Verify `release-nightly` workflow runs successfully with the new otel export job
- [x] Verify `release` workflow runs successfully with the new otel export job
- [x] Confirm traces appear in the configured OTLP endpoint
