Resolve production issues, fast. An open source observability platform unifying session replays, logs, metrics, traces and errors powered by ClickHouse and OpenTelemetry.
Find a file
Drew Davis 74d925949c
feat: Support fetching table metadata for Distributed tables (#1920)
## Summary

This PR updates the `getTableMetadata` and `getSkipIndices` functions to handle distributed tables by looking up primary keys and indexes (respectively) from the underlying local table (since the distributed table does not have them).

- Source config inference works again
- The default order by optimization (adding `toStartOfXX()` to the search page order by when it's present in the primary key) now correctly applies when querying a distributed table source
- The date range filter now correctly filters on both `toStartOfXX(TimestampTime)` and `TimestampTime` when `toStartOfXX(TimestampTime)` is present in the primary key of the local table.
- Source schema preview now shows both the distributed table and the local table, when the source is defined by a distributed table.
- Text indexes are now detected correctly for distributed tables



### Screenshots or video

https://github.com/user-attachments/assets/d1c60964-99f0-4470-9378-a812f963c692

When text index is present, hasAllTokens is used:
<img width="848" height="139" alt="Screenshot 2026-03-16 at 10 55 24 AM" src="https://github.com/user-attachments/assets/2bd780dc-291d-495f-bd12-c636988648c1" />

### How to test locally or on Vercel

<details>
<summary>Testing locally, you'll need to create a distributed logs table with a local table that has a timestamp optimization:</summary>


```sql
CREATE TABLE default.otel_logs_toStartOf on cluster hdx_cluster
(
    `Timestamp` DateTime64(9) CODEC(Delta(8), ZSTD(1)),
    `TimestampTime` DateTime DEFAULT toDateTime(Timestamp),
    `TraceId` String CODEC(ZSTD(1)),
    `SpanId` String CODEC(ZSTD(1)),
    `TraceFlags` UInt8,
    `SeverityText` LowCardinality(String) CODEC(ZSTD(1)),
    `SeverityNumber` UInt8,
    `ServiceName` LowCardinality(String) CODEC(ZSTD(1)),
    `Body` String CODEC(ZSTD(1)),
    `ResourceSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
    `ResourceAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
    `ScopeSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
    `ScopeName` String CODEC(ZSTD(1)),
    `ScopeVersion` LowCardinality(String) CODEC(ZSTD(1)),
    `ScopeAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
    `LogAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
    INDEX idx_trace_id TraceId TYPE bloom_filter(0.001) GRANULARITY 1,
    INDEX idx_res_attr_key mapKeys(ResourceAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
    INDEX idx_res_attr_value mapValues(ResourceAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
    INDEX idx_scope_attr_key mapKeys(ScopeAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
    INDEX idx_scope_attr_value mapValues(ScopeAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
    INDEX idx_log_attr_key mapKeys(LogAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
    INDEX idx_log_attr_value mapValues(LogAttributes) TYPE bloom_filter(0.01) GRANULARITY 1,
    INDEX idx_body Body TYPE tokenbf_v1(32768, 3, 0) GRANULARITY 8
)
ENGINE = MergeTree
PARTITION BY toDate(TimestampTime)
PRIMARY KEY (toStartOfMinute(TimestampTime), ServiceName, TimestampTime)
ORDER BY (toStartOfMinute(TimestampTime), ServiceName, TimestampTime, Timestamp)
TTL TimestampTime + toIntervalDay(30)
SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1;

CREATE TABLE default.otel_logs_toStartOf_distributed on cluster hdx_cluster
(
    `Timestamp` DateTime64(9) CODEC(Delta(8), ZSTD(1)),
    `TimestampTime` DateTime DEFAULT toDateTime(Timestamp),
    `TraceId` String CODEC(ZSTD(1)),
    `SpanId` String CODEC(ZSTD(1)),
    `TraceFlags` UInt8,
    `SeverityText` LowCardinality(String) CODEC(ZSTD(1)),
    `SeverityNumber` UInt8,
    `ServiceName` LowCardinality(String) CODEC(ZSTD(1)),
    `Body` String CODEC(ZSTD(1)),
    `ResourceSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
    `ResourceAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
    `ScopeSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
    `ScopeName` String CODEC(ZSTD(1)),
    `ScopeVersion` LowCardinality(String) CODEC(ZSTD(1)),
    `ScopeAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
    `LogAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1))
)
ENGINE = Distributed('hdx_cluster', 'default', 'otel_logs_toStartOf', rand());

ALTER TABLE otel_logs_toStartOf ON CLUSTER hdx_cluster ADD INDEX text_idx(Body) 
	TYPE text(tokenizer=splitByNonAlpha, preprocessor=lower(Body))
	SETTINGS enable_full_text_index=1;

ALTER TABLE otel_logs_toStartOf ON CLUSTER hdx_cluster MATERIALIZE INDEX text_idx;
```
</details>

<details>
<summary>To test text index detection, first enable full text indexes locally in your users.xml file</summary>

```xml
<clickhouse>
    <profiles>
        <default>
            ...
            <enable_full_text_index>1</enable_full_text_index>
        </default>
    </profiles>
    ...
<clickhouse>
```
</details>

### References



- Linear Issue: Closes HDX-3703
- Related PRs:
2026-03-17 14:35:08 +00:00
.changeset feat: Support fetching table metadata for Distributed tables (#1920) 2026-03-17 14:35:08 +00:00
.claude chore: add playwright agents for cursor and claude (#1847) 2026-03-05 15:16:18 +00:00
.cursor chore: add playwright agents for cursor and claude (#1847) 2026-03-05 15:16:18 +00:00
.github chore: skip claude code review on draft PRs (#1889) 2026-03-12 12:42:14 +01:00
.husky chore: Add automatic api doc generation (#1397) 2025-11-21 21:14:02 +00:00
.vex build(deps): add security resolutions for vulnerable npm packages (#1740) 2026-02-26 02:14:24 +00:00
.vscode add back linting prettier via eslint (#1463) 2025-12-11 12:00:31 -07:00
.yarn/releases feat: move more codes 2024-11-21 21:44:33 -08:00
agent_docs Enable parallel integration testing across multiple worktrees (#1917) 2026-03-16 19:42:08 +00:00
docker feat: Attach service version to all internal telemetry (#1891) 2026-03-12 16:50:46 +00:00
packages feat: Support fetching table metadata for Distributed tables (#1920) 2026-03-17 14:35:08 +00:00
proxy feat: add subpath config (#1236) 2025-10-17 14:43:58 -07:00
scripts chore: Use local clickhouse instance for playwright tests (#1711) 2026-02-13 15:43:12 +00:00
smoke-tests/otel-collector fix(otel-collector): improve log level extraction with word boundaries in regex (#1747) 2026-02-18 22:16:07 +00:00
.env Release HyperDX (#1832) 2026-03-10 18:14:11 -07:00
.gitattributes first commit 2023-09-12 20:08:05 -07:00
.gitignore chore: add playwright agents for cursor and claude (#1847) 2026-03-05 15:16:18 +00:00
.kodiak.toml feat: support HYPERDX_LOG_LEVEL env var (#66) 2023-10-18 19:01:44 +00:00
.mcp.json chore: add playwright agents for cursor and claude (#1847) 2026-03-05 15:16:18 +00:00
.nvmrc chore: Update to next 16, react 19, add react compiler (#1434) 2025-12-04 23:40:59 +00:00
.prettierignore Search Relative Time Queries (#1305) 2025-10-29 15:49:10 +00:00
.prettierrc first commit 2023-09-12 20:08:05 -07:00
.yarnrc.yml feat: move more codes 2024-11-21 21:44:33 -08:00
AGENTS.md Migrate agent instructions to AGENTS.md for multi-agent support (#1925) 2026-03-16 20:27:25 +00:00
CLAUDE.md Migrate agent instructions to AGENTS.md for multi-agent support (#1925) 2026-03-16 20:27:25 +00:00
CONTRIBUTING.md chore: add playwright agents for cursor and claude (#1847) 2026-03-05 15:16:18 +00:00
DEPLOY.md update docs spelling (#1365) 2025-11-14 15:04:26 +00:00
docker-compose.ci.yml Enable parallel integration testing across multiple worktrees (#1917) 2026-03-16 19:42:08 +00:00
docker-compose.dev.yml chore: update clickhouse version for compose files to 26.1 (#1791) 2026-02-24 15:24:43 -05:00
docker-compose.yml chore: update clickhouse version for compose files to 26.1 (#1791) 2026-02-24 15:24:43 -05:00
LICENSE first commit 2023-09-12 20:08:05 -07:00
LOCAL.md chore: pull images from custom registry domain (#523) 2024-12-09 20:18:18 -08:00
Makefile Enable parallel integration testing across multiple worktrees (#1917) 2026-03-16 19:42:08 +00:00
nx.json chore: enable nx caching on common-utils builds during dev (#1880) 2026-03-11 02:19:33 +00:00
package.json build(deps): add security resolutions for vulnerable npm packages (#1740) 2026-02-26 02:14:24 +00:00
README.md chore: Update docs for ClickStack and HyperDX v2 Launch (#878) 2025-05-29 16:32:47 +00:00
tsconfig.base.json revert: api esbuild (#1280) 2025-10-21 09:27:47 +00:00
version.sh chore: align all versions on 2.0.0 (#886) 2025-06-03 20:48:08 +00:00
yarn.lock chore: Bump ai sdk to v6 (#1898) 2026-03-13 14:44:01 +00:00

hyperdx logo


HyperDX

HyperDX, a core component of ClickStack, helps engineers quickly figure out why production is broken by making it easy to search & visualize logs and traces on top of any ClickHouse cluster (imagine Kibana, for ClickHouse).

DocumentationChat on DiscordLive DemoBug ReportsContributingWebsite

  • 🕵️ Correlate/search logs, metrics, session replays and traces all in one place
  • 📝 Schema agnostic, works on top of your existing ClickHouse schema
  • 🔥 Blazing fast searches & visualizations optimized for ClickHouse
  • 🔍 Intuitive full-text search and property search syntax (ex. level:err), SQL optional!
  • 📊 Analyze trends in anomalies with event deltas
  • 🔔 Set up alerts in just a few clicks
  • 📈 Dashboard high cardinality events without a complex query language
  • { Native JSON string querying
  • Live tail logs and traces to always get the freshest events
  • 🔭 OpenTelemetry supported out of the box
  • ⏱️ Monitor health and performance from HTTP requests to DB queries (APM)

Search logs and traces all in one place

Spinning Up HyperDX

HyperDX can be deployed as part of ClickStack, which includes ClickHouse, HyperDX, OpenTelemetry Collector and MongoDB.

docker run -p 8080:8080 -p 4317:4317 -p 4318:4318 docker.hyperdx.io/hyperdx/hyperdx-all-in-one

Afterwards, you can visit http://localhost:8080 to access the HyperDX UI.

If you already have an existing ClickHouse instance, want to use a single container locally, or are looking for production deployment instructions, you can view the different deployment options in our deployment docs.

If your server is behind a firewall, you'll need to open/forward port 8080, 8000 and 4318 on your firewall for the UI, API and OTel collector respectively.

We recommend at least 4GB of RAM and 2 cores for testing.

Hosted ClickHouse Cloud

You can also deploy HyperDX with ClickHouse Cloud, you can sign up for free and get started in just minutes.

Instrumenting Your App

To get logs, metrics, traces, session replay, etc into HyperDX, you'll need to instrument your app to collect and send telemetry data over to your HyperDX instance.

We provide a set of SDKs and integration options to make it easier to get started with HyperDX, such as Browser, Node.js, and Python

You can find the full list in our docs.

OpenTelemetry

Additionally, HyperDX is compatible with OpenTelemetry, a vendor-neutral standard for instrumenting your application backed by CNCF. Supported languages/platforms include:

  • Kubernetes
  • Javascript
  • Python
  • Java
  • Go
  • Ruby
  • PHP
  • .NET
  • Elixir
  • Rust

(Full list here)

Once HyperDX is running, you can point your OpenTelemetry SDK to the OpenTelemetry collector spun up at http://localhost:4318.

Contributing

We welcome all contributions! There's many ways to contribute to the project, including but not limited to:

Motivation

Our mission is to help engineers ship reliable software. To enable that, we believe every engineer needs to be able to easily leverage production telemetry to quickly solve burning production issues.

However, in our experience, the existing tools we've used tend to fall short in a few ways:

  1. They're expensive, and the pricing has failed to scale with TBs of telemetry becoming the norm, leading to teams aggressively cutting the amount of data they can collect.
  2. They're hard to use, requiring full-time SREs to set up, and domain experts to use confidently.
  3. They requiring hopping from tool to tool (logs, session replay, APM, exceptions, etc.) to stitch together the clues yourself.

We hope you give HyperDX in ClickStack a try and let us know how we're doing!

Contact

HyperDX Usage Data

HyperDX collects anonymized usage data for open source deployments. This data supports our mission for observability to be available to any team and helps support our open source product run in a variety of different environments. While we hope you will continue to support our mission in this way, you may opt out of usage data collection by setting the USAGE_STATS_ENABLED environment variable to false. Thank you for supporting the development of HyperDX!

License

MIT