Commit graph

229 commits

Author SHA1 Message Date
Mike Shi
d72d1d2d26
Add ingestion key check to otel collector via OpAMP (#825)
HDX-1698
2025-05-23 01:41:35 +00:00
Warren
9004826d7c
fix: remove total number of webhook limit (#842)
Ref: HDX-1773
2025-05-23 00:03:54 +00:00
Warren
c60b975e29
chore: bump node to v22.16.0 (#839)
Ref: HDX-1757
2025-05-22 17:05:30 +00:00
Aaron Knudtson
fa7875c427
feat: add Summary and Exponential Histogram metrics to source form (#832) 2025-05-21 13:22:04 -04:00
Warren
e7262d1288
feat: introduce all-one-one (auth vs noauth) multi-stage build (#821)
1. Merge 'fullstack' and 'local' (auth + noauth) builds into a single Dockerfile
2. Introduce 'all-in-one-auth' and 'all-in-one-noauth' build stages
3. Lock `IS_LOCAL_APP_MODE` env var
4. Fix bug in ctrl-c exit with docker run
5. Enable alerts in local mode (no-auth)
6. Build `common-utils` on the fly (no longer needing pulling pkg from npm)

Ref: HDX-1709
Ref: HDX-1713
Ref: HDX-1254
Ref: HDX-1729

To match v2 product definition, we are going to release three images:
- hyperdx/hyperdx (--target=prod): app only without any other deps (clickhouse, otelcol, mongodb), used in default compose + helm deployment
- hyperdx/hyperdx-all-in-one (--target=all-in-one-auth): all-in-one build + auth
- hyperdx/hyperdx-local (--target=all-in-one-noauth): all-in-one build + no-auth

Production impacts:
- hyperdx/hyperdx: none
- hyperdx/hyperdx-all-in-one: new
- hyperdx/hyperdx-local: add server components (alerts, saved searches, dashboards)
2025-05-19 22:35:48 +00:00
Aaron Knudtson
43a9ca1901
Adopt clickhouse-js and POST method for all queries (#829)
Ref: HDX-1688

All modes for querying a clickhouse instance now go through clickhouse-js. This change is agnostic of the clickhouse server instance. This was tested in localmode tested against image tags
- 23-alpine
- 24-alpine
- 25.4-alpine

The docker command run for these was `docker run --ulimit nofile=262144:262144 -p 8123:8123 -p 9000:9000 --env CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 clickhouse/clickhouse-server:IMAGE_TAG_HERE`. Then I just ran `yarn dev` with a modified docker-compose.dev.yml where the `ports` section under `ch-server` is removed, and the otel-collector sends to `host.docker.internal:9000` instead of `ch-server:9000`.
2025-05-19 20:40:51 +00:00
Dan Hable
96b8c50898
fix(metrics): fix histogram metric query (#823)
Fix the query to address issues with the value calculation as well as allow for grouping.

Ref: HDX-1726
2025-05-19 14:47:24 +00:00
Mike Shi
8ecf44b790
feat: Set up default sources/connections with env var in server mode (#817)
Resolves HDX-1705
2025-05-16 23:17:37 +00:00
Warren
321e24f968
fix: alerting time range filtering bug (#814)
Ref: HDX-1701

1. fix alerting time range filtering
2. add time range info to the alert body

<img width="620" alt="image" src="https://github.com/user-attachments/assets/205d6537-e177-4be9-888f-a9328c8a2b8a" />
2025-05-16 17:40:32 +00:00
Aaron Knudtson
a825961fda
feat: rename team and rotate api key (#809)
Ref: HDX-1699
2025-05-16 17:37:15 +00:00
Warren
daf1b63298
style: reorder deps (package.json) (#806) 2025-05-13 00:19:38 +00:00
Tom Alexander
3ee0bdbf88
fix: Make dynamic import for swagger stuff (#804)
Remove swagger as a main dependency and make it a devDependency. 

Dynamically import the swagger utils only when needed.
2025-05-09 19:06:50 +00:00
github-actions[bot]
123e96e701
Release HyperDX (beta) (#802)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
2025-05-07 15:38:06 -07:00
Mike Shi
931d7387d9
fix: bugs with showing non otel spans (ex. clickhouse opentelemetry span logs) (#789)
<img width="1645" alt="image" src="https://github.com/user-attachments/assets/2f7eb93f-9648-4c98-8bfd-a2d0f65be9d5" />

fixing a few bugs that prevented us from properly rendering trace view for `system.opentelemetry_span_log`

fix HDX-1676

Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
2025-05-07 16:08:21 +00:00
Aaron Knudtson
1674ab8672
Move swagger to prod dependency (#801)
Noticed that images can't run because swagger was listed as a dev dependency
2025-05-07 16:05:15 +00:00
github-actions[bot]
cb52a48b5b
chore(release): bump HyperDX app/package versions (beta) (#771)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`v2` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `v2`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @hyperdx/common-utils@0.2.0-beta.4

### Minor Changes

-   79fe30f: Queries depending on numeric aggregates now use the type's default value (e.g. 0) instead of null when dealing with non-numeric data.

### Patch Changes

-   cfdd523: feat: clickhouse queries are by default conducted through the clickhouse library via POST request. localMode still uses GET for CORS purposes
-   92a4800: feat: move rrweb event fetching to the client instead of an api route
-   7f0b397: feat: queryChartConfig method + events chart ratio

## @hyperdx/api@2.0.0-beta.15

### Minor Changes

-   79fe30f: Queries depending on numeric aggregates now use the type's default value (e.g. 0) instead of null when dealing with non-numeric data.

### Patch Changes

-   9a9581b: Adds external API for alerts and dashboards
-   293a2af: Adds openapidoc annotations for spec generation and swagger route for development
-   92a4800: feat: move rrweb event fetching to the client instead of an api route
-   7f0b397: feat: queryChartConfig method + events chart ratio
-   b4b5f6b: style: remove unused routes/components + clickhouse utils (api)
-   Updated dependencies [79fe30f]
-   Updated dependencies [cfdd523]
-   Updated dependencies [92a4800]
-   Updated dependencies [7f0b397]
    -   @hyperdx/common-utils@0.2.0-beta.4

## @hyperdx/app@2.0.0-beta.15

### Patch Changes

-   7de8916: Removes trailing slash for connection urls
-   cfdd523: feat: clickhouse queries are by default conducted through the clickhouse library via POST request. localMode still uses GET for CORS purposes
-   6dc6989: feat: Automatically use last used source when loading search page
-   92a4800: feat: move rrweb event fetching to the client instead of an api route
-   7f0b397: feat: queryChartConfig method + events chart ratio
-   b4b5f6b: style: remove unused routes/components + clickhouse utils (api)
-   Updated dependencies [79fe30f]
-   Updated dependencies [cfdd523]
-   Updated dependencies [92a4800]
-   Updated dependencies [7f0b397]
    -   @hyperdx/common-utils@0.2.0-beta.4


Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
2025-05-07 05:34:27 +00:00
Mike Shi
15bb9e6c7e
chore: remove unused dependencies (#791)
- removes redis from stack (using in memory rate limiter)
- deletes unused miner code (moved to browser-based pyodide module)
- fixes rate limit bug

fixes HDX-1248
2025-05-06 03:13:20 +00:00
Dan Hable
79fe30f503
fix: aggFn use default instead of null (#782)
Switched to the `-OrDefault` version of the float conversion when combined with the aggregate functions to prevent emitting null.

Ref: HDX-1379
2025-04-29 18:29:37 +00:00
Tom Alexander
293a2affc8
feat: Adds auto openapi doc generation + swagger UI for API (#779)
* Adds support for auto-openapi doc generation
* Adds a swagger route for local development
* Adds a script to manually generate the openapi doc (to be used later if we publish to our site, etc...)

Ref: HDX-1661
2025-04-29 17:54:34 +00:00
Tom Alexander
9a9581b658
feat: Add dashboards and alerts external routes (#778)
* Adds routes and tests for alerts and dashboards
* Adds tests for v2 `/`

Ref: HDX-1649
Ref: HDX-1650
2025-04-29 17:09:32 +00:00
Aaron Knudtson
5b48f044a1
fix: remove authorization header from frontend (#781) 2025-04-28 11:57:29 +02:00
Aaron Knudtson
92a48003a4
feat: move rrweb event stream to client and query through /api/clickhouse-proxy (#755) 2025-04-22 18:01:09 -04:00
Warren
b4b5f6ba9b
style: remove unused routes/components + clickhouse utils (api) (#772)
1. deprecate `metrics`, `logs` and `chart` routes
2. deprecate `logStreamTableVersion` in team model
3. cleanup api clickhouse index file
4. deprecate propertyTypeMappingModel + searchQueryParser on the api side
5. deprecate `SearchInput`, `MetricTagFilterInput` and `MetricTagValueSelect` components

Ref: HDX-1647

Cut off 15% of codes 🎉
2025-04-22 17:19:25 +00:00
Warren
7f0b397969
feat: queryChartConfig method + events chart ratio (#759)
Ref: HDX-1631

1. use temp centralized `queryChartConfig` to handle multi-series chart (metrics specifically)
2. move ratio computation logics (events chart) to the renderChartConfig
3. fix missing `seriesReturnType` prop in chartConfig in the checkAlert file
2025-04-21 22:52:55 +00:00
github-actions[bot]
4172743ee9
chore(release): bump HyperDX app/package versions (beta) (#743)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`v2` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `v2`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @hyperdx/common-utils@0.2.0-beta.3

### Patch Changes

-   092a292: fix: autocomplete for key-values complete for v2 lucene
-   2f626e1: fix: metric name filtering for some metadata
-   b16c8e1: feat: compute charts ratio
-   4865ce7: Fixes the histogram query to perform quantile calculation across all data points

## @hyperdx/api@2.0.0-beta.14

### Patch Changes

-   e5dfefb: Added test cases for the webhook and source routes.
-   f5e9a07: chore: bump node version to v22
-   Updated dependencies [092a292]
-   Updated dependencies [2f626e1]
-   Updated dependencies [b16c8e1]
-   Updated dependencies [4865ce7]
    -   @hyperdx/common-utils@0.2.0-beta.3

## @hyperdx/app@2.0.0-beta.14

### Patch Changes

-   56e39dc: 36c3edc fix: remove several source change forms throughout the log drawer
-   092a292: fix: autocomplete for key-values complete for v2 lucene
-   2f626e1: fix: metric name filtering for some metadata
-   f5e9a07: chore: bump node version to v22
-   b16c8e1: feat: compute charts ratio
-   08009ac: feat: add saved filters for searches
-   db761ba: fix: remove originalWhere tag from view. not used anyways
-   8c95b9e: Add search history
-   Updated dependencies [092a292]
-   Updated dependencies [2f626e1]
-   Updated dependencies [b16c8e1]
-   Updated dependencies [4865ce7]
    -   @hyperdx/common-utils@0.2.0-beta.3


Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
2025-04-21 21:13:20 +00:00
Dan Hable
e5dfefb746
chore: adding test cases for additional routes (#762)
Ref: HDX-1619
2025-04-21 20:07:12 +00:00
Warren
f5e9a07ef1
chore: bump node version to v22 (#744) 2025-04-15 16:56:56 +00:00
Dan Hable
4865ce7a62
fix: fix histogram metric query (#737)
Fix for the histogram query based on the late night session at KubeCon.

Ref: HDX-1572
2025-04-14 22:04:13 +00:00
github-actions[bot]
a8f18bae2e
chore(release): bump HyperDX app/package versions (beta) (#684)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`v2` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `v2`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @hyperdx/common-utils@0.2.0-beta.2

### Minor Changes

-   a9dfa14: Added support to CTE rendering where you can now specify a CTE using a full chart config object instance. This CTE capability is then used to avoid the URI too long error for delta event queries.
-   e002c2f: Support querying a sum metric as a value instead of a rate

### Patch Changes

-   50ce38f: Histogram metric query test cases
-   2e350e2: feat: implement logs > metrics correlation flow + introduce convertV1ChartConfigToV2
-   a6fd5e3: feat: introduce k8s preset dashboard
-   b9f7d32: Refactored renderWith to simplify logic and ship more tests with the changes.
-   eaa6bfa: fix: transform partition_key to be the same format as others
-   bd9dc18: perf: reuse existing queries promises to avoid duplicate requests
-   5db2767: Fixed CI linting and UI release task.
-   414ff92: feat: export 'Connection' type
-   e884d85: fix: metrics > logs correlation flow
-   e5a210a: feat: support search on multi implicit fields (BETA)

## @hyperdx/app@2.0.0-beta.13

### Minor Changes

-   9579251: Stores the collapse vs expand status of the side navigation in local storage so it's carried across browser windows/sessions.

### Patch Changes

-   3be7f4d: fix: input does not overlap with language select button anymore
-   2e350e2: feat: implement logs > metrics correlation flow + introduce convertV1ChartConfigToV2
-   a6fd5e3: feat: introduce k8s preset dashboard
-   a9dfa14: Added support to CTE rendering where you can now specify a CTE using a full chart config object instance. This CTE capability is then used to avoid the URI too long error for delta event queries.
-   5a10ae1: fix: delete huge z-value for tooltip
-   6864836: fix: don't show ellipses on search when query is in-flight
-   b99236d: fix: autocomplete options for dashboard page
-   5db2767: Fixed CI linting and UI release task.
-   2580ddd: chore: bump next to v13.5.10
-   5044083: Session Replay tab for traces is disabled unless the source is configured with a sessionId
-   6dafb87: fix: View Events not shown for multiple series; grabs where clause when single series
-   decd622: fix: k8s dashboard uptime metrics + warning k8s event body
-   e884d85: fix: metrics > logs correlation flow
-   e5a210a: feat: support search on multi implicit fields (BETA)
-   Updated dependencies [50ce38f]
-   Updated dependencies [2e350e2]
-   Updated dependencies [a6fd5e3]
-   Updated dependencies [a9dfa14]
-   Updated dependencies [e002c2f]
-   Updated dependencies [b9f7d32]
-   Updated dependencies [eaa6bfa]
-   Updated dependencies [bd9dc18]
-   Updated dependencies [5db2767]
-   Updated dependencies [414ff92]
-   Updated dependencies [e884d85]
-   Updated dependencies [e5a210a]
    -   @hyperdx/common-utils@0.2.0-beta.2

## @hyperdx/api@2.0.0-beta.13

### Patch Changes

-   50ce38f: Histogram metric query test cases
-   2e350e2: feat: implement logs > metrics correlation flow + introduce convertV1ChartConfigToV2
-   b9f7d32: Refactored renderWith to simplify logic and ship more tests with the changes.
-   5db2767: Fixed CI linting and UI release task.
-   d326610: feat: introduce RUN_SCHEDULED_TASKS_EXTERNALLY + enable in-app task
-   414ff92: perf + fix: single clickhouse proxy middleware instance
-   Updated dependencies [50ce38f]
-   Updated dependencies [2e350e2]
-   Updated dependencies [a6fd5e3]
-   Updated dependencies [a9dfa14]
-   Updated dependencies [e002c2f]
-   Updated dependencies [b9f7d32]
-   Updated dependencies [eaa6bfa]
-   Updated dependencies [bd9dc18]
-   Updated dependencies [5db2767]
-   Updated dependencies [414ff92]
-   Updated dependencies [e884d85]
-   Updated dependencies [e5a210a]
    -   @hyperdx/common-utils@0.2.0-beta.2


Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
2025-04-02 02:33:29 +00:00
Warren
d32661032b
feat: introduce RUN_SCHEDULED_TASKS_EXTERNALLY (#733)
1. new env var RUN_SCHEDULED_TASKS_EXTERNALLY to opt out in-app task process
2. introduce new `prod-extended` build that includes mongodb process
3. GA k8s dashboard (only picking the connection)
4. bake check-alert task into fullstack app build
2025-04-02 02:21:49 +00:00
Dan Hable
e002c2f9c6
feat: query sum metric without rate logic (#717)
Add the ability to query a sum metric and obtain the underlying values instead of the rate of change between those points.

Ref: HDX-1543
2025-03-27 19:09:58 +00:00
Dan Hable
50ce38f1a9
test: histogram metric query integration tests (#692)
Pulls a set of test cases from the v1 code base that checks histogram metric queries against different quantile and queries for edge bounds as well.

Ref: HDX-1425
2025-03-25 18:59:15 +00:00
Warren
2e350e22ba
feat: implement logs > metrics correlation flow (#709)
1. users now should be able to go from k8s logs to metrics through 'Infrastructure' tab within details panel
2. introduce `convertV1ChartConfigToV2` to convert V1 multi-series chart config to V2 (only metrics)

Ref: HDX-1333, HDX-1536

<img width="1354" alt="image" src="https://github.com/user-attachments/assets/c7f893b3-794c-4e23-9aa0-9ab37532a57f" />
2025-03-25 18:40:16 +00:00
Warren
414ff922c0
perf + fix: single clickhouse proxy middleware instance (#687)
Ref: HDX-1423

create clickhouse proxy middleware once to prevent memory leaks issue
2025-03-18 18:07:12 +00:00
Dan Hable
b9f7d32efa
refactor: clean up the chart config CTE render logic (#686)
Some additional refactoring and testing around the more complex CTE rendering.

Ref: HDX-1511
2025-03-17 14:45:26 +00:00
Mike Huang
bbdc53457a
Fix: slack alert empty body (#685)
- fix an issue where select query does not work correctly

ref: hdx-1476
2025-03-14 20:11:03 +00:00
Dan Hable
5db2767015
ci: fixes linting and release-local-ui target (#683)
1. Created separate tsconfig files for tests and prod code, including the test files in the test config.

2. Fixed linting error in DBEditTimeChartForm found while running local linting.

3. Excluded coverage CSS from the style lint task.

4. Updated app Dockerfile with test configuration and ran locally to ensure it was working.

Ref: HDX-1502
2025-03-13 23:11:46 +00:00
github-actions[bot]
184402dccf
chore(release): bump HyperDX app/package versions (beta) (#609)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Warren <warren@Warrens-MacBook-Air.local>
2025-03-13 03:32:40 -07:00
Warren
4492daa5b9
fix: gauge metric attribute conflicts issue (#670)
Ref: HDX-1468
2025-03-12 02:16:15 +00:00
Dan Hable
99b60d50b2
fix: update sum metric query based on v1 integration test (#650)
Fix the sum query to produce the correct results from the min/max test case from v1.

Ref: HDX-1421
2025-03-07 07:03:03 +00:00
Warren
7f1a5a0b09
fix: metric alerts handling (#655) 2025-03-06 00:11:17 +00:00
Warren
cd0e4fd71c
fix: correct handling of gauge metrics in renderChartConfig (#654) 2025-03-06 00:06:57 +00:00
Warren
4310a5621a
fix: include common-utils node_modules (api dockerfile) (#646)
- perf: copy root node_modules from builder stage to cut the deps size by half 
- fix: include `common-utils` node_modules for conflicts pkg like '@clickhouse/client'
2025-03-03 03:37:03 +00:00
Warren
2b2e31f206
feat: include common-utils build (app/api Dockerfile) (#645) 2025-03-02 05:30:02 +00:00
Warren
8720ee5c67
ci: port v1 metric tests - Pt1 (#638) 2025-02-27 08:07:47 +00:00
Mike Huang
c389b69dc9
fix: alert for generic webhook (#619)
- Alert will use correct body for generic webhook now
- Add webhook body UI for generic webhook
- Body input will lint check if it is correct JSON 
- Default body will be `{"text": "{{title}} | {{body}} | {{link}}"}`, failed to parse webhook body will use default body
- Add test for Bad webhook body case

<img width="771" alt="image" src="https://github.com/user-attachments/assets/a130db90-2985-42d8-bdb5-968043dde073" />


<img width="1490" alt="image" src="https://github.com/user-attachments/assets/0a798771-ff48-4480-900c-9488ec675bef" />


ref: hdx-1450
2025-02-25 04:36:09 +00:00
Warren
1e5d97c131
fix+perf: filter rrweb events by service name and sort by timestamp (#628)
1. Filter rrweb events by `ServiceName` (primary key)
2. Sort events by timestamp ASC
3. Add 4 hours offset to the original time range
2025-02-24 07:25:39 +00:00
Dan Hable
759da7a283
feat(metrics): Support multiple OTEL tables (#610) 2025-02-12 11:04:50 -08:00
github-actions[bot]
8b5a8182d3
chore(release): bump HyperDX app/package versions (beta) (#607)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`v2` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `v2`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @hyperdx/api@2.0.0-beta.10

### Patch Changes

-   adc2a0b: fix: Ensure errors from proxy are shown to the user

## @hyperdx/app@2.0.0-beta.10

### Patch Changes

-   459267a: feat: introduce session table model form
2025-02-11 21:01:11 +00:00
Tom Alexander
adc2a0bc26
fix: Ensure errors from proxy are shown to the user (#601)
1. Ensures errors from the ch proxy are properly returned 
2. Adds a connection check hook to periodically check connections and display a toast if connections are bad
3. Fixes the ConnectionForm to ensure that toast notifications are shown when test connections fail or when there's an error saving the form.
2025-02-11 20:44:43 +00:00