mirror of
https://github.com/datahaven-xyz/datahaven
synced 2026-05-24 09:50:01 +00:00
1997c298a1
## Changes - New CI file for making Docker Prod images - Changed E2E tests use an image built from a local dockerfile - Some cargo build options to make it quicker - Fix the cache hit rate - added `tsgo` preview to the project 😎 - Can be invoked with `bun tsgo` to typecheck - Install in IDE [VSCode](https://code.visualstudio.com/docs/configure/extensions/extension-marketplace) & [Zed](https://github.com/zed-extensions/tsgo) for super-fast inline typechecking (as you type basically) ## Context This PR attempts to make the frankly unacceptable CI times better. This achieves that aim by making a crappy image for day-to-day usage and let the prod issue take ages since that will be infrequently used. The reason why the original design didn't work for us is because: 1) we are using the free GH runners 2) when we goto baremetal runners we'll lose our rapid caching abilities which make using docker cheap. Also, we add `tsgo` support to improve devex. The improvement is astounding. ```sh hyperfine -n tsc "bun tsc --incremental false --extendedDiagnostics" -n tsgo "bun tsgo --incremental false --extendedDiagnostics" Benchmark 1: tsc Time (mean ± σ): 5.500 s ± 0.221 s [User: 8.939 s, System: 0.400 s] Range (min … max): 5.196 s … 5.845 s 10 runs Benchmark 2: tsgo Time (mean ± σ): 99.1 ms ± 8.4 ms [User: 392.8 ms, System: 54.1 ms] Range (min … max): 88.3 ms … 116.0 ms 29 runs Summary tsgo ran 55.48 ± 5.22 times faster than tsc ```
91 lines
2.6 KiB
YAML
91 lines
2.6 KiB
YAML
# Rust Tests: CI for Rust components (DataHaven runtime and node Rust tests)
|
|
#
|
|
# Overview:
|
|
# 1. Prepare: This job handles the setup phase where the cargo nextest archive is created
|
|
# and uploaded to the workflow for use in the subsequent jobs
|
|
# 2. All Rust Tests: Executes the full suite of Rust tests across two partitions to
|
|
# to reduce total execution time.
|
|
|
|
name: DataHave Operator Rust Tests
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
workflow_call:
|
|
|
|
jobs:
|
|
|
|
prepare:
|
|
name: Prepare artifacts for Rust tests
|
|
runs-on: ubuntu-latest
|
|
env:
|
|
RUSTC_WRAPPER: "sccache"
|
|
CARGO_INCREMENTAL: "0"
|
|
CARGO_TERM_COLOR: always
|
|
RUSTFLAGS: "-C linker=clang -C link-arg=-fuse-ld=mold"
|
|
SCCACHE_GHA_ENABLED: "true"
|
|
defaults:
|
|
run:
|
|
working-directory: ./operator
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
submodules: recursive
|
|
fetch-depth: 0
|
|
- uses: ./.github/workflows/actions/setup-env
|
|
with:
|
|
cache-key: "TEST"
|
|
install-deps: true
|
|
- uses: ./.github/workflows/actions/cleanup-runner
|
|
- name: Install nextest
|
|
uses: taiki-e/install-action@nextest
|
|
- name: Build and archive tests
|
|
run: cargo nextest archive --archive-file nextest-archive.tar.zst
|
|
- name: Upload archive to workflow
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: nextest-archive
|
|
path: ./operator/nextest-archive.tar.zst
|
|
|
|
all-rust-tests:
|
|
name: Run all Operator Rust tests (/w partitioning)
|
|
needs: [prepare]
|
|
runs-on: ubuntu-latest
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
partition: [1, 2]
|
|
defaults:
|
|
run:
|
|
working-directory: ./operator
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha }}
|
|
- name: Install nextest
|
|
uses: taiki-e/install-action@nextest
|
|
- name: Download archive
|
|
uses: actions/download-artifact@v4
|
|
with:
|
|
name: nextest-archive
|
|
- name: Run Tests for All Projects!
|
|
run: |
|
|
~/.cargo/bin/cargo-nextest nextest run \
|
|
--archive-file ../nextest-archive.tar.zst \
|
|
--partition count:${{ matrix.partition }}/2
|
|
|
|
tests-result-checker:
|
|
name: Check tests were successful
|
|
needs: [all-rust-tests]
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Validate test results
|
|
run: |
|
|
echo "matrix result: ${{ needs.all-rust-tests.result }}"
|
|
|
|
if [ "${{ needs.all-rust-tests.result }}" != "success" ]; then
|
|
echo "Rust tests failed or were cancelled"
|
|
exit 1
|
|
else
|
|
echo "Rust tests passed"
|
|
fi
|
|
|