Commit graph

84 commits

Author SHA1 Message Date
Steve Degosserie
51ffcae5f0
Revert "feat: statically build binary (#292)" (#330)
This reverts commit f84b6debb7.
2025-12-02 15:42:43 +01:00
Steve Degosserie
e38843455b
fix: 🔨 Disable static binary build for now (#328) 2025-12-02 14:53:05 +01:00
undercover-cactus
f84b6debb7
feat: statically build binary (#292)
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
2025-11-28 13:38:05 +00:00
Steve Degosserie
71b5e5185f
fix: consolidate session timing and simplify docker release workflow (#321)
## Summary

- Consolidates `SessionsPerEra` definition in common runtime (removes
duplicate definitions)
- Simplifies docker release workflow to always use full Docker builds
- Removes binary reuse path from release workflow

## Changes

### Runtime Configuration
- Remove duplicate `SessionsPerEra` definitions from individual runtimes
- Import `SessionsPerEra` from `datahaven_runtime_common::time` instead
- This fixes inconsistency where individual runtimes had
`prod_or_fast!(6, 1)` while common had `prod_or_fast!(6, 3)`

### Docker Release Workflow
- Remove binary reuse path - now always does full Docker build
- Remove `binary-hash` input from `workflow_call`
- Consolidate to single build step using `datahaven-build.Dockerfile`
- `docker-build-release` now runs in parallel on main branch (no
dependency on `build-operator`)

## Timing Configuration

### Production Runtime
| Parameter        | Value       | Duration   |
|------------------|-------------|------------|
| Session          | 600 blocks  | 1 hour     |
| Sessions per era | 6           | -          |
| Era              | 6 sessions  | 6 hours    |
| Bonding duration | 28 eras     | 7 days     |

### Fast Runtime (for testing)
| Parameter        | Value       | Duration   |
|------------------|-------------|------------|
| Session          | 10 blocks   | 1 minute   |
| Sessions per era | 1           | -          |
| Era              | 1 session   | 1 minute   |
| Bonding duration | 3 eras      | 3 minutes  |

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-26 10:25:24 +01:00
undercover-cactus
53d209bbae
test: only inject contracts in e2e tests if INJECT_CONTRACTS env is 'true' (#315)
In this PR we add an environment variable `INJECT_CONTRACTS`. This
environment variable specify if the contracts should be injected in the
e2e tests. By default it is false. The environment variable is set to
`true` in the CI job that run the e2e tests.

We are using a environment variable because `bun test` doesn't allow for
passing extra arguments.

A note about the new variable has been added in the documentation to
inform about the new behavior.

---------

Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
2025-11-24 12:07:36 +01:00
Steve Degosserie
ba1cc63cb0
fix: 🔨 Run publish binary task on ephemeral runner (#307)
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-11-22 15:54:49 +01:00
Steve Degosserie
37a4ba990f
fix: 🪳 Quick CI build fix (#300) 2025-11-15 12:25:15 +01:00
Ahmad Kaouk
dd7b72ca29
chore: pin Bun version and migrate to bun.lock (#290)
## Summary

Pins Bun version to 1.3.2 and migrates workflows to use text-based
`bun.lock` instead of binary `bun.lockb`. This fixes CI failures caused
by Bun version mismatches between local development and GitHub Actions.

## Changes

- Created `test/.bun-version` to pin Bun to v1.3.2
- Updated all workflows to use `bun-version-file: test/.bun-version`
- Migrated workflow cache keys from `bun.lockb` to `bun.lock`
- Removed deprecated `test/bun.lockb` binary lockfile

## Why?

**Version Consistency:**
- Local environments and CI were using different Bun versions
- Different versions generate different lockfile formats → CI failures

**Lockfile Migration:**
- Bun v1.2+ uses text-based `bun.lock` as default
- Binary `bun.lockb` is still supported but deprioritized
- Text format provides better git diffs and merge conflict resolution

## Affected Workflows

- `.github/workflows/task-check-metadata.yml`
- `.github/workflows/task-e2e.yml`
- `.github/workflows/task-moonwall-tests.yml`
- `.github/workflows/task-ts-build.yml`
- `.github/workflows/task-ts-lint.yml`

## After Merge

Developers should upgrade their local Bun:
```bash
bun upgrade --stable  # Should install v1.3.2
bun --version         # Verify version
bun install           # Regenerate lockfile if needed
```

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-10 22:37:39 +01:00
Steve Degosserie
8fa8c18dfd
CI: ♻️ Trigger CI actions on perm-* branches (#284) 2025-11-07 13:25:53 +01:00
Steve Degosserie
45cc9101ea
chore: 🛡️ Add CODEOWNERS file (#281) 2025-11-06 22:04:44 +02:00
Ahmad Kaouk
470f5fc916
feat: update eigenlayer contracts to v1.8.0 (#270)
## Summary
- sync `contracts/lib/eigenlayer-contracts` to tag
`v1.8.0-testnet-final` and refresh `EIGENLAYER.md` with the new commit
reference
- update local/test deployment flows to deploy the upstream
`EigenStrategy`, feed it into `AllocationManager`/`StrategyManager`, and
adopt the revised `EigenPod` constructor
- drop the obsolete `AllocationManagerMock` stub and replace its usage
with targeted `vm.mockCall` stubs that return `slashOperator` share data
- adjust slasher unit tests to match the new ABI so DataHaven stays
aligned with EigenLayer 1.8 semantics

## Testing
- forge build
- forge test
2025-11-04 16:30:18 +01:00
Steve Degosserie
10a7805648
feat: Add CI license check (#269)
## Summary

- Adds automated license compliance checking via GitHub Actions CI
workflow
- Implements a license verification script that validates all Rust
dependencies against approved licenses, authors, and packages
- Standardizes author metadata across Cargo manifests to "Moonsong Labs"

## Changes

**CI Workflow** (`.github/workflows/task-check-licenses.yml`)
- Triggers on pull requests and manual dispatch
- Installs Rust 1.88.0 toolchain and `cargo-license` tool
- Executes license verification script to enforce compliance

**License Verification Script** (`operator/scripts/verify-licenses.sh`)
- Uses `cargo-license` to extract dependency license information
- Maintains three allowlists:
- **Licenses**: Apache-2.0, MIT, BSD variants, GPL-3.0, MPL-2.0, and
compatible combinations
- **Authors**: PureStake, Parity Technologies, Moonsong Labs, Frontier
developers, StorageHub Team
  - **Package Names**: Known safe packages like ring
- Fails the build if any dependency has unapproved license/author/name
combination

**Cargo Manifest Updates**
- `operator/Cargo.toml`: Standardized workspace author to "Moonsong
Labs"
- `operator/precompiles/precompile-registry/Cargo.toml`: Uses workspace
author field
- `operator/runtime/common/Cargo.toml`: Added workspace author field

## Benefits

- **Legal Compliance**: Ensures all dependencies use OSI-approved or
compatible licenses
- **Supply Chain Security**: Validates dependencies come from trusted
sources
- **Automated Enforcement**: Catches licensing issues during PR review
rather than at release time
- **Transparency**: Provides clear audit trail of approved licenses and
authors
2025-11-02 23:32:59 +02:00
Steve Degosserie
62a4a1fb60
fix: 🔧 Fix e2e test workflow (#260) 2025-10-28 17:43:29 +01:00
Steve Degosserie
b5bc2de11e
fix: 🔧 Fix incorrect args in release Docker image publishing workflow (#256) 2025-10-27 15:13:14 +02:00
Ahmad Kaouk
48f8add3c4
ci: fix fetch submodule (#248)
This PR fixes the E2E checkout failure by fetching full history instead
of a depth-1 clone so the Snowbridge forge-std submodule can resolve its
pinned commit.
2025-10-24 13:31:17 +03:00
undercover-cactus
4eca467514
ci: pin forge version when installing it (#243)
In this PR, we pin the forge version use in the linter task. When forge
make a new release it brokes the linter task.

In the future we can update  the forge version explicitly.

---------

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-10-23 14:33:27 +00:00
undercover-cactus
908a2a7ed5
ci: remove unused volumes after e2e tests and remove logs collection (#246)
In this PR, we fix the ci error indicating we reached the number of
volumes allow by deleting automatically after tests the volumes.

We also remove the step that collect logs because the container that are
interesting to us to debug are being removed entirely. Therefore the
logs from the nodes are not being collected in this step.
2025-10-23 14:02:36 +00:00
undercover-cactus
eced179b09
misc: simplify Dockerfile to speed up build (#216)
This PR remove the `cargo chef` step used to build the docker image used
in deployment. We noticed that `cargo chef` was adding more time to the
build and that removing it was saving us 40min.

Also in this PR, we removed the base image from parity which was really
heavy and was filling the rest of the disk space. This broke the build.
After some investigation it doesn't seem to add a lot to the build. It
has been replace with the official rust image as a base to build our
node.

The image used to run the image has been replaced with
`debian:trixie-slim`.

In the end those changed **should not** break any of the current
behavior and makes save a bit of CI time.
2025-10-22 13:36:30 +02:00
Steve Degosserie
72cac823af
fix: 🔧 Fix invalid condition on workflow_call in Docker release task (#238) 2025-10-15 21:33:54 +02:00
Steve Degosserie
d202869438
fix: 🔧 Fix Docker release extract tag logic (#237) 2025-10-15 19:50:59 +02:00
Steve Degosserie
ff694b0055
fix: 🔧 Fix Docker release extract tag logic (#236) 2025-10-15 18:52:27 +02:00
Steve Degosserie
8be3c0f979
fix: 🔧 Fix Docker image tags in Docker release task (#235)
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-15 18:30:37 +02:00
Steve Degosserie
dda9111ee6
fix: 🔧 Copy all shared libraries required by the DataHaven node in build & production images (#234) 2025-10-15 14:46:07 +02:00
Steve Degosserie
9a5404de82
refactor: Consolidate and optimize Docker image architecture (#233)
## Overview

This PR consolidates and optimizes the Docker build system, reducing
redundancy and improving CI/CD performance. The changes eliminate
duplicate Dockerfiles, introduce a flexible build template, and optimize
release builds to reuse CI artifacts.

## Changes Summary

### 🐳 Docker Images Restructured

**Before:** 5 Dockerfiles with significant overlap
**After:** 4 focused images + 1 utility

#### Final Structure:

1. **`operator/Dockerfile`**  Updated
   - **Standard operator image** for CI and release builds
   - Minimal node image (accepts pre-built binaries)
   - GHCR: `ghcr.io/datahaven-xyz/datahaven/datahaven` (CI)
   - DockerHub: `datahavenxyz/datahaven` (releases)

2. **`docker/datahaven-build.Dockerfile`** (moved from
`operator/Dockerfile`)
   - Full source-to-binary build for manual releases
   - DockerHub: `datahavenxyz/datahaven:{label}`
   - Supports custom RUSTFLAGS and fast-runtime feature
   - Only used for manual workflow_dispatch builds

3. **`docker/datahaven-production.Dockerfile`** (kept)
   - Binary builder for CPU-specific releases
   - Used by build-prod-binary workflow template
   - Supports custom target-cpu flags

4. **`docker/datahaven-dev.Dockerfile`**  NEW (local dev only)
   - **FOR LOCAL DEVELOPMENT/TROUBLESHOOTING ONLY**
   - Includes debug tools: gdb, strace, vim, sudo
   - Extra dependencies: librocksdb-dev, curl
   - RUST_BACKTRACE enabled by default
   - **DO NOT USE for CI or production builds**

5. **`test/docker/crossbuild-mac-libpq.dockerfile`** (kept)
   - Utility for macOS → Linux cross-compilation

#### Removed (Redundant):
-  `docker/datahaven.Dockerfile` → replaced by operator/Dockerfile
-  `test/docker/datahaven-node-local.dockerfile` → replaced by
datahaven-dev.Dockerfile

---

### 🔄 Workflow Improvements

#### Enhanced `publish-docker` Template
- Supports both GHCR and DockerHub registries
- Flexible inputs: dockerfile, context, build-args, cache scope
- Auto-generates OCI-compliant labels
- Reduces code duplication (~70 lines → ~15 per workflow)

#### Refactored CI Pipeline
- **`docker-build-ci`**: Builds `operator/Dockerfile` → GHCR for CI/E2E
testing
- **`docker-build-release`**: Builds `operator/Dockerfile` → DockerHub
(main branch only)
- Both CI and release workflows now use the same minimal operator image
- Release builds **reuse CI binaries** instead of rebuilding from source

#### Optimized Release Workflow
The `task-docker-release` workflow now has dual modes:

**Mode 1: `workflow_call` (CI - main pushes)**
-  Reuses binary from CI's build-operator task
-  Uses lightweight `operator/Dockerfile`
-  Tags: `latest`, `sha-{short}`
-  **Fast**: ~5 minutes (vs ~30 min previously)

**Mode 2: `workflow_dispatch` (Manual)**
-  Full source build with `datahaven-build.Dockerfile`
-  Custom branch and label selection
-  Optional fast-runtime feature
-  Tags: `PROD-{label}` or user-defined

---

### 🔧 Additional Optimizations

- Copy libpq5 from builder stage instead of reinstalling (smaller,
faster)
- Remove redundant protobuf-compiler package (use protoc v21.12
directly)
- Standardize user UID to 1000 across all runtime images
- Consistent OCI labeling and metadata

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-15 01:33:20 +02:00
Steve Degosserie
750e8f391c
fix: 🔧 Fix Docker production image (#230) 2025-10-13 17:53:34 +02:00
Steve Degosserie
678a8fb161
fix: 🔧 Use standard Github runners for the publish runtime task (#225) 2025-10-11 10:37:19 +02:00
Steve Degosserie
8c950af4a4
fix: 🔧 Add Podman support to srtool runtime build script (#222)
## Summary

- Adds support for both Docker and Podman container engines in
`build-runtime-srtool.sh` via `IS_PODMAN` environment variable
- Uses `--userns=keep-id` for Podman (proper user namespace handling)
and `--user $(id -u):$(id -g)` for Docker
- Sets `IS_PODMAN=true` in `task-publish-runtime.yml` workflow to enable
Podman by default

## Changes

**`operator/scripts/build-runtime-srtool.sh`:**
- Added conditional logic to detect `IS_PODMAN` env var
- Dynamically selects between `podman` and `docker` as container engine
- Sets appropriate user/namespace flags based on container engine

**`.github/workflows/task-publish-runtime.yml`:**
- Added `IS_PODMAN: true` environment variable to the srtool build step
- Updated comment to use generic "container user" instead of "docker
user"

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-10 22:57:34 +02:00
undercover-cactus
514a16ac1f
ci: remove sccache from image build for prod (#200)
In this PR, we remove the caching of the sccache folder because it is
too big (~3GB) and fill our cache too fast.

What to expect ?  
* It will make the build a bit slower but it is fine because it only
build on `main`. We are preparing another PR that will speed up the
build of the prod image. Also we are not sure the cache is actually
being used (`gha` cache is in beta).
* Will free some space for caching and stop deleting our cache which
make other jobs work faster.

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
2025-10-09 12:33:35 +00:00
Steve Degosserie
72bfe9bb62
fix: 🔧 Add revision number to rust toolchain channel version to match with srtool image version (#209) 2025-10-07 11:34:21 +02:00
Steve Degosserie
0110a94978
fix: 🔧 Fix invalid runs-on label in Publish runtime task (#207) 2025-10-06 15:38:29 +02:00
Ahmad Kaouk
17c706dc64
test: Integrate moonwall (#185)
### Description

This PR introduces the **Moonwall** end-to-end (E2E) testing framework.
The primary motivation for this is to enable the porting of existing
Mobeam tests into the `DataHaven` repository.

### Key Changes

*   **Node Manual Sealing:**
* Introduced a `--sealing=manual` flag for the `datahaven-node`. When
enabled, blocks are only produced on demand via an RPC call. This is the
core mechanism that allows for deterministic tests.

*   **Moonwall Framework Integration:**
* Added `@moonwall/cli` and `@moonwall/util` dependencies to the
`test/package.json`.
* A new `test/moonwall.config.json` file configures the test
environment, defining how Moonwall should launch the `datahaven-node`
with the manual sealing flag.
* Added a `moonwall:test` script to `package.json` for running the
tests.

*   **CI Workflow:**
* A new reusable workflow, `.github/workflows/task-moonwall-tests.yml`,
has been created to handle the setup, execution, and reporting of
Moonwall tests.
* The main `CI.yml` now includes a `moonwall-tests` job that runs after
the `build-operator` job, ensuring it always tests the correct,
freshly-built binary.

*   **Example Test Suite:**
* A new test suite, `test/datahaven/suites/dev/test-block.ts`, had been
copied from moonbeam.

### How to Run Locally

1.  Navigate to the `test` directory.
2.  Install dependencies: `bun install`
3.  Run the tests: `bun run moonwall:test`

---------

Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
2025-09-30 14:47:39 +00:00
Steve Degosserie
066a416349
feat: Publish runtime GitHub action (#198) 2025-09-30 15:24:35 +02:00
Steve Degosserie
a62319961c
feat: Publish runtime GitHub action (#197) 2025-09-30 15:11:54 +02:00
Steve Degosserie
51f4e1b0f3
feat: Publish runtime release Github action (#196) 2025-09-30 14:53:15 +02:00
undercover-cactus
e5e3fbe22c
fix: fix RPC for storage hub (#190)
In this PR we actually add the storage hub RPC config to our node in
order to allow RPC call for storage hub service.
2025-09-27 19:03:32 +02:00
Steve Degosserie
af61ef685c
fix: Fix Use Publish Draft Binary gh action (#184) 2025-09-23 01:02:20 +02:00
Steve Degosserie
696a716ff8
fix: Use DH self-hosted runners for the Publish Draft Binary gh action (#183) 2025-09-23 00:05:06 +02:00
Steve Degosserie
34488aab25
fix: Use DH self-hosted runners for the Publish Draft Binary gh action (#182) 2025-09-22 22:06:39 +02:00
Steve Degosserie
72f8963fdf
fix: 🔧 Temporarily disable Skylake & Zenv3 binaries on client release (#179)
Temporarily disable Skylake & Zenv3 binaries when publishing a client
release
2025-09-18 16:14:38 +02:00
Steve Degosserie
db5e86c703
fix: Fix publish binary gh action (#178) 2025-09-18 08:12:07 +02:00
Steve Degosserie
5c2fe2a7a1
fix: Fix publish binary gh action (#177) 2025-09-18 08:04:54 +02:00
Steve Degosserie
5b55659bba
fix: Fix publish binary gh action (#176) 2025-09-18 07:46:33 +02:00
Steve Degosserie
a2b385889f
fix: Fix publish binary gh action (#175) 2025-09-18 07:39:04 +02:00
Steve Degosserie
cb877be46b
fix: Fix publish draft binary gh action (#174) 2025-09-18 01:21:49 +02:00
Steve Degosserie
ec200fdcc3
feat: Implement gh workflow to publish a draft client release (#172) 2025-09-18 00:43:47 +02:00
Steve Degosserie
89cff99da0
ci: Enforce PR labels for breaking changes and auditability (#169)
## Summary
- Add GitHub Action workflow to enforce required labels on pull requests
- Ensure PRs are properly categorized for breaking changes and audit
requirements
- Validate breaking change documentation when breaking label is present

## Details

This PR introduces a new GitHub Actions workflow
(`enforce-pr-labels.yml`) that runs on all pull request events to
enforce proper labeling standards:

### Label Requirements

1. **Noteworthiness labels** (one required):
   - `B0-silent` - No client/runtime impact
   - `B5-clientnoteworthy` - Client-facing changes
   - `B7-runtimenoteworthy` - Runtime changes

2. **Breaking change labels** (required when noteworthy):
   - `breaking` - Contains breaking changes
   - `not-breaking` - No breaking changes
   
3. **Auditability labels** (one required):
   - `D1-audited👍` - Already audited
   - `D5-nicetohaveaudit⚠️` - Audit would be beneficial
   - `D9-needsaudit👮` - Requires audit
   - `D2-notlive` - Not for production
   - `D3-trivial` - Trivial change

### Validation

When a PR is marked with the `breaking` label, the workflow validates
that the PR description contains a "## ⚠️ Breaking Changes ⚠️" section
to ensure breaking changes are properly documented.

🤖 Generated with [Claude Code](https://claude.ai/code)
2025-09-17 14:21:33 +02:00
Ahmad Kaouk
82bafc2f1e
ci: fix snowbridge relay volume leak (#168)
## Summary

Each E2E run pulls `datahavenxyz/snowbridge-relay:latest` via `docker
create`. Because the image declares an anonymous `VOLUME`, Docker
allocates a new named volume for every run. We were removing the temp
container without `-v`, so the volume stayed on disk. After ~2,048 runs,
Docker refused to allocate new locks and aborted with exit code 125:

<img width="1425" height="86" alt="2025-09-16_18-01-06"
src="https://github.com/user-attachments/assets/ca05ac54-512d-4fa9-871c-e0b259071019"
/>

## Fix
- update `.github/workflows/task-e2e.yml` to use `docker rm -fv temp`,
ensuring the anonymous volume is removed when the temp container is
deleted

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-09-16 22:18:52 +00:00
undercover-cactus
60d0e2c901
ci: remove nextest archive (#164)
Removing the nextest archive steps from the CI because it is taking a
lot of time to download.

From latest ci runs downloading take 17min.

---------

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-09-16 13:44:16 +02:00
Steve Degosserie
ca66df8d3e
fix: Restore runner cleanup step (#163) 2025-09-12 15:12:14 +02:00
Steve Degosserie
9c961a2bf7
fix: Fix Production Docker image build (#158)
Self-hosted runners w/ Podman don't support Docker buildx.
2025-09-11 12:30:57 +02:00