Commit graph

145 commits

Author SHA1 Message Date
Gonza Montiel
c18c9dc364
feat: add FreeHeadersInterval parameter to Ethereum client config (#279)
## Add FreeHeadersInterval parameter to Ethereum client config

Configure parameter `FreeHeadersInterval` set to `32` (1 epoch = 6.4
minutes) across `mainnet`, `stagenet`, and `testnet` configurations.

### Rationale
1. Aligns with Ethereum's epoch change, so it's easier to identify in
which epoch we are in
2. It's the value used in Snowbridge's test configuration

The value can be changed via pallet parameters.

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-06 09:31:00 +00:00
Steve Degosserie
c7d73af4ca
feat: Bump client version to v0.6.0 & runtime version to RT600 (#276) 2025-11-04 12:51:23 +02:00
Steve Degosserie
c09ff91a66
feat: Bump client version to v0.5.0 & runtime version to RT500 (#274) 2025-11-03 16:35:22 +02:00
Facundo Farall
1fd0abccd7
build: ⬆️ Upgrade to SH v0.1.1 (#273)
Upgrade to SH release
[v0.1.1](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.1.1)

---------

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-03 15:49:30 +02:00
Gonza Montiel
87eb1c1621
fix: add timestamp safe mode (#267)
Following up https://github.com/datahaven-xyz/datahaven/pull/265, we
also need to add Timestamp to the whitelisted Runtime calls.

- [x] Add `RuntimeCall::Timestamp` to `SafeModeWhitelistedCalls`
- [x] Add safe mode test to check it produces blocks

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-03 14:00:04 +02:00
undercover-cactus
e248a48385
feat: add Slashing mode has a runtime configurable parameter (#272)
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-03 11:55:31 +02:00
Steve Degosserie
10a7805648
feat: Add CI license check (#269)
## Summary

- Adds automated license compliance checking via GitHub Actions CI
workflow
- Implements a license verification script that validates all Rust
dependencies against approved licenses, authors, and packages
- Standardizes author metadata across Cargo manifests to "Moonsong Labs"

## Changes

**CI Workflow** (`.github/workflows/task-check-licenses.yml`)
- Triggers on pull requests and manual dispatch
- Installs Rust 1.88.0 toolchain and `cargo-license` tool
- Executes license verification script to enforce compliance

**License Verification Script** (`operator/scripts/verify-licenses.sh`)
- Uses `cargo-license` to extract dependency license information
- Maintains three allowlists:
- **Licenses**: Apache-2.0, MIT, BSD variants, GPL-3.0, MPL-2.0, and
compatible combinations
- **Authors**: PureStake, Parity Technologies, Moonsong Labs, Frontier
developers, StorageHub Team
  - **Package Names**: Known safe packages like ring
- Fails the build if any dependency has unapproved license/author/name
combination

**Cargo Manifest Updates**
- `operator/Cargo.toml`: Standardized workspace author to "Moonsong
Labs"
- `operator/precompiles/precompile-registry/Cargo.toml`: Uses workspace
author field
- `operator/runtime/common/Cargo.toml`: Added workspace author field

## Benefits

- **Legal Compliance**: Ensures all dependencies use OSI-approved or
compatible licenses
- **Supply Chain Security**: Validates dependencies come from trusted
sources
- **Automated Enforcement**: Catches licensing issues during PR review
rather than at release time
- **Transparency**: Provides clear audit trail of approved licenses and
authors
2025-11-02 23:32:59 +02:00
Gonza Montiel
96c4408682
fix: add missing weights (#271)
### Add missing weights for BABE, GRANDPA, and Randomness

#### Summary
Adds generated weights and wires them into the runtime for the BABE,
GRANDPA, and Randomness pallets to replace defaults and ensure accurate
execution costs across networks.

#### What’s changed
- **New weights added** for `pallet_grandpa`, `pallet_babe` and
`pallet_randomness`
- **Runtime configs updated to use new weights**
  - `operator/runtime/mainnet/src/configs/mod.rs`
  - `operator/runtime/stagenet/src/configs/mod.rs`
  - `operator/runtime/testnet/src/configs/mod.rs`
 
 #### For follow-up PRS
- fix `pallet_identity` failure at running benchmarks
- fix `pallet_collective` benchmarking missmatch (related to
https://github.com/paritytech/polkadot-sdk/pull/6435)
- add `pallet_session_benchmarking` without including `pallet_staking`
(or some workaround)
- add StorageHub weights to our benchmarked pallets (`pallet_nfts`,
`pallet_storage_providers`, `pallet_payment_streams`,
`pallet_proofs_dealer`, `pallet_file_system`, `pallet_bucket_nfts`, etc)
2025-11-02 22:50:55 +02:00
Steve Degosserie
e860c503c8
feat: Bump client version to v0.4.0 & runtime version to RT400 (#268) 2025-10-30 18:02:25 +02:00
Ahmad Kaouk
0b636e0d79
fix: Fix EVM gas-to-weight handling for call/create (#266)
## Summary
- Replace the legacy “estimated transaction length” heuristic in the EVM
`call` runtime API across mainnet, stagenet, and testnet with a direct
`GasWeightMapping::gas_to_weight` lookup. The resulting weight is now
always forwarded to the runner (`Some(weight_limit)`), so zero-gas
requests no longer slip through without a cap.
- Update the EVM `create` runtime API the same way. Previously it always
passed `None` for `weight_limit`, effectively running contract-deploy
dry-runs without any weight ceiling; we now map the gas limit and pass
the explicit weight instead.
- For both `call` and `create`, set the proof-size base cost to `None`
to match our solo-chain assumption that PoV size isn’t budgeted in these
simulated paths.

## Why
We use these runtime APIs when serving `eth_call` and `eth_estimateGas`.
The old behavior meant a zero gas limit (or any `create` dry-run) ran
with unlimited weight, diverging from what the extrinsic path enforces.
Passing the mapped weight—zero included—keeps RPC simulations aligned
with real execution, while dropping the proof-size estimate removes a
guessy value we don’t charge on-chain.
2025-10-30 17:09:15 +02:00
Ahmad Kaouk
2f6c6e39c2
fix: add explicit sovereign account balance check in unlock_tokens (#253)
Add defensive validation to ensure the Ethereum sovereign account has
sufficient balance before unlocking tokens. This addresses an audit
finding where the lack of explicit balance checking created an
unreliable security control that depended on implicit runtime behavior.

Changes:
- Add InsufficientSovereignBalance error variant for clear error
messaging
- Add explicit balance check in unlock_tokens before transfer
- Update tests across all runtimes (testnet, stagenet, mainnet) to
validate the specific error is returned when sovereign account has
insufficient funds

The explicit check provides better error messages that can propagate
through the Ethereum bridge and makes debugging sovereign account
balance issues easier.
2025-10-30 11:19:14 +00:00
Steve Degosserie
45b5551b21
chore: ♻️ Remove unused API declarations in Testnet runtime (#262)
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-10-30 09:30:26 +00:00
Gonza Montiel
e079cdc404
fix: 🩹 add mandatory extrinsics to safe mode whitelisted calls (#265)
# Fix: Safe Mode Whitelisted Calls - enable block production

## Problem
The safe mode whitelist was missing critical runtime calls needed for
block production, generating this error:

```
2025-10-29 17:29:48 Proposing failed: Import failed: Extrinsic is not valid: TransactionValidityError::Invalid(InvalidTransaction::BadMandatory)
```

The SafeMode filter needs to include all RuntimeCalls that have
inherents marked as `DispatchClass::Mandatory`, as you can see
[here](bbc435c766/substrate/frame/executive/src/lib.rs (L806)).

If a single inherent is missing the whole block will not be valid,
causing the chain to stall.

## Solution
Bisect all the calls to find the culprit, until find it was the pallet
Randomness. I included it in `SafeModeWhitelistedCalls` and blocks are
being produced in SafeMode.
2025-10-30 08:38:18 +00:00
undercover-cactus
7c8227f1ab
feat: set slashing mode in genesis config (#264)
In this PR we set the slashing mode value in the genesis config. For the
3 different runtime we specify the slashing mode : `mainnet/testnet` is
set to `Disabled` and for `stagenet` to `LogOnly`.

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-10-29 18:24:49 +02:00
Gonza Montiel
70a505e9d8
fix: update weight configurations for the three runtimes (#249)
# Update Runtime Weights and WeightInfo Implementations

## Summary
Updated runtime weights across all three runtimes (mainnet, stagenet,
testnet) and configured WeightInfo implementations to use the new
benchmarked weights.

## Changes
- All weight files have been updated with new benchmark data from
Substrate Benchmark CLI v51.0.0
- Updated some WeightInfo configurations to use custom weights instead
of the default `()`, such as `pallet_im_online`.

## Weights comparison against Moonbeam

Some weights are much higher than it's counterpart in Moonbeam, as we
can observe in the following table. We need to investigate if this is
expected.

EDIT:

### Weights using the `production` profile comparing stagenet vs
moonbase

| **Pallet** | **Function** | **DataHaven Weight** | **Moonbase Weight**
| **Difference** | **Notes** |

|------------|--------------|----------------------|---------------------|----------------|-----------|
| **pallet_balances** | transfer_allow_death | 68,497,000 | 43,117,000 |
**+59%** | DataHaven higher |
| | transfer_keep_alive | 55,245,000 | 41,490,000 | **+33%** | DataHaven
higher |
| | force_set_balance_creating | 22,034,000 | 21,537,000 | **+2%** |
Similar |
| | force_set_balance_killing | 31,246,000 | 23,467,000 | **+33%** |
DataHaven higher |
| | force_transfer | 72,181,000 | 45,384,000 | **+59%** | DataHaven
higher |
| | transfer_all | 67,765,000 | 51,403,000 | **+32%** | DataHaven higher
|
| | force_unreserve | 25,876,000 | 19,490,000 | **+33%** | DataHaven
higher |
| | upgrade_accounts (base) | 23,200,000 | 17,599,000 | **+32%** |
DataHaven higher |
| | force_adjust_total_issuance | 8,780,000 | 6,780,000 | **+29%** |
DataHaven higher |
| | burn_allow_death | 43,258,000 | 25,822,000 | **+68%** | DataHaven
higher |
| | burn_keep_alive | 30,368,000 | 22,961,000 | **+32%** | DataHaven
higher |
| **pallet_evm** | withdraw | 3,174,000 | 2,104,000 | **+51%** |
DataHaven higher |
| **frame_system** | remark (base) | 33,694,157 | 38,668,753 | **-13%**
| Moonbase higher |
| | remark_with_event (base) | 8,210,000 | 5,745,000 | **+43%** |
DataHaven higher |
| | set_heap_pages | 5,204,000 | 3,734,000 | **+39%** | DataHaven higher
|
| | set_code | 139,034,247,000 | 102,532,308,000 | **+36%** | DataHaven
higher |
| | set_storage (base) | 3,248,000 | 2,094,000 | **+55%** | DataHaven
higher |
| | kill_storage (base) | 3,344,000 | 2,120,000 | **+58%** | DataHaven
higher |
| | kill_prefix (base) | 6,028,000 | 4,258,000 | **+42%** | DataHaven
higher |
| | authorize_upgrade | 12,866,000 | 8,966,000 | **+44%** | DataHaven
higher |
| | apply_authorized_upgrade | 142,435,318,000 | 106,561,461,000 |
**+34%** | DataHaven higher |
| **pallet_multisig** | as_multi_threshold_1 (base) | 30,107,366 |
17,588,619 | **+71%** | DataHaven higher |
| | as_multi_create (base) | 45,353,453 | 32,782,725 | **+38%** |
DataHaven higher |
| | as_multi_approve (base) | 22,928,808 | 18,299,865 | **+25%** |
DataHaven higher |
| | as_multi_complete (base) | 57,510,735 | 41,555,387 | **+38%** |
DataHaven higher |
| | approve_as_multi_create (base) | 42,860,897 | 32,390,857 | **+32%**
| DataHaven higher |
| | approve_as_multi_approve (base) | 21,640,639 | 16,839,439 | **+29%**
| DataHaven higher |
| | cancel_as_multi (base) | 44,014,197 | 32,643,060 | **+35%** |
DataHaven higher |
| **pallet_treasury** | spend_local | 15,605,000 | 12,992,000 | **+20%**
| DataHaven higher |
| | remove_approval | 8,641,000 | 7,766,000 | **+11%** | DataHaven
higher |
| | on_initialize_proposals (base) | 19,249,676 | 16,973,066 | **+13%**
| DataHaven higher |
| | spend | 14,082,000 | 26,612,000 | **-47%** | Moonbase higher |
| | payout | 74,847,000 | 247,397,000 | **-70%** | Moonbase
significantly higher |
| | check_status | 16,325,000 | 16,223,000 | **+1%** | Similar |
| | void_spend | 15,059,000 | 15,167,000 | **-1%** | Similar |
| **pallet_scheduler** | service_agendas_base | 4,583,000 | 6,215,000 |
**-26%** | Moonbase higher |
| | service_agenda_base (base) | 8,728,153 | 6,640,549 | **+31%** |
DataHaven higher |
| | service_task_base | 4,875,000 | 3,455,000 | **+41%** | DataHaven
higher |
| | service_task_fetched (base) | 24,253,000 | 18,898,000 | **+28%** |
DataHaven higher |
| | service_task_named | 7,176,000 | 5,027,000 | **+43%** | DataHaven
higher |
| | service_task_periodic | 4,842,000 | 3,415,000 | **+42%** | DataHaven
higher |
| | execute_dispatch_signed | 10,378,000 | 4,837,000 | **+115%** |
DataHaven significantly higher |
| | execute_dispatch_unsigned | 3,511,000 | 2,451,000 | **+43%** |
DataHaven higher |
| | schedule (base) | 17,795,637 | 13,165,173 | **+35%** | DataHaven
higher |
| | cancel (base) | 20,509,326 | 15,513,197 | **+32%** | DataHaven
higher |

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-10-29 17:14:21 +02:00
undercover-cactus
d748d7d61b
fix: keep slahing mode default to enable (#263)
To avoid breaking the tests we should not change the default value for
the slashing mode.

Bring back `Enabled` as the default slashing mode value.
2025-10-29 15:01:10 +01:00
undercover-cactus
f0896907ae
feat: add slashing support (#242)
## 🔨 Add Slashing Support for Runtime

This PR introduces the slashing functionality for the DataHaven runtime,
enabling punitive measures against misbehaving validators.


### Features
- Deferred slashing with configurable veto periods
- Cross-chain slashing message delivery trough Snowbridge
- Governance controls for slashing parameters and emergency cancellation

We introduced the `external-validator-slashes` pallet, which allows to
slash validators that misbehave. The slashing is triggered when an
offence is reported via the offence pallet (which is already
implemented). The message is sent through Snowbrige's outbound queue and
the real slashing happens in the contracts side, which will come in a
follow up PR.

There is a configurable window of time between the time the validator is
being reported, and the time the slash is triggered. This allows that in
case of an error we are still able to cancel the slashing, using a sudo
account.

For convenience, we also have extrinsics for corner cases:

- **`force_inject_slash`**: Root-only function to manually inject
slashes for specific validators with custom percentages. Useful for
emergency situations or governance-directed slashing outside normal
offence detection
- **`cancel_deferred_slash`**: Allows governance to cancel pending
slashes during the defer period by specifying era and slash indices.
Provides safety mechanism against false positives or malicious slash
reports
- **`set_slashing_mode`**: Configurable slashing behavior with three
modes - `Enabled` (normal operation), `LogOnly` (track offences without
applying slashes), and `Disabled` (completely halt slashing). Critical
for emergency response and testing

---------

Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
2025-10-29 10:43:55 +00:00
Gonza Montiel
782321e5d0
feat: Implement dynamic fee adjustment (#251)
#  Implement Dynamic Fee Adjustment Mechanism

## Overview
Implements a dynamic fee adjustment mechanism, replacing the constant
fee multiplier with an adaptive multiplier that responds to network
congestion, following Moonbeam's pattern.

## Changes
- Replaces `ConstFeeMultiplier` with `TargetedFeeAdjustment` across all
runtime configurations (mainnet, stagenet, testnet)
- Implements an EIP-1559-like slow-adjusting fee mechanism that prevents
DoS attacks by adjusting fees based on block fullness
- **Configurable Parameters**: 
  - Target block fullness: 35%
- Adjustment variable: 4/1000 (responds in ~1 hour at extreme
congestion)
  - Two modes:
    -  `SlowAdjustingFeeUpdate` for mainnet and testnet.
    - `FastAdjustingFeeUpdate` for stagenet.
- Adds tests coverage for different fee scenarios

## Technical Details

The fee adjustment algorithm works as follows:
```
diff = (previous_block_weight - target) / maximum_block_weight
next_multiplier = prev_multiplier * (1 + (v * diff) + ((v * diff)^2 / 2))
assert(next_multiplier > min)
```
**Where:**
- `v` = AdjustmentVariable
- `target` = TargetBlockFullness  
- `min` = MinimumMultiplier

`SlowAdjustingFeeUpdate` sets a minimum multiplier of `1x` for a
conservative fee adjustment, while `FastAdjustingFeeUpdate` sets it to
`0.1x`, which is mainly used for dev networks / testing.
2025-10-28 10:06:45 +00:00
Steve Degosserie
830d4baf8a
fix: 🔧 Remove redundant operator Dockerfile.local (#257)
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
2025-10-27 19:14:13 +01:00
Steve Degosserie
879be715b5
feat: Bump client version to v0.3.1 & runtime spec_version to 310 (#250) 2025-10-24 19:57:07 +03:00
Steve Degosserie
06574a4cf8
fix: 🔧 Lower StorageHub replication targets for the Testnet environment (#245)
We initially use the same values as for Stagenet.
2025-10-23 14:14:30 +03:00
Steve Degosserie
22d7ee4b94
fix: 🔧 Bump SH storage providers pallet's MaxMultiAddressSize parameter to 200 (#244) 2025-10-23 13:35:23 +03:00
Steve Degosserie
5988691a2f
feat: Add deployment charts for StorageHub MSP, BSP & Indexer nodes (Local & Stagenet envs) (#160)
## Summary

This PR adds comprehensive Kubernetes deployment infrastructure for
StorageHub components, enabling deployment of the full StorageHub
network stack (MSP, BSP, Indexer, and Fisherman nodes) alongside
DataHaven nodes in both local and stagenet environments.

### What's Added

**1. New Helm Chart: StorageHub MSP Backend API**
(`deploy/charts/backend/`)
- REST API service for StorageHub operations
- Connects to PostgreSQL database for indexed blockchain data
- Connects to RPC nodes for real-time blockchain queries
- Configurable via TOML configuration file
- Supports environment-specific overrides
- Includes comprehensive documentation

**2. StorageHub Node Deployment Charts**
(`deploy/charts/node/storagehub/`)
- **MSP Node** (`sh-mspnode`): Main Service Provider nodes with charging
capabilities
- **BSP Node** (`sh-bspnode`): Backup Service Provider nodes for
redundancy
- **Indexer Node** (`sh-idxnode`): Full indexing node with PostgreSQL
integration
- **Fisherman Node** (`sh-fisherman`): Network monitoring and
verification node

**3. Environment Configurations**
- **Local environment** (`deploy/environments/local/`): Development
setup with hostpath storage
- **Stagenet environment** (`deploy/environments/stagenet/`):
Production-like setup with AWS EBS
- PostgreSQL database configurations for Indexer and Fisherman nodes
- Proper service discovery and network configuration

**4. Enhanced CLI Tooling** (`test/cli/`)
- New `deploy storagehub` command for deploying StorageHub components
- Updated `launch storagehub` command for local testing
- Interactive deployment with environment selection
- Automatic database provisioning via Bitnami PostgreSQL charts

**5. Node Configuration Improvements**
- Fork-aware transaction pool for DH boot & validator nodes
- Unsafe RPC methods exposed on MSP nodes (for provider operations)
- JWT secret support for MSP Backend authentication
- ECDSA key scheme for StorageHub BCSV keys (DataHaven compatibility)

### Architecture

```
StorageHub Stack:
├── MSP Nodes (2 replicas) → Storage providers with charging
├── BSP Nodes (2 replicas) → Backup storage providers
├── Indexer Node → Database indexing + PostgreSQL
├── Fisherman Node → Monitoring + PostgreSQL (shared with Indexer)
└── MSP Backend API → REST API for StorageHub operations
```

### Testing

**Local Testing**:
```bash
cd test
bun cli launch storagehub  # Interactive launcher
# or
bun cli deploy storagehub  # Deploy via Helm
```

**Stagenet Deployment**:
```bash
cd deploy
helm install sh-mspnode ./charts/node \
  -f ./charts/node/storagehub/sh-mspnode.yaml \
  -f ./environments/stagenet/sh-mspnode.yaml \
  -n datahaven-stagenet
```

### Breaking Changes

None - This is purely additive infrastructure.

### Migration Notes

For existing deployments:
1. DataHaven nodes now use `--pool-type fork-aware` flag
2. Bootnode and validator node configs updated accordingly
3. No action required for existing DataHaven-only deployments
2025-10-21 23:18:50 +03:00
Steve Degosserie
cffdad2358
fix: 🔧 Don't remove /usr/sbin utils in operator Docker image (#240) 2025-10-15 23:54:29 +02:00
Steve Degosserie
931a225f09
fix: 🔧 Fix binary location in Docker img (#239) 2025-10-15 22:54:44 +02:00
Steve Degosserie
9a5404de82
refactor: Consolidate and optimize Docker image architecture (#233)
## Overview

This PR consolidates and optimizes the Docker build system, reducing
redundancy and improving CI/CD performance. The changes eliminate
duplicate Dockerfiles, introduce a flexible build template, and optimize
release builds to reuse CI artifacts.

## Changes Summary

### 🐳 Docker Images Restructured

**Before:** 5 Dockerfiles with significant overlap
**After:** 4 focused images + 1 utility

#### Final Structure:

1. **`operator/Dockerfile`**  Updated
   - **Standard operator image** for CI and release builds
   - Minimal node image (accepts pre-built binaries)
   - GHCR: `ghcr.io/datahaven-xyz/datahaven/datahaven` (CI)
   - DockerHub: `datahavenxyz/datahaven` (releases)

2. **`docker/datahaven-build.Dockerfile`** (moved from
`operator/Dockerfile`)
   - Full source-to-binary build for manual releases
   - DockerHub: `datahavenxyz/datahaven:{label}`
   - Supports custom RUSTFLAGS and fast-runtime feature
   - Only used for manual workflow_dispatch builds

3. **`docker/datahaven-production.Dockerfile`** (kept)
   - Binary builder for CPU-specific releases
   - Used by build-prod-binary workflow template
   - Supports custom target-cpu flags

4. **`docker/datahaven-dev.Dockerfile`**  NEW (local dev only)
   - **FOR LOCAL DEVELOPMENT/TROUBLESHOOTING ONLY**
   - Includes debug tools: gdb, strace, vim, sudo
   - Extra dependencies: librocksdb-dev, curl
   - RUST_BACKTRACE enabled by default
   - **DO NOT USE for CI or production builds**

5. **`test/docker/crossbuild-mac-libpq.dockerfile`** (kept)
   - Utility for macOS → Linux cross-compilation

#### Removed (Redundant):
-  `docker/datahaven.Dockerfile` → replaced by operator/Dockerfile
-  `test/docker/datahaven-node-local.dockerfile` → replaced by
datahaven-dev.Dockerfile

---

### 🔄 Workflow Improvements

#### Enhanced `publish-docker` Template
- Supports both GHCR and DockerHub registries
- Flexible inputs: dockerfile, context, build-args, cache scope
- Auto-generates OCI-compliant labels
- Reduces code duplication (~70 lines → ~15 per workflow)

#### Refactored CI Pipeline
- **`docker-build-ci`**: Builds `operator/Dockerfile` → GHCR for CI/E2E
testing
- **`docker-build-release`**: Builds `operator/Dockerfile` → DockerHub
(main branch only)
- Both CI and release workflows now use the same minimal operator image
- Release builds **reuse CI binaries** instead of rebuilding from source

#### Optimized Release Workflow
The `task-docker-release` workflow now has dual modes:

**Mode 1: `workflow_call` (CI - main pushes)**
-  Reuses binary from CI's build-operator task
-  Uses lightweight `operator/Dockerfile`
-  Tags: `latest`, `sha-{short}`
-  **Fast**: ~5 minutes (vs ~30 min previously)

**Mode 2: `workflow_dispatch` (Manual)**
-  Full source build with `datahaven-build.Dockerfile`
-  Custom branch and label selection
-  Optional fast-runtime feature
-  Tags: `PROD-{label}` or user-defined

---

### 🔧 Additional Optimizations

- Copy libpq5 from builder stage instead of reinstalling (smaller,
faster)
- Remove redundant protobuf-compiler package (use protoc v21.12
directly)
- Standardize user UID to 1000 across all runtime images
- Consistent OCI labeling and metadata

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-15 01:33:20 +02:00
Steve Degosserie
8874a99100
fix: 🔧 Add missing libpq5 lib to DH operator Docker image (#232) 2025-10-14 12:37:31 +02:00
Steve Degosserie
f934707d7c
feat: Bump client version to v0.3.0 & runtime spec_version to 300 (#224) 2025-10-11 00:38:19 +02:00
Gonza Montiel
1ec56681a5
feat: add referenda precompile (#217)
# Add Referenda Precompile

This PR introduces the referenda precompile from Moonbeam to enable
governance functionality through EVM calls.

## Changes Made

- Added `operator/precompiles/referenda/` with complete implementation
- Updated all runtime configs (mainnet, stagenet, testnet) to include
the referenda precompile
- Adapted track processing logic to work with DataHaven's runtime
configuration
- Adapted tests and mock according to our runtime

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-10-11 00:17:21 +02:00
Steve Degosserie
8c950af4a4
fix: 🔧 Add Podman support to srtool runtime build script (#222)
## Summary

- Adds support for both Docker and Podman container engines in
`build-runtime-srtool.sh` via `IS_PODMAN` environment variable
- Uses `--userns=keep-id` for Podman (proper user namespace handling)
and `--user $(id -u):$(id -g)` for Docker
- Sets `IS_PODMAN=true` in `task-publish-runtime.yml` workflow to enable
Podman by default

## Changes

**`operator/scripts/build-runtime-srtool.sh`:**
- Added conditional logic to detect `IS_PODMAN` env var
- Dynamically selects between `podman` and `docker` as container engine
- Sets appropriate user/namespace flags based on container engine

**`.github/workflows/task-publish-runtime.yml`:**
- Added `IS_PODMAN: true` environment variable to the srtool build step
- Updated comment to use generic "container user" instead of "docker
user"

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-10 22:57:34 +02:00
Facundo Farall
c978150582
build: ⬆️ Upgrade to StorageHub v0.0.6-alpha (#212)
This PR upgrades the StorageHub dependencies to tag
[v0.0.6-alpha](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.0.6-alpha).
This includes the fix to connect through TLS to a Postgres DB (allowing
connecting to an AWS hosted DB for instance), and a fix for a missing
indexer DB migration.

Additionally, it adds a new runtime API.

EDIT (previously breaking changes):
As of the new version, the name of a column in the indexer DB has
changed. This can affect the functionality of nodes running a Postgres
DB with the old schema. A
[migration](05d269a26d)
is included in the new
[tag](https://github.com/Moonsong-Labs/storage-hub/commits/v0.0.6-alpha/),
so no need to mark it as breaking.

---------

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-10-10 22:04:32 +02:00
Ahmad Kaouk
55e973b8f0
fix: change pallet_evm alias to EVM to fix eth_getCode (#213)
## Summary
- rename the FRAME alias for `pallet_evm` from `Evm` to `EVM` across the
mainnet, stagenet, and testnet runtimes
- adjust benchmarks, configuration modules, genesis builders, and
runtime tests to rely on the new alias
- keep precompile genesis setup and proxy/precompile tests aligned with
the updated names

  ## Context
Frontier’s `StorageOverrideHandler` (see
`fc_storage::StorageQuerier::account_code`) reads contract bytecode from
`pallet_evm::AccountCodes` using the constant `PALLET_EVM = b"EVM"` to
build the storage key:
  `twox_128("EVM") ++ twox_128("AccountCodes") ++ …`

Our runtimes exported `pallet_evm` as `Evm`, so substrate stored
bytecode under the *camel-cased* prefix (`twox_128("Evm")`). Every call
that ultimately hits the storage override—including `eth_getCode`,
`eth_call`, and state queries during replay—therefore failed to locate
code for *any* account (deployed contracts and precompiles alike).
Renaming the alias to `EVM` realigns the storage prefix with Frontier’s
  expectations so the override layers can pull bytecode correctly.

  ## Testing
  - `cargo check -p datahaven-node`
  - `cargo build --release -p datahaven-node`
- `eth_getCode 0x0000000000000000000000000000000000000802` → returns
`0x60006000fd`
  
## Storage Migration
Renaming a pallet alias changes the storage prefix for all pallet data.
Without migration, existing EVM data (smart contracts, account codes,
storage) would become inaccessible.

  **Migration details:**

  - **Type**: Multi-Block Migration (MBM)
- **Storage migrated**: `AccountCodes`, `AccountCodesMetadata`,
`AccountStorages`
  - **Migration ID**: `datahaven-evm-mbm` (version 0 → 1)

  **Testing the migration:**

  ```bash
  # Build runtime with try-runtime
cargo build --release --features try-runtime -p
datahaven-stagenet-runtime

  # Test against stagenet
try-runtime \
--runtime
./target/release/wbuild/datahaven-stagenet-runtime/datahaven_stagenet_runtime.wasm
\
      on-runtime-upgrade \
      --blocktime 6000 \
      --checks all \
      --disable-spec-version-check \
      live --uri wss://dh-validator-0.datahaven-kt.xyz
```

  Test results from stagenet:
  -  Migration completes in 1 block
  -  PoV size: ~5.3 KB
  -  Weight consumption: <0.1% of block capacity
  -  All 39 keys successfully migrated

  ## ⚠️ Breaking Changes ⚠️
If you are manually computing storage keys for the EVM pallet (e.g., directly querying chain state), you must update your code to use the new storage prefix:
  - Old prefix: twox128("Evm") = 0x8b90cb...
  - New prefix: twox128("EVM") = 0x6a5e91...

  All EVM-facing interfaces remain unchanged.
2025-10-10 17:48:52 +00:00
Steve Degosserie
ba5ce56c5f
Revert "fix: 🔧 Fix build runtime srtool permission issue" (#221)
This reverts commit 5645c2f9f4.
2025-10-10 18:15:22 +02:00
Steve Degosserie
062584d1a2
fix: 🔧 Fix build runtime srtool permission issue (#220)
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-10 16:38:11 +02:00
Steve Degosserie
8c693bfde7
fix: resolve srtool metadata hash build error (#219)
## Summary
- Fixes srtool build failure with `UnknownOpcode(252)` error during
metadata hash generation
- Removes hardcoded `WASM_BUILD_STD=0` to allow srtool to auto-detect
correct settings based on Rust version
- Adds `metadata-hash` feature to node Cargo.toml for CheckMetadataHash
extension

## Root Cause
The `WASM_BUILD_STD=0` setting forced srtool to use pre-built standard
library crates that were compiled with incompatible WASM features.
During metadata hash generation, the runtime builder encountered opcode
252 (likely from bulk memory operations) which the deserializer couldn't
recognize, causing the build to fail with:

```
thread 'main' panicked at metadata_hash.rs:73:10:
`Metadata::metadata_at_version` should exist.: RuntimeConstruction(Other("cannot deserialize module: UnknownOpcode(252)"))
```

## Changes
1. **operator/scripts/build-runtime-srtool.sh**: Removed hardcoded `-e
WASM_BUILD_STD=0` line
- Allows srtool to determine appropriate setting based on Rust version
   - For Rust < 1.84: defaults to WASM_BUILD_STD=1 (enabled)
   - For Rust >= 1.84: defaults to WASM_BUILD_STD=0 (disabled)

2. **operator/node/Cargo.toml**: Added `metadata-hash` feature
propagation
- Enables metadata-hash feature for all runtime variants (stagenet,
mainnet, testnet)
   - Required for CheckMetadataHash extension support

## Test Plan
- [x] Successfully built stagenet runtime with srtool 1.88.0
- [x] Build completed in ~14 minutes without metadata hash errors
- [x] Verified WASM artifacts generated correctly

## Testing Command
```bash
GH_WORKFLOW_MATRIX_CHAIN=stagenet \
RUNTIME_BUILD_OPTS="--features=on-chain-release-build" \
RUNTIME_BUILD_PROFILE="production" \
GH_WORKFLOW_MATRIX_SRTOOL_IMAGE="paritytech/srtool" \
GH_WORKFLOW_MATRIX_SRTOOL_IMAGE_TAG="1.88.0" \
WASM_BUILD_STD=1 \
./operator/scripts/build-runtime-srtool.sh
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-10 15:04:03 +02:00
undercover-cactus
514a16ac1f
ci: remove sccache from image build for prod (#200)
In this PR, we remove the caching of the sccache folder because it is
too big (~3GB) and fill our cache too fast.

What to expect ?  
* It will make the build a bit slower but it is fine because it only
build on `main`. We are preparing another PR that will speed up the
build of the prod image. Also we are not sure the cache is actually
being used (`gha` cache is in beta).
* Will free some space for caching and stop deleting our cache which
make other jobs work faster.

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
2025-10-09 12:33:35 +00:00
Gonza Montiel
2d6056721a
feat: add preimage precompile (#211)
## Add Preimage Precompile

This PR integrates the Preimage precompile from Moonbeam into the
DataHaven runtime across all three environments (mainnet, stagenet,
testnet).

**Key Changes:**
- Added Preimage precompile implementation at address `2067` in all
runtime configurations
- Updated precompile sets in mainnet, stagenet, and testnet runtimes
- Updated `is_governance_precompile()` for the Preimage precompile
2025-10-09 13:16:46 +02:00
Gonza Montiel
0988ce46d0
feat: add conviction voting precompile (#202)
## Add Conviction Voting Precompile

This PR introduces a new EVM precompile for conviction voting
functionality, enabling smart contracts to interact with the Substrate
conviction voting pallet.

### Key Changes

- **New Precompile**: Added `ConvictionVotingPrecompile` at address
`0x0000000000000000000000000000000000000812` (2066)
- **Solidity Interface**: Complete Solidity interface
(`ConvictionVoting.sol`) with all conviction voting operations
- **Runtime Integration**: Integrated precompile across all runtime
configurations (mainnet, testnet, stagenet)
- **Comprehensive Testing**: Inherits test suite from Moonbeam

### Features

- Vote casting with different conviction levels (None, Locked1x-6x)
- Vote delegation and undelegation
- Poll management and tallying
- Support for all conviction voting pallet operations

---------

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-10-09 10:30:50 +02:00
Gonza Montiel
f7d441d9e5
feat: add collective precompile (#204)
## Add Collective Precompile

Adds the pallet collective precompile to `mainnet`, `stagenet`, and
`testnet` according to Moonbeam's configuration.

### Changes:
- Added `pallet-evm-precompile-collective` dependency to workspace
- Configured collective precompile at address `2064` using
`TreasuryCouncilInstanc`
- Configured collective precompile at address `2068` using
`TechnicalCommitteeInstance`

The precompile provides EVM access to collective governance
functionality including proposal execution, voting, and membership
management.

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-10-08 23:16:34 +02:00
Gonza Montiel
65e245a82e
feat: add identity precompile (#205)
## Add Identity Precompile

Adds the pallet Identity precompile to `mainnet`, `stagenet`, and
`testnet` according to Moonbeam's configuration.

### Changes:
- Added `pallet-evm-precompile-identity` dependency to workspace
- Added and configured at address `2072` for all runtimes

The precompile provides a Solidity interface access to the Substrate
Identity pallet functionality.

---------

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-10-08 13:48:17 +00:00
Ahmad Kaouk
de2a1c1181
fix: fix srtool user (#215) 2025-10-08 11:40:13 +00:00
Steve Degosserie
bded4e9d5f
fix: 🔧 Fix build-runtime-srtool script (#210) 2025-10-07 13:45:34 +02:00
Steve Degosserie
72bfe9bb62
fix: 🔧 Add revision number to rust toolchain channel version to match with srtool image version (#209) 2025-10-07 11:34:21 +02:00
Ahmad Kaouk
ac09a4f2bb
feat: Add SafeMode and TxPause Pallets (#192)
### Overview
This PR integrates the `pallet-safe-mode` and `pallet-tx-pause` from
Polkadot SDK to provide comprehensive emergency governance controls
across all DataHaven runtime networks (mainnet, stagenet, testnet).

### Key Changes

#### 🔧 **Core Integration**
- **Dependencies**: Added `pallet-safe-mode` and `pallet-tx-pause` from
`polkadot-stable2412-6`
- **Runtime Integration**: Integrated both pallets across all three
runtime networks with pallet indices 103 and 104
- **Call Filtering**: Implemented unified `RuntimeCallFilter` that
combines Normal, SafeMode, and TxPause restrictions

#### 🛡️ **SafeMode Pallet Configuration**
- **Duration**: 1 day activation period (`DAYS` constant)
- **Deposits**: Disabled permissionless entry/extension (all `None`)
- **Origins**: Root-only for all force operations (`force_enter`,
`force_exit`, `force_extend`, etc.)
- **Whitelisting**: SafeMode and Sudo calls are immune to restrictions

#### ⏸️ **TxPause Pallet Configuration** 
- **Origins**: Root-only pause/unpause control
- **Whitelisting**: SafeMode and Sudo calls cannot be paused
- **Max Call Name Length**: 256 characters

#### 🏗️ **Architecture**
- **Shared Types**: Created `operator/runtime/common/src/safe_mode.rs`
with reusable configurations
- **Combined Filtering**: `RuntimeCallFilter` applies all three filter
layers (Normal + SafeMode + TxPause)
- **Consistent Config**: Identical configuration across mainnet,
stagenet, and testnet

#### 📊 **Infrastructure Updates**
- **Benchmarking**: Added both pallets to benchmark suites across all
networks
- **Weight Mappings**: Placeholder weights using Substrate defaults
(ready for chain-specific benchmarking)
- **Metadata**: Updated runtime metadata for new pallet exposure

#### 🧪 **Testing Framework**
- **Coverage**: Tests for individual pallet behavior, combined
restrictions, whitelisting, and edge cases

### Emergency Control Capabilities

**SafeMode Pallet** (8 calls):
- User calls: `enter`, `extend`, `release_deposit`
- Force calls: `force_enter`, `force_exit`, `force_extend`,
`force_slash_deposit`, `force_release_deposit`

**TxPause Pallet** (2 calls):
- `pause_call` / `unpause_call` - Granular transaction type pausing

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-10-06 17:00:10 +00:00
Steve Degosserie
b4f697f954
chore: ♻️ Update README files (#206)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-10-06 18:12:55 +02:00
Ahmad Kaouk
48442258ab
fix: fix block authoring and finalization (#203) 2025-10-02 10:58:40 +02:00
Steve Degosserie
83e036e47f
fix: 🔧 Initialize dummy revert bytecode at precompile addresses in genesis presets (#199)
To ensure that the runtime precompiles they can be called from within
contracts, there must be some code deployed at the precompile address.
This PR ensure the simplest bytecode (which reverts) is deployed at
genesis time for all addresses of precompiles present in the runtime.
2025-10-01 17:14:14 +02:00
undercover-cactus
d2ff687dc0
feat: added the possibility to provide a config file for SH (#191)
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-09-30 17:39:11 +00:00
Ahmad Kaouk
17c706dc64
test: Integrate moonwall (#185)
### Description

This PR introduces the **Moonwall** end-to-end (E2E) testing framework.
The primary motivation for this is to enable the porting of existing
Mobeam tests into the `DataHaven` repository.

### Key Changes

*   **Node Manual Sealing:**
* Introduced a `--sealing=manual` flag for the `datahaven-node`. When
enabled, blocks are only produced on demand via an RPC call. This is the
core mechanism that allows for deterministic tests.

*   **Moonwall Framework Integration:**
* Added `@moonwall/cli` and `@moonwall/util` dependencies to the
`test/package.json`.
* A new `test/moonwall.config.json` file configures the test
environment, defining how Moonwall should launch the `datahaven-node`
with the manual sealing flag.
* Added a `moonwall:test` script to `package.json` for running the
tests.

*   **CI Workflow:**
* A new reusable workflow, `.github/workflows/task-moonwall-tests.yml`,
has been created to handle the setup, execution, and reporting of
Moonwall tests.
* The main `CI.yml` now includes a `moonwall-tests` job that runs after
the `build-operator` job, ensuring it always tests the correct,
freshly-built binary.

*   **Example Test Suite:**
* A new test suite, `test/datahaven/suites/dev/test-block.ts`, had been
copied from moonbeam.

### How to Run Locally

1.  Navigate to the `test` directory.
2.  Install dependencies: `bun install`
3.  Run the tests: `bun run moonwall:test`

---------

Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
2025-09-30 14:47:39 +00:00
Steve Degosserie
51f4e1b0f3
feat: Publish runtime release Github action (#196) 2025-09-30 14:53:15 +02:00