Commit graph

310 commits

Author SHA1 Message Date
Steve Degosserie
15a518697e
feat: Bump runtime version to 840 (#401) 2026-01-19 21:33:38 +02:00
Ahmad Kaouk
b88f051bdf
fix: Increase MaxBatchConfirmStorageRequests from 10 to 100 (#400)
- Increase `MaxBatchConfirmStorageRequests` runtime constant from 10 to
100
- Applied across all runtime environments: mainnet, stagenet, and
testnet
- [x] Verify builds pass for all runtime configurations

(cherry picked from commit db4608f9dd)
2026-01-19 21:05:54 +02:00
Steve Degosserie
c0c34a1a0c
feat: RT830 runtime release (SH v0.2.6) (#359) 2025-12-16 17:28:56 +01:00
Facundo Farall
ae35a0824d
build: ⬆️ Upgrade to StorageHub version 0.2.6 (#357)
Upgrade to StorageHub release v0.2.6

No breaking changes, just a patch release

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
(cherry picked from commit d5e64d59e8)
2025-12-16 17:04:05 +01:00
Facundo Farall
d2dc4b846d
build: ⬆️ Upgrade to StorageHub v0.2.5 (#347)
Upgrades to StorageHub patch release v0.2.5

(cherry picked from commit 0d5f294097)
2025-12-09 15:27:09 +01:00
Steve Degosserie
915cd172c4
feat: RT820 runtime release (SH v0.2.5) (#350) 2025-12-09 15:23:40 +01:00
Steve Degosserie
7532b15d33
chore: ♻️ update metadata 2025-12-04 21:45:31 +01:00
Tobi Demeco
f5c0397960
build: ⬆️ Upgrade to SH v0.2.3 (#340)
Upgrade StorageHub to v0.2.3 which includes some fixes for the indexer
plus minor changes to the runtime (which required a DataHaven metadata
update).

(cherry picked from commit 5f2b366031)
2025-12-04 21:39:40 +01:00
Steve Degosserie
f0e5bd449b
feat: RT810 runtime release (#339)
This PR bumps the runtime version to RT810 for an upcoming runtime
hotfix release (base branch is perm-runtime-rt800).
2025-12-04 17:19:58 +01:00
Gonza Montiel
e1e566568d
fix: update weights for RT800 (#329)
Weights update for release v0.8.0 (RT800).

- [x] Stagenet
- [x] Testnet
- [x] Mainnet

Showing some +10% changes:

Network | Pallet | Call | Old weight | New weight | Change %
-- | -- | -- | -- | -- | --
mainnet | frame_system | authorize_upgrade | 12496000 | 22629000 |
81.09%
mainnet | frame_system | remark | 30798441 | 8695261 | -71.77%
mainnet | pallet_conviction_voting | undelegate | 21395950 | 23795285 |
11.21%
mainnet | pallet_external_validators | remove_whitelisted | 17773467 |
15404854 | -13.33%
mainnet | pallet_multisig | as_multi_complete | 58059065 | 50189845 |
-13.55%
mainnet | pallet_preimage | request_no_deposit_preimage | 18808000 |
29216000 | 55.34%
mainnet | pallet_preimage | request_preimage | 26608000 | 40720000 |
53.04%
mainnet | pallet_preimage | request_requested_preimage | 13880000 |
16912000 | 21.84%
mainnet | pallet_preimage | request_unnoted_preimage | 19245000 |
33023000 | 71.59%
mainnet | pallet_preimage | unnote_no_deposit_preimage | 32229000 |
46218000 | 43.41%
mainnet | pallet_preimage | unnote_preimage | 70700000 | 91185000 |
28.97%
mainnet | pallet_preimage | unrequest_multi_referenced_preimage |
13961000 | 16754000 | 20.01%
mainnet | pallet_preimage | unrequest_preimage | 27274000 | 42435000 |
55.59%
mainnet | pallet_preimage | unrequest_unnoted_preimage | 13946000 |
16113000 | 15.54%
mainnet | pallet_scheduler | service_task_fetched | 24397000 | 4128742 |
-83.08%
mainnet | pallet_sudo | check_only_sudo_account | 5248000 | 6034000 |
14.98%
mainnet | pallet_utility | batch | 18376045 | 14956939 | -18.61%
mainnet | pallet_utility | batch_all | 13623778 | 9310421 | -31.66%
mainnet | pallet_utility | force_batch | 13126526 | 11589563 | -11.71%
stagenet | frame_system | authorize_upgrade | 12866000 | 23295000 |
81.06%
stagenet | frame_system | remark | 33694157 | 25734885 | -23.62%
stagenet | pallet_external_validators_rewards | on_era_end | 1583951000
| 1749571000 | 10.46%
stagenet | pallet_multisig | as_multi_complete | 57510735 | 51491532 |
-10.47%
stagenet | pallet_preimage | request_no_deposit_preimage | 18612000 |
30396000 | 63.31%
stagenet | pallet_preimage | request_preimage | 26129000 | 37317000 |
42.82%
stagenet | pallet_preimage | request_requested_preimage | 13912000 |
17268000 | 24.12%
stagenet | pallet_preimage | request_unnoted_preimage | 19198000 |
35428000 | 84.54%
stagenet | pallet_preimage | unnote_no_deposit_preimage | 31195000 |
45321000 | 45.28%
stagenet | pallet_preimage | unnote_preimage | 70675000 | 89916000 |
27.22%
stagenet | pallet_preimage | unrequest_multi_referenced_preimage |
13422000 | 16452000 | 22.57%
stagenet | pallet_preimage | unrequest_preimage | 27349000 | 43421000 |
58.77%
stagenet | pallet_preimage | unrequest_unnoted_preimage | 14104000 |
16640000 | 17.98%
stagenet | pallet_sudo | check_only_sudo_account | 5247000 | 5923000 |
12.88%
stagenet | pallet_utility | batch | 17077975 | 10949120 | -35.89%
stagenet | pallet_utility | batch_all | 11105082 | 4628520 | -58.32%
stagenet | pallet_utility | force_batch | 9291805 | 4409497 | -52.54%
testnet | frame_system | authorize_upgrade | 12576000 | 24096000 |
91.60%
testnet | frame_system | remark | 49594571 | 14515706 | -70.73%
testnet | pallet_external_validators_rewards | on_era_end | 1547405000 |
1732185000 | 11.94%
testnet | pallet_preimage | request_no_deposit_preimage | 17857000 |
30407000 | 70.28%
testnet | pallet_preimage | request_preimage | 25375000 | 42504000 |
67.50%
testnet | pallet_preimage | request_requested_preimage | 14069000 |
16591000 | 17.93%
testnet | pallet_preimage | request_unnoted_preimage | 19174000 |
34375000 | 79.28%
testnet | pallet_preimage | unnote_no_deposit_preimage | 29893000 |
46805000 | 56.58%
testnet | pallet_preimage | unnote_preimage | 71971000 | 89858000 |
24.85%
testnet | pallet_preimage | unrequest_multi_referenced_preimage |
13934000 | 16085000 | 15.44%
testnet | pallet_preimage | unrequest_preimage | 28700000 | 43751000 |
52.44%
testnet | pallet_preimage | unrequest_unnoted_preimage | 13989000 |
17138000 | 22.51%
testnet | pallet_sudo | check_only_sudo_account | 5279000 | 5881000 |
11.40%
testnet | pallet_utility | batch | 17940544 | 20806579 | 15.98%
testnet | pallet_utility | batch_all | 20328522 | 18215679 | -10.39%
testnet | pallet_utility | force_batch | 13240132 | 18456500 | 39.40%

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-12-02 22:46:27 +01:00
Facundo Farall
c1202e5d51
build: ⬆️ Upgrade to SH release 0.2.1 (#331) 2025-12-02 21:38:59 +01:00
Steve Degosserie
51ffcae5f0
Revert "feat: statically build binary (#292)" (#330)
This reverts commit f84b6debb7.
2025-12-02 15:42:43 +01:00
Steve Degosserie
e38843455b
fix: 🔨 Disable static binary build for now (#328) 2025-12-02 14:53:05 +01:00
Gonza Montiel
82c581d495
feat: Add datahaven native transfer precompile (#309)
## DataHaven Native Transfer Precompile

Implements EVM precompile at address
`0x00000000000000000000000000000007F5` (2073) to expose
`pallet-datahaven-native-transfer` functionality to the EVM layer.

### Features
- **Transfer to Ethereum**: Locks native tokens and sends them via
Snowbridge to Ethereum addresses
- **Pause/Unpause**: Admin controls to halt/resume transfers
- **View Functions**: Query paused state, total locked balance, and
sovereign account address

### Implementation
- Precompile using `#[precompile_utils::precompile]` macro with proper
gas accounting
- 15+ test cases covering success/failure scenarios
- Solidity interface with NatSpec documentation for contract integration

Enables seamless cross-chain transfers of DataHaven native tokens to
Ethereum L1.

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-12-02 13:57:40 +01:00
Steve Degosserie
063773eb05
fix: 🔨 Update Snowbridge Ethereum network configuration with correct genesis hashes and fork versions (#310)
## Summary

This PR fixes the Snowbridge Ethereum network configuration across all
three runtimes (Mainnet, Stagenet, and Testnet) by replacing placeholder
genesis hashes with correct values and updating fork versions to match
the target Ethereum networks.

### Changes Made

#### **Stagenet Runtime**
-  Updated genesis hash from `[3u8; 32]` to Hoodi testnet genesis:
`0xbbe312868b376a3001692a646dd2d7d1e4406380dfd86b98aa8a34d1557c971b`
-  Fixed chain ID from `3151908` to `560048` (Hoodi testnet)
-  Updated fork versions to Hoodi testnet configuration (0x10000910
series)
-  Added Fulu fork at epoch 50688 (activated Oct 28, 2025)

#### **Testnet Runtime**
-  Updated genesis hash from `[2u8; 32]` to Hoodi testnet genesis:
`0xbbe312868b376a3001692a646dd2d7d1e4406380dfd86b98aa8a34d1557c971b`
-  Updated fork versions from Holesky to Hoodi testnet configuration
-  Added Fulu fork at epoch 50688

#### **Mainnet Runtime**
-  Updated genesis hash from `[1u8; 32]` to Ethereum mainnet genesis:
`0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3`
-  Updated fork versions from Holesky to Ethereum mainnet configuration
(0x00000000 series)
-  Added Fulu fork at epoch 411392 (scheduled for Dec 3, 2025)

#### **Core Updates**
-  Added `fulu` field to `ForkVersions` struct in
`snowbridge-beacon-primitives`
-  Updated `select_fork_version` function in ethereum-client pallet to
handle Fulu fork

### Technical Details

**Hoodi Testnet (Stagenet & Testnet):**
- Chain ID: 560048
- Genesis:
`0xbbe312868b376a3001692a646dd2d7d1e4406380dfd86b98aa8a34d1557c971b`
- Fork versions: Genesis (0x10000910) → Altair → Bellatrix → Capella →
Deneb → Electra (epoch 2048) → Fulu (epoch 50688)
- Source: https://github.com/eth-clients/hoodi

**Ethereum Mainnet:**
- Chain ID: 1
- Genesis:
`0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3`
- Fork versions: Genesis (0x00000000) → Altair (epoch 74240) → Bellatrix
(144896) → Capella (194048) → Deneb (269568) → Electra (364032) → Fulu
(411392)
- Source: https://github.com/ethereum/consensus-specs

### Why This Matters

The Snowbridge light client relies on accurate Ethereum network
parameters to verify consensus proofs. Incorrect genesis hashes or fork
versions would cause signature verification failures and prevent the
bridge from functioning correctly.

### Fulu Fork Support

The Fulu fork (part of Fusaka upgrade) adds a `proposer_lookahead` field
to BeaconState but does **not** change generalized indices for sync
committees or finalized roots, so it reuses Electra's configuration
constants.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
2025-12-02 12:49:18 +01:00
Facundo Farall
4d5716fdcb
build(operator): ⬆️ Upgrade to storage-hub 0.2.0 (#327)
## ⚠️ Breaking Changes ⚠️

Upgrades to SH version 0.2.0. Breaking changes for this version are
outlined in the corresponding
[release](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.2.0).

Particularly, in this PR, the following breaking changes are implemented
for DH node operators:

### Breaking CLI changes vs `main`

- **Fisherman vs provider role**  
- `--fisherman` now has `conflicts_with = "provider"`
(`FishermanConfigurations::fisherman`).
- Any existing scripts that started a node with both `--provider` and
`--fisherman` will now fail clap validation.

- **Removed / replaced fisherman tuning flags**  
- The following flags no longer exist and will cause errors if still
used:
- `--fisherman-incomplete-sync-max` (field
`fisherman_incomplete_sync_max`)
- `--fisherman-incomplete-sync-page-size` (field
`fisherman_incomplete_sync_page_size`)
- `--fisherman-sync-mode-min-blocks-behind` (field
`fisherman_sync_mode_min_blocks_behind`)
  - They are replaced by:  
- `--fisherman-batch-interval-seconds`
(`fisherman_batch_interval_seconds`, default `60`)
- `--fisherman-batch-deletion-limit` (`fisherman_batch_deletion_limit`,
default `1000`)

- **MSP DB wiring no longer piggybacks on the indexer DB**  
- Previously, enabling the indexer (`IndexerConfigurations`) also wired
its DB pool into the MSP move‑bucket path via
`with_indexer_db_pool(maybe_indexer_db_pool)`.
- Now, MSP DB access is **only** configured if you pass the new
`--msp-database-url` provider flag; the indexer’s `--indexer` /
`--indexer-database-url` no longer implicitly provide DB access to MSP
logic. This will change behaviour for MSP nodes that relied on just the
indexer flags.

### New / additive CLI flags (non‑breaking but behaviourally relevant)

- **Provider flags**  
- `--pending-db-url` (`pending_db_url`, env `SH_PENDING_DB_URL`) for
persisting pending extrinsics.
- `--internal-buffer-size` (`internal_buffer_size`, default `1024`) for
DB chunk batching during file transfer.

- **Reordered but unchanged**  
- `--msp-distribute-files` still exists (bool flag), just moved within
`ProviderConfigurations`; name and type are unchanged, but now also
explicitly toggles `enable_msp_distribute_files` only when
`provider_type == msp`.
2025-12-02 11:55:31 +01:00
undercover-cactus
f84b6debb7
feat: statically build binary (#292)
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
2025-11-28 13:38:05 +00:00
Ahmad Kaouk
b737bc03ba
refactor: remove BSP and MSP operator sets (#323) 2025-11-28 14:01:28 +01:00
Ahmad Kaouk
ffd01d8f1d
Fix: command cli deploy contracts (#319)
## Summary

This PR fixes several issues with the CLI deploy-contracts command to
properly support local Anvil deployments and improves the overall
contract deployment workflow.

  ### Key fixes:
  - Add support for anvil chain in the CLI deploy contracts command
- Rename PRIVATE_KEY to DEPLOYER_PRIVATE_KEY for consistency and clarity
across the deployment flow
- Fix EigenLayer contract status display for local/anvil chains by
reading addresses from the deployments file instead of config
- Fix runShellCommandWithLogger to properly throw errors on command
failure
  - Correct totalSteps in DeployTestnet.s.sol from 2 to 4

  ### Housekeeping:
- Update .gitignore to ignore the entire broadcast/ folder
(autogenerated Foundry artifacts)
- Streamline contracts/README.md with clearer structure and deployment
instructions
2025-11-27 15:06:04 +01:00
Ahmad Kaouk
0618e84268
feat: set POV gas limit ratio to zero for solo chain (#313)
Set `GasLimitPovSizeRatio` to 0 across all runtime environments
(mainnet, stagenet, testnet) since DataHaven operates as a solo chain
and doesn't need to account for Proof-of-Validity size constraints that
parachains require.

  ## ⚠️ Breaking Changes ⚠️
  - `GasLimitPovSizeRatio` is now set to 0 across all runtimes
  - Gas calculations will no longer account for POV size constraints
2025-11-27 11:14:41 +01:00
Steve Degosserie
71b5e5185f
fix: consolidate session timing and simplify docker release workflow (#321)
## Summary

- Consolidates `SessionsPerEra` definition in common runtime (removes
duplicate definitions)
- Simplifies docker release workflow to always use full Docker builds
- Removes binary reuse path from release workflow

## Changes

### Runtime Configuration
- Remove duplicate `SessionsPerEra` definitions from individual runtimes
- Import `SessionsPerEra` from `datahaven_runtime_common::time` instead
- This fixes inconsistency where individual runtimes had
`prod_or_fast!(6, 1)` while common had `prod_or_fast!(6, 3)`

### Docker Release Workflow
- Remove binary reuse path - now always does full Docker build
- Remove `binary-hash` input from `workflow_call`
- Consolidate to single build step using `datahaven-build.Dockerfile`
- `docker-build-release` now runs in parallel on main branch (no
dependency on `build-operator`)

## Timing Configuration

### Production Runtime
| Parameter        | Value       | Duration   |
|------------------|-------------|------------|
| Session          | 600 blocks  | 1 hour     |
| Sessions per era | 6           | -          |
| Era              | 6 sessions  | 6 hours    |
| Bonding duration | 28 eras     | 7 days     |

### Fast Runtime (for testing)
| Parameter        | Value       | Duration   |
|------------------|-------------|------------|
| Session          | 10 blocks   | 1 minute   |
| Sessions per era | 1           | -          |
| Era              | 1 session   | 1 minute   |
| Bonding duration | 3 eras      | 3 minutes  |

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-26 10:25:24 +01:00
Steve Degosserie
92d3c42f79
chore: ♻️ Remove executed runtime migrations (#318)
## Summary

Removes old runtime migrations that have already been executed on
Stagenet and Testnet environments, reducing code complexity and
maintenance burden.

## Changes

### Migration Cleanup
- **Removed `evm_alias::EvmAliasMigration`** (~532 lines)
- Multi-block migration that renamed the Frontier EVM pallet alias from
`Evm` to `EVM`
  - Migrated AccountCodes, AccountCodesMetadata, and AccountStorages
  
- **Removed `evm_chain_id::EvmChainIdMigration`**
- Single-step migration that updated stored EVM chain IDs to match new
configuration
  - Applied to testnet (55931) and stagenet (55932)

### Runtime Updates
- **Simplified `MultiBlockMigrationList`** to empty tuple `()` in
`runtime/common/src/migrations.rs`
- **Updated all runtime configs** to use simplified migration list:
  - `runtime/mainnet/src/configs/mod.rs`
  - `runtime/stagenet/src/configs/mod.rs`
  - `runtime/testnet/src/configs/mod.rs`
- Removed `Runtime` type parameter from migration configurations

### What Remains
- `pallet_migrations` infrastructure stays in place for future
migrations
- Migration test file (`mainnet/tests/migrations.rs`) preserved for
testing pallet administrative functions
- Configuration types and constants (cursor/identifier lengths,
handlers)

## Impact

- **Code reduction**: 532 lines removed
- **No functional change**: These migrations have already executed
successfully
- **Future-ready**: Migration infrastructure remains for new migrations
when needed

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-11-25 18:44:06 +01:00
Steve Degosserie
72c3457183
fix: 🔨 Replace FreezeChainOnFailedMigration with SafeMode handler (#308)
## Summary

Replaced the `DefaultFailedMigrationHandler` (which completely froze the
chain on migration failures) with `EnterSafeModeOnFailedMigration`
across all three runtimes (mainnet, stagenet, testnet). When a migration
fails, the chain now automatically enters SafeMode instead of freezing,
allowing governance to intervene and fix issues while preventing regular
user transactions.

## Problem

Previously, when a runtime migration failed, the chain would use
`FreezeChainOnFailedMigration`, which completely halted all operations
including governance functions. This made it impossible to recover from
migration failures without manual intervention at the node level.

## Solution

Implemented `EnterSafeModeOnFailedMigration` which:
- **Enters SafeMode** when a migration fails: the chain remains
_indefinitely_ under safe mode until it is disabled, either with Sudo or
Governance.
- **Allows governance operations** to continue (Sudo, SafeMode, TxPause,
Preimage, Scheduler, etc.)
- **Blocks regular user transactions** to prevent interaction with
potentially inconsistent storage
- **Falls back to freezing** if SafeMode cannot be entered

## Changes

### Core Implementation
- **`runtime/common/src/migrations.rs`**: Added
`FailedMigrationHandler<SafeMode>` type alias that wraps
`EnterSafeModeOnFailedMigration` with comprehensive documentation
- **All three runtimes** (`mainnet`, `stagenet`, `testnet`):
- Updated `pallet_migrations::Config::FailedMigrationHandler` to use
`FailedMigrationHandler<SafeMode>`
  - Removed obsolete TODO comments

### Tests
Added comprehensive migration failure tests to all three runtimes:
- **`failed_migration_enters_safe_mode`**: Verifies SafeMode is
activated, expiry is set, and event is emitted
- **`safe_mode_allows_governance_during_migration_failure`**: Confirms
governance can exit SafeMode after migration failure
- **`migrations_force_calls_are_root_only`**: Existing test for
migration management permissions

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-11-25 16:09:19 +01:00
Steve Degosserie
79339c5c3d
fix: 🔨 Enable BEEFY equivocation reporting in all runtimes (#320)
## Summary

- Enables BEEFY equivocation reporting which was previously disabled
(set to `()`)
- Configures `pallet_beefy::EquivocationReportSystem` in all three
runtimes (mainnet, stagenet, testnet)
- Without this fix, validators could sign conflicting BEEFY commitments
without any slashing consequences

## Problem

The `EquivocationReportSystem` type in `pallet_beefy::Config` was set to
`()`, which completely disabled BEEFY equivocation reporting. This is a
security issue because:

1. BEEFY validators could sign two different commitments at the same
block height (equivocation)
2. There was no mechanism to report and slash such misbehavior
3. This undermines the security guarantees of the BEEFY consensus
protocol

## Solution

Configure the proper equivocation report system using the same pattern
as BABE and GRANDPA:

```rust
type EquivocationReportSystem =
    pallet_beefy::EquivocationReportSystem<Self, Offences, Historical, ReportLongevity>;
```

This uses:
- `Offences` pallet to record equivocations
- `Historical` pallet for validator proof verification  
- `ReportLongevity` parameter (based on bonding duration) for the
reporting window

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-25 14:56:52 +01:00
Gonza Montiel
82c9be6af1
fix: build contracts with test/mocks (#317)
Apparently, EigenLayer started including mock files inside
`test/mocks/*`, that weren't in our remappings. That's why the e2e
contract building was failing, both for e2e tests and CLI launch, while
trying to solve `src/test/mocks/AllocationManagerMock.sol` in our own
contracts source. I added the necessary remappings and the tests run
now.
2025-11-24 17:36:29 +01:00
undercover-cactus
53d209bbae
test: only inject contracts in e2e tests if INJECT_CONTRACTS env is 'true' (#315)
In this PR we add an environment variable `INJECT_CONTRACTS`. This
environment variable specify if the contracts should be injected in the
e2e tests. By default it is false. The environment variable is set to
`true` in the CI job that run the e2e tests.

We are using a environment variable because `bun test` doesn't allow for
passing extra arguments.

A note about the new variable has been added in the documentation to
inform about the new behavior.

---------

Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
2025-11-24 12:07:36 +01:00
Steve Degosserie
ba1cc63cb0
fix: 🔨 Run publish binary task on ephemeral runner (#307)
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-11-22 15:54:49 +01:00
Steve Degosserie
a5522659bf
feat: Add Docker Compose setup for local DataHaven network (#314)
## 🎯 Overview

This PR introduces a comprehensive Docker Compose configuration for
running a complete local DataHaven network, making it significantly
easier for developers to spin up and test the entire stack locally.

## 🏗️ Architecture

```mermaid
graph TB
    subgraph "DataHaven Network (Docker)"
        subgraph "Consensus Layer"
            Alice["🔷 Alice (Validator)<br/>:9944 RPC<br/>4 Keys: GRAN, BABE, IMON, BEEF"]
            Bob["🔷 Bob (Validator)<br/>:9945 RPC<br/>4 Keys: GRAN, BABE, IMON, BEEF"]
            Alice <-->|P2P/mDNS| Bob
        end

        subgraph "Storage Provider Layer"
            MSP["💾 MSP (Charlie)<br/>:9946 RPC<br/>1 GiB Storage<br/>1 Key: BCSV"]
            BSP01["💾 BSP01 (Dave)<br/>:9947 RPC<br/>1 GiB Storage<br/>1 Key: BCSV"]
            BSP02["💾 BSP02 (Eve)<br/>:9948 RPC<br/>1 GiB Storage<br/>1 Key: BCSV"]
        end

        subgraph "Monitoring Layer"
            Indexer["📊 Indexer<br/>:9949 RPC<br/>Full Mode<br/>No Keys"]
            Fisherman["🎣 Fisherman (Gustavo)<br/>:9950 RPC<br/>Storage Monitor<br/>1 Key: BCSV"]
            DB["🗄️ PostgreSQL<br/>:5432<br/>indexer/datahaven"]
        end

        Alice -.->|Syncs| MSP
        Bob -.->|Syncs| MSP
        Alice -.->|Syncs| BSP01
        Bob -.->|Syncs| BSP02
        Alice -.->|Syncs| Indexer
        Bob -.->|Syncs| Fisherman
        
        MSP -.->|Monitors| Fisherman
        BSP01 -.->|Monitors| Fisherman
        BSP02 -.->|Monitors| Fisherman
        
        Indexer -->|Writes| DB
        Fisherman -->|Writes| DB
    end

    style Alice fill:#4a90e2,stroke:#2e5c8a,color:#fff
    style Bob fill:#4a90e2,stroke:#2e5c8a,color:#fff
    style MSP fill:#50c878,stroke:#2d7a4a,color:#fff
    style BSP01 fill:#50c878,stroke:#2d7a4a,color:#fff
    style BSP02 fill:#50c878,stroke:#2d7a4a,color:#fff
    style Indexer fill:#f5a623,stroke:#b87818,color:#fff
    style Fisherman fill:#f5a623,stroke:#b87818,color:#fff
    style DB fill:#9b59b6,stroke:#6c3a82,color:#fff
```

**Legend:**
- 🔷 Validators - Consensus and block production
- 💾 Storage Providers - File storage and retrieval
- 📊 Indexer - Full blockchain indexing
- 🎣 Fisherman - Storage provider monitoring
- 🗄️ PostgreSQL - Database for indexer/fisherman
- `<-->` P2P Communication | `-.->` Network Sync | `-->` Database
Connection

##  What's Included

### Core Network (8 Services)
- **2 Validator Nodes** (Alice & Bob) - Consensus and block production
- **1 Main Storage Provider (MSP)** - Charlie with 1 GiB storage
capacity
- **2 Backup Storage Providers (BSPs)** - Dave and Eve with 1 GiB each
- **1 StorageHub Indexer** - Full blockchain indexer with PostgreSQL
- **1 Fisherman Node** - Gustavo monitoring storage provider behavior
- **1 PostgreSQL Database** - Shared database for indexer and fisherman

### Key Features
 **Automated Key Injection** - All validator and storage provider keys
automatically injected on startup
 **Health Checks** - Validators have health checks that verify RPC port
is listening before dependent services start
 **Orchestrated Startup** - Services start in correct order with
health-based dependencies
 **Persistent Storage** - Chain data, keystores, and database all
persisted in Docker volumes
 **Unified Entrypoint Script** - Single script handles all node types
(validator, MSP, BSP, fisherman)
 **Proper User Permissions** - Root for setup, switches to datahaven
user for node execution
 **mDNS Peer Discovery** - Nodes automatically discover each other on
the Docker network
 **Comprehensive Documentation** - Full setup guide, troubleshooting,
and verification steps

### 🔄 Startup Sequence

The network starts in a carefully orchestrated sequence to ensure
stability:

1. **Alice (Validator)** starts first
   - Injects 4 keys (GRAN, BABE, IMON, BEEF)
   - Health check waits for RPC port 9944 to be listening
   - Uses `/proc/net/tcp` for minimal-dependency port checking

2. **Bob (Validator)** waits for Alice to be healthy
   - Ensures at least one validator is fully operational
   - Enables block production to start immediately

3. **Storage Providers & Monitoring** wait for both validators to be
healthy
   - MSP, BSP01, BSP02 start after validators are ready
   - Indexer and Fisherman wait for validators before syncing
   - PostgreSQL starts independently with its own health check

This dependency chain prevents race conditions and ensures reliable
network formation.

## 🚀 Quick Start

```bash
cd operator

# Build the binary (development mode with fast blocks)
./scripts/docker-prepare.sh --fast

# Start the entire network
docker-compose up -d

# View logs
docker-compose logs -f

# Check service health
docker-compose ps

# Stop the network
docker-compose down -v
```

## 📋 Port Mappings

| Service | RPC/WebSocket | Prometheus | P2P | Database/API |
|---------|---------------|------------|-----|--------------|
| Alice | 9944 | 9615 | 30333 | - |
| Bob | 9945 | 9616 | 30334 | - |
| MSP | 9946 | 9617 | 30335 | - |
| BSP01 | 9947 | 9618 | 30336 | - |
| BSP02 | 9948 | 9619 | 30337 | - |
| Indexer | 9949 | 9620 | 30338 | - |
| Fisherman | 9950 | 9621 | 30339 | - |
| PostgreSQL | - | - | - | 5432 |

## 🔑 Key Injection

All cryptographic keys are automatically injected on startup:

**Validators (Alice & Bob)** - 4 keys each:
- GRANDPA (ed25519) - Finality
- BABE (sr25519) - Block authoring
- ImOnline (sr25519) - Heartbeat
- BEEFY (ecdsa) - Bridge consensus

**Storage Providers (MSP, BSPs, Fisherman)** - 1 key each:
- BCSV (ecdsa) - Storage provider identity

**Indexer** - No keys required (non-validating node)

## 🩺 Health Checks

Validator nodes (Alice & Bob) implement health checks to ensure proper
startup sequencing:

**Health Check Method:**
- Reads `/proc/net/tcp` directly to check if RPC port is listening
- Zero dependencies - works in minimal debian:stable-slim containers
- Converts port to hex and searches for LISTEN state (0A)

**Configuration:**
- Start period: 30s (allows node initialization)
- Interval: 10s (check every 10 seconds)
- Timeout: 5s (per health check)
- Retries: 5 (must pass 5 consecutive checks)

**Benefits:**
- Prevents dependent services from starting before validators are ready
- Eliminates race conditions during network formation
- No additional packages required in Docker image

## 🍎 macOS Requirement

**Important:** On Docker Desktop for macOS, you must use the
**experimental DockerVMM** virtualization framework:

1. Open Docker Desktop settings
2. Go to "General" tab
3. Enable "Use experimental virtualization framework (DockerVMM)"
4. Restart Docker Desktop

The default Apple Virtualization Framework causes networking issues with
P2P connections.

## 📁 Files Changed

- `operator/docker-compose.yml` - Main orchestration configuration
- `operator/scripts/docker-entrypoint.sh` - Unified key injection script
- `operator/scripts/docker-prepare.sh` - Binary build helper
- `operator/scripts/docker-healthcheck.sh` - Health check script for
validators
- `operator/DOCKER-COMPOSE.md` - Comprehensive documentation

## 🔍 Testing

The configuration has been tested on:
-  Docker Desktop for macOS (with DockerVMM)
-  Docker on Linux/Ubuntu

All nodes successfully:
- Inject required keys
- Pass health checks
- Discover peers via mDNS
- Sync blocks and finalize
- Connect to PostgreSQL database (indexer/fisherman)

## 📝 Notes

- All settings are configured for **local development only**
- Uses well-known test seed phrase (⚠️ never use in production!)
- RPC exposed without authentication
- Unsafe flags enabled for convenience

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-22 14:07:46 +01:00
Gonza Montiel
6dae38f587
feat: 🚀 add storage-hub nodes to CLI (#287)
##  Add StorageHub nodes to CLI

This PR adds StorageHub node infrastructure to the CLI and fixes CLI
flag handling and improves the CLI logic a bit.

### Fix: CLI safeguards
- Prevents contract deployment and validator operations when `--ndc`
flag is used
- Skips Kurtosis service display in summary when `--nlk` flag is used

### Feat: Dockerized Storage Hub Nodes
- Adds 5 Docker containers: PostgreSQL, MSP, BSP, Indexer, and Fisherman
nodes
- New CLI flags: `--storagehub` `--no-storagehub` to control StorageHub
nodes launch
- Automatic provider funding and registration (Charleth for MSP and
Dorothy for BSP)
- Exposes nodes on ports 9945-9948 for local development

**TODO**
- [x] MSP & BSP associated pre-funded account.
- [x] Call `forceMspSignUp` and `forceBspSignUp` extrinsics

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-22 11:49:14 +01:00
Steve Degosserie
7f09949e64
feat: Implement inflation mechanism for validator rewards (#304)
## Summary

This PR introduces a configurable inflation system for validator rewards
with an annual target rate and optional treasury allocation.

## Changes

### Inflation Mechanism
- **Annual inflation rate runtime parameter**: Set to 5% default
- **EraInflationProvider**: Calculates per-era inflation based on total
issuance and annual rate
- Formula: `per_era_inflation = (total_issuance × annual_rate) /
eras_per_year`

### Treasury Allocation
- **InflationTreasuryProportion parameter**: Set to 20% default
- **ExternalRewardsInflationHandler**: Mints inflation and distributes
between:
  - 80% to rewards account (for validator rewards)
  - 20% to treasury account
- Treasury receives allocation via `mul_floor()`, with remainder going
to rewards to ensure no tokens lost to rounding

### Runtime Integration
- Configured across all three runtimes: mainnet, testnet, and stagenet
- Consistent parameters across all environments

### Testing
- Updated all tests to account for 80/20 split between rewards and
treasury
- Added precision tolerance (±1 unit) for Perbill rounding edge cases

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-22 11:00:50 +01:00
Ahmad Kaouk
2cc1a4d3f0
test: port ethereum tests from moonbeam (#278)
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-22 10:02:05 +01:00
Steve Degosserie
0fa701f900
fix: 🔨 Don't run Moonwall tests in parallel (#316) 2025-11-22 01:07:54 +01:00
undercover-cactus
077cc9ed29
feat: injecting contracts feature for e2e testing (#295)
In this PR, we introduce a way to save Ethereum state into a file. This
saved state can then be injected into Ethereum to speed up e2e initial
test setup.

This is a rewrite of the now closed PR
https://github.com/datahaven-xyz/datahaven/pull/90 .

It uses a an external tool written in rust to save state from the
Ethereum running container : https://github.com/undercover-cactus/Chaos

---------

Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-20 12:42:12 +01:00
Steve Degosserie
32480f1bbc
feat: Bump client version to v0.8.0 & Runtime version to RT800 (#312) 2025-11-19 18:30:02 +00:00
Ahmad Kaouk
7e6033097e
doc: Fix fast-runtime documentation (#311)
`fast-runtime` features does not shortens block time to 3 s. It keeps it at 6 s and instead shortens epochs (1‑minute) and eras (3 sessions) to speed up validator churn and testing workflows.
2025-11-19 17:23:10 +00:00
Steve Degosserie
9ecad7f119
fix: 🔨 Use correct TreasuryAccount in StorageHub pallets configuration (#305) 2025-11-18 20:57:26 +01:00
Steve Degosserie
bb410e0fa9
fix: 🔨 Correctly set the offences pallet's OnOffenceHandler to the external-validator-slashes pallet (#303) 2025-11-18 08:51:06 +01:00
Facundo Farall
ae9eef7307
build: ⬆️ Upgrade to StorageHub release 0.1.4 (#298)
Upgrades to StorageHub release
[v0.1.4](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.1.4).

## ⚠️ Breaking Changes ⚠️
- A DB migration for the indexer DB. Should be auto-applied by the
indexer node on startup, if this is not disabled by the env var
`SH_INDEXER_DB_AUTO_MIGRATE`. By default, it applies them.
- A new runtime API (`shp_tx_implicits_runtime_api::TxImplicitsApi`)
needed for StorageHub's Blockchain Service to build transactions using
the runtime spec version from the currently run runtime.

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-16 16:44:17 +01:00
Steve Degosserie
4c8384499f
feat: Bump client version to v0.7.0 & Runtime version to RT700 (#299) 2025-11-16 11:43:11 +01:00
Steve Degosserie
37a4ba990f
fix: 🪳 Quick CI build fix (#300) 2025-11-15 12:25:15 +01:00
Steve Degosserie
a3659548c8
fix: 🪳 Add --msp-distribute-files flag to MSP node in Devnet config (#297)
.. and remove an unused type in client.
2025-11-13 14:09:34 +01:00
Ahmad Kaouk
c22297ccaf
fix: use binary search for eth gas estimate (#296)
## Summary

- Build the node with Frontier’s rpc-binary-search-estimate feature so
eth_estimateGas runs the same iterative search as Moonbeam.
- Instead of returning the gas spent by a single max-allowance dry run,
the RPC now repeatedly replays the transaction while shrinking the gas
cap until it finds the smallest limit that still succeeds.
2025-11-12 18:23:08 +01:00
Ahmad Kaouk
61a5cbab51
fix: bun cli launch command fails with locally built images (#282)
## Changes

Fixes `bun cli launch --all` command failing when using locally built
Docker images.

### What changed
- Check local Docker images first before querying Docker Hub in
`checkTagExists`

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-10 23:52:45 +01:00
Ahmad Kaouk
dd7b72ca29
chore: pin Bun version and migrate to bun.lock (#290)
## Summary

Pins Bun version to 1.3.2 and migrates workflows to use text-based
`bun.lock` instead of binary `bun.lockb`. This fixes CI failures caused
by Bun version mismatches between local development and GitHub Actions.

## Changes

- Created `test/.bun-version` to pin Bun to v1.3.2
- Updated all workflows to use `bun-version-file: test/.bun-version`
- Migrated workflow cache keys from `bun.lockb` to `bun.lock`
- Removed deprecated `test/bun.lockb` binary lockfile

## Why?

**Version Consistency:**
- Local environments and CI were using different Bun versions
- Different versions generate different lockfile formats → CI failures

**Lockfile Migration:**
- Bun v1.2+ uses text-based `bun.lock` as default
- Binary `bun.lockb` is still supported but deprioritized
- Text format provides better git diffs and merge conflict resolution

## Affected Workflows

- `.github/workflows/task-check-metadata.yml`
- `.github/workflows/task-e2e.yml`
- `.github/workflows/task-moonwall-tests.yml`
- `.github/workflows/task-ts-build.yml`
- `.github/workflows/task-ts-lint.yml`

## After Merge

Developers should upgrade their local Bun:
```bash
bun upgrade --stable  # Should install v1.3.2
bun --version         # Verify version
bun install           # Regenerate lockfile if needed
```

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-10 22:37:39 +01:00
Facundo Farall
0d2333ed02
build: ⬆️ Upgrade to SH release 0.1.3 (#294)
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-11-10 21:58:40 +01:00
Facundo Farall
aaef407eb5
feat: Update SH client initialisation according to new SH releases (#293) 2025-11-10 21:02:39 +01:00
Steve Degosserie
e6cba95563
fix: 🪳 Fix gas cost in collective precompile (#291)
Cherry-pick fix from
https://github.com/moonbeam-foundation/moonbeam/pull/3540 for the
Collective precompile.

> In the `members` and `is_member` functions, the `MaxProposals` value
was being used instead of `MaxMembers` to record gas costs for database
access.
2025-11-10 14:55:57 +01:00
Steve Degosserie
5a7983f0d8
chore: ♻️ Add missing license header in operator & AVS contracts source code (#285)
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
2025-11-10 12:56:41 +01:00
Steve Degosserie
7bd5bc8784
fix: 🔧 Remove slashing pallet runtime upgrade logic (post RT400) (#277)
This PR reverts https://github.com/datahaven-xyz/datahaven/pull/272,
included in
[RT400](https://github.com/datahaven-xyz/datahaven/releases/tag/RT400),
as Stagenet / Testnet were upgraded, and the correct `SlashingMode` is
now set for both.

---------

Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
2025-11-10 12:15:29 +01:00
Facundo Farall
ff5cd8dc59
build: ⬆️ Upgrade to SH v0.1.2 (#289) 2025-11-07 21:34:43 +01:00