Elgato_dark/datahaven
1
0
Fork 0
mirror of https://github.com/datahaven-xyz/datahaven synced 2026-05-24 09:50:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
255 commits 43 branches 56 tags 33 MiB
Commit graph

255 commits

This branch
This branch
All branches
Author SHA1 Message Date
Steve Degosserie
59404351b9
feat: ✨ Bump runtime version to RT510 (#283) 2025-11-07 13:30:09 +01:00
Steve Degosserie
8e1d9c074f
CI: ♻️ Trigger CI actions on perm-* branches (#284) 
(cherry picked from commit 8fa8c18dfd)
 2025-11-07 13:28:08 +01:00
Steve Degosserie
a31118da2b
feat: ✨ Update chain IDs & native token tickers for all 3 environments (#280) 
Update the 3 DataHaven environments' chain IDs & native token ticker as
follows:

* **Mainnet**
  * **Chain ID**: 55930
  * **Ticker**: HAVE
* **TestNet**
  * **Chain ID**: 55931
  * **Ticker**: MOCK
* **Stagenet**
  * **Chain ID**: 55932
  * **Ticker**: STAGE

The PR includes a storage migration for the Stagenet & Testnet
environments, that are already live, to update the EVM Chain ID stored
in the `pallet-evm-chain-id` pallet.

Note: the token symbol will only be updated with the genesis config
presets or newly generated chain specs. For already live networks, the
existing chain spec must be updated (i.e. the tokenSymbol property
changed) and used by all nodes in the network. This change in the chain
spec will not alter the chain genesis so it safe to do (in the very
early stages of the chain obviously).

---------

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit a97f0547a9)
 2025-11-07 12:24:02 +01:00
Steve Degosserie
c09ff91a66
feat: ✨ Bump client version to v0.5.0 & runtime version to RT500 (#274) 2025-11-03 16:35:22 +02:00
Facundo Farall
1fd0abccd7
build: ⬆️ Upgrade to SH v0.1.1 (#273) 
Upgrade to SH release
[v0.1.1](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.1.1)

---------

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
 2025-11-03 15:49:30 +02:00
Gonza Montiel
87eb1c1621
fix: add timestamp safe mode (#267) 
Following up https://github.com/datahaven-xyz/datahaven/pull/265, we
also need to add Timestamp to the whitelisted Runtime calls.

- [x] Add `RuntimeCall::Timestamp` to `SafeModeWhitelistedCalls`
- [x] Add safe mode test to check it produces blocks

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
 2025-11-03 14:00:04 +02:00
undercover-cactus
e248a48385
feat: add Slashing mode has a runtime configurable parameter (#272) 
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
 2025-11-03 11:55:31 +02:00
Steve Degosserie
10a7805648
feat: ✨ Add CI license check (#269) 
## Summary

- Adds automated license compliance checking via GitHub Actions CI
workflow
- Implements a license verification script that validates all Rust
dependencies against approved licenses, authors, and packages
- Standardizes author metadata across Cargo manifests to "Moonsong Labs"

## Changes

**CI Workflow** (`.github/workflows/task-check-licenses.yml`)
- Triggers on pull requests and manual dispatch
- Installs Rust 1.88.0 toolchain and `cargo-license` tool
- Executes license verification script to enforce compliance

**License Verification Script** (`operator/scripts/verify-licenses.sh`)
- Uses `cargo-license` to extract dependency license information
- Maintains three allowlists:
- **Licenses**: Apache-2.0, MIT, BSD variants, GPL-3.0, MPL-2.0, and
compatible combinations
- **Authors**: PureStake, Parity Technologies, Moonsong Labs, Frontier
developers, StorageHub Team
  - **Package Names**: Known safe packages like ring
- Fails the build if any dependency has unapproved license/author/name
combination

**Cargo Manifest Updates**
- `operator/Cargo.toml`: Standardized workspace author to "Moonsong
Labs"
- `operator/precompiles/precompile-registry/Cargo.toml`: Uses workspace
author field
- `operator/runtime/common/Cargo.toml`: Added workspace author field

## Benefits

- **Legal Compliance**: Ensures all dependencies use OSI-approved or
compatible licenses
- **Supply Chain Security**: Validates dependencies come from trusted
sources
- **Automated Enforcement**: Catches licensing issues during PR review
rather than at release time
- **Transparency**: Provides clear audit trail of approved licenses and
authors
 2025-11-02 23:32:59 +02:00
Gonza Montiel
96c4408682
fix: add missing weights (#271) 
### Add missing weights for BABE, GRANDPA, and Randomness

#### Summary
Adds generated weights and wires them into the runtime for the BABE,
GRANDPA, and Randomness pallets to replace defaults and ensure accurate
execution costs across networks.

#### What’s changed
- **New weights added** for `pallet_grandpa`, `pallet_babe` and
`pallet_randomness`
- **Runtime configs updated to use new weights**
  - `operator/runtime/mainnet/src/configs/mod.rs`
  - `operator/runtime/stagenet/src/configs/mod.rs`
  - `operator/runtime/testnet/src/configs/mod.rs`
 
 #### For follow-up PRS
- fix `pallet_identity` failure at running benchmarks
- fix `pallet_collective` benchmarking missmatch (related to
https://github.com/paritytech/polkadot-sdk/pull/6435)
- add `pallet_session_benchmarking` without including `pallet_staking`
(or some workaround)
- add StorageHub weights to our benchmarked pallets (`pallet_nfts`,
`pallet_storage_providers`, `pallet_payment_streams`,
`pallet_proofs_dealer`, `pallet_file_system`, `pallet_bucket_nfts`, etc)
 2025-11-02 22:50:55 +02:00
Steve Degosserie
e860c503c8
feat: ✨ Bump client version to v0.4.0 & runtime version to RT400 (#268) 2025-10-30 18:02:25 +02:00
Ahmad Kaouk
0b636e0d79
fix: Fix EVM gas-to-weight handling for call/create (#266) 
## Summary
- Replace the legacy “estimated transaction length” heuristic in the EVM
`call` runtime API across mainnet, stagenet, and testnet with a direct
`GasWeightMapping::gas_to_weight` lookup. The resulting weight is now
always forwarded to the runner (`Some(weight_limit)`), so zero-gas
requests no longer slip through without a cap.
- Update the EVM `create` runtime API the same way. Previously it always
passed `None` for `weight_limit`, effectively running contract-deploy
dry-runs without any weight ceiling; we now map the gas limit and pass
the explicit weight instead.
- For both `call` and `create`, set the proof-size base cost to `None`
to match our solo-chain assumption that PoV size isn’t budgeted in these
simulated paths.

## Why
We use these runtime APIs when serving `eth_call` and `eth_estimateGas`.
The old behavior meant a zero gas limit (or any `create` dry-run) ran
with unlimited weight, diverging from what the extrinsic path enforces.
Passing the mapped weight—zero included—keeps RPC simulations aligned
with real execution, while dropping the proof-size estimate removes a
guessy value we don’t charge on-chain.
 2025-10-30 17:09:15 +02:00
Ahmad Kaouk
2f6c6e39c2
fix: add explicit sovereign account balance check in unlock_tokens (#253) 
Add defensive validation to ensure the Ethereum sovereign account has
sufficient balance before unlocking tokens. This addresses an audit
finding where the lack of explicit balance checking created an
unreliable security control that depended on implicit runtime behavior.

Changes:
- Add InsufficientSovereignBalance error variant for clear error
messaging
- Add explicit balance check in unlock_tokens before transfer
- Update tests across all runtimes (testnet, stagenet, mainnet) to
validate the specific error is returned when sovereign account has
insufficient funds

The explicit check provides better error messages that can propagate
through the Ethereum bridge and makes debugging sovereign account
balance issues easier.
 2025-10-30 11:19:14 +00:00
Steve Degosserie
839fe6399f
fix: 🔧 Fix datahaven-dev.Dockerfile uid issue (1000 is ubuntu user) (#261) 
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
 2025-10-30 10:10:11 +00:00
Steve Degosserie
45b5551b21
chore: ♻️ Remove unused API declarations in Testnet runtime (#262) 
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
 2025-10-30 09:30:26 +00:00
Gonza Montiel
e079cdc404
fix: 🩹 add mandatory extrinsics to safe mode whitelisted calls (#265) 
# Fix: Safe Mode Whitelisted Calls - enable block production

## Problem
The safe mode whitelist was missing critical runtime calls needed for
block production, generating this error:

```
2025-10-29 17:29:48 Proposing failed: Import failed: Extrinsic is not valid: TransactionValidityError::Invalid(InvalidTransaction::BadMandatory)
```

The SafeMode filter needs to include all RuntimeCalls that have
inherents marked as `DispatchClass::Mandatory`, as you can see
[here](bbc435c766/substrate/frame/executive/src/lib.rs (L806)).

If a single inherent is missing the whole block will not be valid,
causing the chain to stall.

## Solution
Bisect all the calls to find the culprit, until find it was the pallet
Randomness. I included it in `SafeModeWhitelistedCalls` and blocks are
being produced in SafeMode.
 2025-10-30 08:38:18 +00:00
undercover-cactus
7c8227f1ab
feat: set slashing mode in genesis config (#264) 
In this PR we set the slashing mode value in the genesis config. For the
3 different runtime we specify the slashing mode : `mainnet/testnet` is
set to `Disabled` and for `stagenet` to `LogOnly`.

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
 2025-10-29 18:24:49 +02:00
Gonza Montiel
70a505e9d8
fix: update weight configurations for the three runtimes (#249) 
# Update Runtime Weights and WeightInfo Implementations

## Summary
Updated runtime weights across all three runtimes (mainnet, stagenet,
testnet) and configured WeightInfo implementations to use the new
benchmarked weights.

## Changes
- All weight files have been updated with new benchmark data from
Substrate Benchmark CLI v51.0.0
- Updated some WeightInfo configurations to use custom weights instead
of the default `()`, such as `pallet_im_online`.

## Weights comparison against Moonbeam

Some weights are much higher than it's counterpart in Moonbeam, as we
can observe in the following table. We need to investigate if this is
expected.

EDIT:

### Weights using the `production` profile comparing stagenet vs
moonbase

| **Pallet** | **Function** | **DataHaven Weight** | **Moonbase Weight**
| **Difference** | **Notes** |

|------------|--------------|----------------------|---------------------|----------------|-----------|
| **pallet_balances** | transfer_allow_death | 68,497,000 | 43,117,000 |
**+59%** | DataHaven higher |
| | transfer_keep_alive | 55,245,000 | 41,490,000 | **+33%** | DataHaven
higher |
| | force_set_balance_creating | 22,034,000 | 21,537,000 | **+2%** |
Similar |
| | force_set_balance_killing | 31,246,000 | 23,467,000 | **+33%** |
DataHaven higher |
| | force_transfer | 72,181,000 | 45,384,000 | **+59%** | DataHaven
higher |
| | transfer_all | 67,765,000 | 51,403,000 | **+32%** | DataHaven higher
|
| | force_unreserve | 25,876,000 | 19,490,000 | **+33%** | DataHaven
higher |
| | upgrade_accounts (base) | 23,200,000 | 17,599,000 | **+32%** |
DataHaven higher |
| | force_adjust_total_issuance | 8,780,000 | 6,780,000 | **+29%** |
DataHaven higher |
| | burn_allow_death | 43,258,000 | 25,822,000 | **+68%** | DataHaven
higher |
| | burn_keep_alive | 30,368,000 | 22,961,000 | **+32%** | DataHaven
higher |
| **pallet_evm** | withdraw | 3,174,000 | 2,104,000 | **+51%** |
DataHaven higher |
| **frame_system** | remark (base) | 33,694,157 | 38,668,753 | **-13%**
| Moonbase higher |
| | remark_with_event (base) | 8,210,000 | 5,745,000 | **+43%** |
DataHaven higher |
| | set_heap_pages | 5,204,000 | 3,734,000 | **+39%** | DataHaven higher
|
| | set_code | 139,034,247,000 | 102,532,308,000 | **+36%** | DataHaven
higher |
| | set_storage (base) | 3,248,000 | 2,094,000 | **+55%** | DataHaven
higher |
| | kill_storage (base) | 3,344,000 | 2,120,000 | **+58%** | DataHaven
higher |
| | kill_prefix (base) | 6,028,000 | 4,258,000 | **+42%** | DataHaven
higher |
| | authorize_upgrade | 12,866,000 | 8,966,000 | **+44%** | DataHaven
higher |
| | apply_authorized_upgrade | 142,435,318,000 | 106,561,461,000 |
**+34%** | DataHaven higher |
| **pallet_multisig** | as_multi_threshold_1 (base) | 30,107,366 |
17,588,619 | **+71%** | DataHaven higher |
| | as_multi_create (base) | 45,353,453 | 32,782,725 | **+38%** |
DataHaven higher |
| | as_multi_approve (base) | 22,928,808 | 18,299,865 | **+25%** |
DataHaven higher |
| | as_multi_complete (base) | 57,510,735 | 41,555,387 | **+38%** |
DataHaven higher |
| | approve_as_multi_create (base) | 42,860,897 | 32,390,857 | **+32%**
| DataHaven higher |
| | approve_as_multi_approve (base) | 21,640,639 | 16,839,439 | **+29%**
| DataHaven higher |
| | cancel_as_multi (base) | 44,014,197 | 32,643,060 | **+35%** |
DataHaven higher |
| **pallet_treasury** | spend_local | 15,605,000 | 12,992,000 | **+20%**
| DataHaven higher |
| | remove_approval | 8,641,000 | 7,766,000 | **+11%** | DataHaven
higher |
| | on_initialize_proposals (base) | 19,249,676 | 16,973,066 | **+13%**
| DataHaven higher |
| | spend | 14,082,000 | 26,612,000 | **-47%** | Moonbase higher |
| | payout | 74,847,000 | 247,397,000 | **-70%** | Moonbase
significantly higher |
| | check_status | 16,325,000 | 16,223,000 | **+1%** | Similar |
| | void_spend | 15,059,000 | 15,167,000 | **-1%** | Similar |
| **pallet_scheduler** | service_agendas_base | 4,583,000 | 6,215,000 |
**-26%** | Moonbase higher |
| | service_agenda_base (base) | 8,728,153 | 6,640,549 | **+31%** |
DataHaven higher |
| | service_task_base | 4,875,000 | 3,455,000 | **+41%** | DataHaven
higher |
| | service_task_fetched (base) | 24,253,000 | 18,898,000 | **+28%** |
DataHaven higher |
| | service_task_named | 7,176,000 | 5,027,000 | **+43%** | DataHaven
higher |
| | service_task_periodic | 4,842,000 | 3,415,000 | **+42%** | DataHaven
higher |
| | execute_dispatch_signed | 10,378,000 | 4,837,000 | **+115%** |
DataHaven significantly higher |
| | execute_dispatch_unsigned | 3,511,000 | 2,451,000 | **+43%** |
DataHaven higher |
| | schedule (base) | 17,795,637 | 13,165,173 | **+35%** | DataHaven
higher |
| | cancel (base) | 20,509,326 | 15,513,197 | **+32%** | DataHaven
higher |

---------

Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
 2025-10-29 17:14:21 +02:00
undercover-cactus
d748d7d61b
fix: keep slahing mode default to enable (#263) 
To avoid breaking the tests we should not change the default value for
the slashing mode.

Bring back `Enabled` as the default slashing mode value.
 2025-10-29 15:01:10 +01:00
undercover-cactus
f0896907ae
feat: add slashing support (#242) 
## 🔨 Add Slashing Support for Runtime

This PR introduces the slashing functionality for the DataHaven runtime,
enabling punitive measures against misbehaving validators.


### Features
- Deferred slashing with configurable veto periods
- Cross-chain slashing message delivery trough Snowbridge
- Governance controls for slashing parameters and emergency cancellation

We introduced the `external-validator-slashes` pallet, which allows to
slash validators that misbehave. The slashing is triggered when an
offence is reported via the offence pallet (which is already
implemented). The message is sent through Snowbrige's outbound queue and
the real slashing happens in the contracts side, which will come in a
follow up PR.

There is a configurable window of time between the time the validator is
being reported, and the time the slash is triggered. This allows that in
case of an error we are still able to cancel the slashing, using a sudo
account.

For convenience, we also have extrinsics for corner cases:

- **`force_inject_slash`**: Root-only function to manually inject
slashes for specific validators with custom percentages. Useful for
emergency situations or governance-directed slashing outside normal
offence detection
- **`cancel_deferred_slash`**: Allows governance to cancel pending
slashes during the defer period by specifying era and slash indices.
Provides safety mechanism against false positives or malicious slash
reports
- **`set_slashing_mode`**: Configurable slashing behavior with three
modes - `Enabled` (normal operation), `LogOnly` (track offences without
applying slashes), and `Disabled` (completely halt slashing). Critical
for emergency response and testing

---------

Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
 2025-10-29 10:43:55 +00:00
Steve Degosserie
62a4a1fb60
fix: 🔧 Fix e2e test workflow (#260) 2025-10-28 17:43:29 +01:00
Ahmad Kaouk
82c7156fd2
test: port common tests from moonbeam (#223) 
## Overview
Ports essential test suites and helper utilities from Moonbeam's
battle-tested framework to validate DataHaven's EVM compatibility and
precompile functionality.

## What's Included

### 🧪 Test Suites
- **Contract Creation Tests** (`test-contract-creation.ts`)
  - EVM contract deployment verification
  - CREATE/CREATE2 opcode validation  
  - Smart contract nonce management
  - Bytecode storage and retrieval
  - Block fee verification

- **Precompile Batch Tests** (`test-precompile-batch.ts`)
  - Batch operation gas consumption
  - Recursive batch calls
  - Call permit integration
  - EIP-712 signature validation

### 🛠️ Helper Utilities
- **Block helpers**: Fee verification, transaction analysis, block
exploration
- **EVM helpers**: Result validation, signature parsing, receipt
handling
- **Fee helpers**: Treasury/burn split calculations
- **Runtime helpers**: Parameter fetching, versioned constants
- **Contract helpers**: Artifact loading, bytecode management

### 📝 Test Contracts
- `SimpleContractFactory`: CREATE/CREATE2 test harness
- `CallBatchFromConstructor`: Precompile integration tests
 2025-10-28 11:03:40 +00:00
Gonza Montiel
782321e5d0
feat: ✨ Implement dynamic fee adjustment (#251) 
# ✨ Implement Dynamic Fee Adjustment Mechanism

## Overview
Implements a dynamic fee adjustment mechanism, replacing the constant
fee multiplier with an adaptive multiplier that responds to network
congestion, following Moonbeam's pattern.

## Changes
- Replaces `ConstFeeMultiplier` with `TargetedFeeAdjustment` across all
runtime configurations (mainnet, stagenet, testnet)
- Implements an EIP-1559-like slow-adjusting fee mechanism that prevents
DoS attacks by adjusting fees based on block fullness
- **Configurable Parameters**: 
  - Target block fullness: 35%
- Adjustment variable: 4/1000 (responds in ~1 hour at extreme
congestion)
  - Two modes:
    -  `SlowAdjustingFeeUpdate` for mainnet and testnet.
    - `FastAdjustingFeeUpdate` for stagenet.
- Adds tests coverage for different fee scenarios

## Technical Details

The fee adjustment algorithm works as follows:
```
diff = (previous_block_weight - target) / maximum_block_weight
next_multiplier = prev_multiplier * (1 + (v * diff) + ((v * diff)^2 / 2))
assert(next_multiplier > min)
```
**Where:**
- `v` = AdjustmentVariable
- `target` = TargetBlockFullness  
- `min` = MinimumMultiplier

`SlowAdjustingFeeUpdate` sets a minimum multiplier of `1x` for a
conservative fee adjustment, while `FastAdjustingFeeUpdate` sets it to
`0.1x`, which is mainly used for dev networks / testing.
 2025-10-28 10:06:45 +00:00
Steve Degosserie
830d4baf8a
fix: 🔧 Remove redundant operator Dockerfile.local (#257) 
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
 2025-10-27 19:14:13 +01:00
Steve Degosserie
b5bc2de11e
fix: 🔧 Fix incorrect args in release Docker image publishing workflow (#256) 2025-10-27 15:13:14 +02:00
Steve Degosserie
4b0aba0f38
fix: 🔧 Fix incorrect shared lib name in DH operator Docker image (#254) 2025-10-27 13:05:38 +02:00
Steve Degosserie
d55310d1f4
fix: 🔧 Add missing libldap2-dev lib to DH operator Docker image (#252) 2025-10-27 00:33:24 +02:00
Steve Degosserie
879be715b5
feat: ✨ Bump client version to v0.3.1 & runtime spec_version to 310 (#250) 2025-10-24 19:57:07 +03:00
Ahmad Kaouk
911aca1c6f
test: remove redundant E2E test suites (#247) 
Remove 4 test suites that only tested basic infrastructure/connectivity:
- cross-chain.test.ts: Only basic chain queries, no actual cross-chain
testing
- ethereum-basic.test.ts: Tests Ethereum RPC itself, not DataHaven
features
- contracts.test.ts: Trivial deployment address checks
- datahaven-substrate.test.ts: Redundant Substrate RPC connectivity
tests

Keep 3 comprehensive test suites covering core DataHaven functionality:
- validator-set-update.test.ts: Validator onboarding and cross-chain
updates
- native-token-transfer.test.ts: Bidirectional token bridging
- rewards-message.test.ts: Era transitions and reward distribution
 2025-10-24 15:07:30 +02:00
Ahmad Kaouk
48f8add3c4
ci: fix fetch submodule (#248) 
This PR fixes the E2E checkout failure by fetching full history instead
of a depth-1 clone so the Snowbridge forge-std submodule can resolve its
pinned commit.
 2025-10-24 13:31:17 +03:00
undercover-cactus
4eca467514
ci: pin forge version when installing it (#243) 
In this PR, we pin the forge version use in the linter task. When forge
make a new release it brokes the linter task.

In the future we can update  the forge version explicitly.

---------

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
 2025-10-23 14:33:27 +00:00
undercover-cactus
908a2a7ed5
ci: remove unused volumes after e2e tests and remove logs collection (#246) 
In this PR, we fix the ci error indicating we reached the number of
volumes allow by deleting automatically after tests the volumes.

We also remove the step that collect logs because the container that are
interesting to us to debug are being removed entirely. Therefore the
logs from the nodes are not being collected in this step.
 2025-10-23 14:02:36 +00:00
Steve Degosserie
06574a4cf8
fix: 🔧 Lower StorageHub replication targets for the Testnet environment (#245) 
We initially use the same values as for Stagenet.
 2025-10-23 14:14:30 +03:00
Steve Degosserie
22d7ee4b94
fix: 🔧 Bump SH storage providers pallet's MaxMultiAddressSize parameter to 200 (#244) 2025-10-23 13:35:23 +03:00
undercover-cactus
eced179b09
misc: simplify Dockerfile to speed up build (#216) 
This PR remove the `cargo chef` step used to build the docker image used
in deployment. We noticed that `cargo chef` was adding more time to the
build and that removing it was saving us 40min.

Also in this PR, we removed the base image from parity which was really
heavy and was filling the rest of the disk space. This broke the build.
After some investigation it doesn't seem to add a lot to the build. It
has been replace with the official rust image as a base to build our
node.

The image used to run the image has been replaced with
`debian:trixie-slim`.

In the end those changed **should not** break any of the current
behavior and makes save a bit of CI time.
 2025-10-22 13:36:30 +02:00
Steve Degosserie
5988691a2f
feat: ✨ Add deployment charts for StorageHub MSP, BSP & Indexer nodes (Local & Stagenet envs) (#160) 
## Summary

This PR adds comprehensive Kubernetes deployment infrastructure for
StorageHub components, enabling deployment of the full StorageHub
network stack (MSP, BSP, Indexer, and Fisherman nodes) alongside
DataHaven nodes in both local and stagenet environments.

### What's Added

**1. New Helm Chart: StorageHub MSP Backend API**
(`deploy/charts/backend/`)
- REST API service for StorageHub operations
- Connects to PostgreSQL database for indexed blockchain data
- Connects to RPC nodes for real-time blockchain queries
- Configurable via TOML configuration file
- Supports environment-specific overrides
- Includes comprehensive documentation

**2. StorageHub Node Deployment Charts**
(`deploy/charts/node/storagehub/`)
- **MSP Node** (`sh-mspnode`): Main Service Provider nodes with charging
capabilities
- **BSP Node** (`sh-bspnode`): Backup Service Provider nodes for
redundancy
- **Indexer Node** (`sh-idxnode`): Full indexing node with PostgreSQL
integration
- **Fisherman Node** (`sh-fisherman`): Network monitoring and
verification node

**3. Environment Configurations**
- **Local environment** (`deploy/environments/local/`): Development
setup with hostpath storage
- **Stagenet environment** (`deploy/environments/stagenet/`):
Production-like setup with AWS EBS
- PostgreSQL database configurations for Indexer and Fisherman nodes
- Proper service discovery and network configuration

**4. Enhanced CLI Tooling** (`test/cli/`)
- New `deploy storagehub` command for deploying StorageHub components
- Updated `launch storagehub` command for local testing
- Interactive deployment with environment selection
- Automatic database provisioning via Bitnami PostgreSQL charts

**5. Node Configuration Improvements**
- Fork-aware transaction pool for DH boot & validator nodes
- Unsafe RPC methods exposed on MSP nodes (for provider operations)
- JWT secret support for MSP Backend authentication
- ECDSA key scheme for StorageHub BCSV keys (DataHaven compatibility)

### Architecture

```
StorageHub Stack:
├── MSP Nodes (2 replicas) → Storage providers with charging
├── BSP Nodes (2 replicas) → Backup storage providers
├── Indexer Node → Database indexing + PostgreSQL
├── Fisherman Node → Monitoring + PostgreSQL (shared with Indexer)
└── MSP Backend API → REST API for StorageHub operations
```

### Testing

**Local Testing**:
```bash
cd test
bun cli launch storagehub  # Interactive launcher
# or
bun cli deploy storagehub  # Deploy via Helm
```

**Stagenet Deployment**:
```bash
cd deploy
helm install sh-mspnode ./charts/node \
  -f ./charts/node/storagehub/sh-mspnode.yaml \
  -f ./environments/stagenet/sh-mspnode.yaml \
  -n datahaven-stagenet
```

### Breaking Changes

None - This is purely additive infrastructure.

### Migration Notes

For existing deployments:
1. DataHaven nodes now use `--pool-type fork-aware` flag
2. Bootnode and validator node configs updated accordingly
3. No action required for existing DataHaven-only deployments
 2025-10-21 23:18:50 +03:00
undercover-cactus
dc5105869f
fix: linter and allow noNonNullAssertion typescript coding style (#218) 
In this PR we have a fix for the linter. We also allow for the non-null
assertion operator `!` to be used in typescript.

The typescript being used to setup tests and some convenient scripts we
should allow some the convenient feature of typescript.

Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
 2025-10-20 11:44:40 +02:00
Steve Degosserie
387c056912
fix: Resolve Foundry build errors and apply code formatting (#241) 
## Summary

Fixes the CI build failure in the `task-ts-build` workflow caused by
Foundry v1.4.2's Solar linter not being able to resolve Snowbridge's
context-specific import remappings.

## Problem

The Snowbridge submodule uses context-specific remappings (prefixed with
`:`) for its dependencies:
- `lib/snowbridge/contracts/:openzeppelin/` → OpenZeppelin contracts
- `lib/snowbridge/contracts/:prb/math/` → PRB Math library

Foundry v1.4.2's Solar linter doesn't understand these context-specific
remappings and fails with errors like:
```
error: file openzeppelin/utils/cryptography/MerkleProof.sol not found
error: file prb/math/src/UD60x18.sol not found
```

## Solution

Added global remappings that the linter can understand:
```toml
"openzeppelin/=lib/snowbridge/contracts/lib/openzeppelin-contracts/contracts/",
"prb/math/=lib/snowbridge/contracts/lib/prb-math/",
```

### Why This Works
- The linter can now resolve `openzeppelin/` and `prb/math/` imports
globally
- These global remappings take **lower precedence** than
context-specific ones during compilation
- The compiler still uses the context-specific remappings (with `:`)
when compiling Snowbridge contracts
- The linter uses the global remappings when checking all files

## Changes

### Commit 1: Add global remappings
- `contracts/foundry.toml`: Added 2 global remapping entries

### Commit 2: Apply forge fmt
- Applied automatic formatting via `forge fmt` to ensure code style
consistency
- Multi-line formatting for long import statements and function
signatures
- No functional changes - purely formatting updates

## Testing

✅ Local build succeeds with `forge build`
✅ No Snowbridge import resolution errors
✅ `forge fmt --check` passes with no formatting issues
✅ Only linting notes/warnings remain (not errors)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2025-10-20 11:20:59 +03:00
Steve Degosserie
cffdad2358
fix: 🔧 Don't remove /usr/sbin utils in operator Docker image (#240) 2025-10-15 23:54:29 +02:00
Steve Degosserie
931a225f09
fix: 🔧 Fix binary location in Docker img (#239) 2025-10-15 22:54:44 +02:00
Steve Degosserie
72cac823af
fix: 🔧 Fix invalid condition on workflow_call in Docker release task (#238) 2025-10-15 21:33:54 +02:00
Steve Degosserie
d202869438
fix: 🔧 Fix Docker release extract tag logic (#237) 2025-10-15 19:50:59 +02:00
Steve Degosserie
ff694b0055
fix: 🔧 Fix Docker release extract tag logic (#236) 2025-10-15 18:52:27 +02:00
Steve Degosserie
8be3c0f979
fix: 🔧 Fix Docker image tags in Docker release task (#235) 
Co-authored-by: Claude <noreply@anthropic.com>
 2025-10-15 18:30:37 +02:00
Steve Degosserie
dda9111ee6
fix: 🔧 Copy all shared libraries required by the DataHaven node in build & production images (#234) 2025-10-15 14:46:07 +02:00
Steve Degosserie
9a5404de82
refactor: Consolidate and optimize Docker image architecture (#233) 
## Overview

This PR consolidates and optimizes the Docker build system, reducing
redundancy and improving CI/CD performance. The changes eliminate
duplicate Dockerfiles, introduce a flexible build template, and optimize
release builds to reuse CI artifacts.

## Changes Summary

### 🐳 Docker Images Restructured

**Before:** 5 Dockerfiles with significant overlap
**After:** 4 focused images + 1 utility

#### Final Structure:

1. **`operator/Dockerfile`** ✨ Updated
   - **Standard operator image** for CI and release builds
   - Minimal node image (accepts pre-built binaries)
   - GHCR: `ghcr.io/datahaven-xyz/datahaven/datahaven` (CI)
   - DockerHub: `datahavenxyz/datahaven` (releases)

2. **`docker/datahaven-build.Dockerfile`** (moved from
`operator/Dockerfile`)
   - Full source-to-binary build for manual releases
   - DockerHub: `datahavenxyz/datahaven:{label}`
   - Supports custom RUSTFLAGS and fast-runtime feature
   - Only used for manual workflow_dispatch builds

3. **`docker/datahaven-production.Dockerfile`** (kept)
   - Binary builder for CPU-specific releases
   - Used by build-prod-binary workflow template
   - Supports custom target-cpu flags

4. **`docker/datahaven-dev.Dockerfile`** ✨ NEW (local dev only)
   - **FOR LOCAL DEVELOPMENT/TROUBLESHOOTING ONLY**
   - Includes debug tools: gdb, strace, vim, sudo
   - Extra dependencies: librocksdb-dev, curl
   - RUST_BACKTRACE enabled by default
   - **DO NOT USE for CI or production builds**

5. **`test/docker/crossbuild-mac-libpq.dockerfile`** (kept)
   - Utility for macOS → Linux cross-compilation

#### Removed (Redundant):
- ❌ `docker/datahaven.Dockerfile` → replaced by operator/Dockerfile
- ❌ `test/docker/datahaven-node-local.dockerfile` → replaced by
datahaven-dev.Dockerfile

---

### 🔄 Workflow Improvements

#### Enhanced `publish-docker` Template
- Supports both GHCR and DockerHub registries
- Flexible inputs: dockerfile, context, build-args, cache scope
- Auto-generates OCI-compliant labels
- Reduces code duplication (~70 lines → ~15 per workflow)

#### Refactored CI Pipeline
- **`docker-build-ci`**: Builds `operator/Dockerfile` → GHCR for CI/E2E
testing
- **`docker-build-release`**: Builds `operator/Dockerfile` → DockerHub
(main branch only)
- Both CI and release workflows now use the same minimal operator image
- Release builds **reuse CI binaries** instead of rebuilding from source

#### Optimized Release Workflow
The `task-docker-release` workflow now has dual modes:

**Mode 1: `workflow_call` (CI - main pushes)**
- ✅ Reuses binary from CI's build-operator task
- ✅ Uses lightweight `operator/Dockerfile`
- ✅ Tags: `latest`, `sha-{short}`
- ⚡ **Fast**: ~5 minutes (vs ~30 min previously)

**Mode 2: `workflow_dispatch` (Manual)**
- ✅ Full source build with `datahaven-build.Dockerfile`
- ✅ Custom branch and label selection
- ✅ Optional fast-runtime feature
- ✅ Tags: `PROD-{label}` or user-defined

---

### 🔧 Additional Optimizations

- Copy libpq5 from builder stage instead of reinstalling (smaller,
faster)
- Remove redundant protobuf-compiler package (use protoc v21.12
directly)
- Standardize user UID to 1000 across all runtime images
- Consistent OCI labeling and metadata

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2025-10-15 01:33:20 +02:00
Steve Degosserie
8874a99100
fix: 🔧 Add missing libpq5 lib to DH operator Docker image (#232) 2025-10-14 12:37:31 +02:00
Steve Degosserie
3e07decea3
fix: 🔧 Add missing libpq5 lib to DH node Docker images (#231) 2025-10-13 23:25:34 +02:00
Steve Degosserie
750e8f391c
fix: 🔧 Fix Docker production image (#230) 2025-10-13 17:53:34 +02:00
Steve Degosserie
db99f62d3b
fix: 🔧 Fix publish runtime draft release (#226) 2025-10-12 23:59:32 +02:00
Steve Degosserie
678a8fb161
fix: 🔧 Use standard Github runners for the publish runtime task (#225) 2025-10-11 10:37:19 +02:00