Skip to content

feat(beacon): follow gloas live - #639

Merged
MegaRedHand merged 53 commits into
feat/beacon-plataberget-devnetfrom
feat/beacon-gloas-live
Oct 26, 2026
Merged

MegaRedHand merged 53 commits into
feat/beacon-plataberget-devnetfrom
feat/beacon-gloas-live

Conversation

@MegaRedHand

Copy link
Copy Markdown
Collaborator

🗒️ Description / Motivation

#634 implements gloas (EIP-7732, ePBS) in the state transition and fork choice, and #635 adds Platåberget, the one public chain already past gloas. The follower still could not follow it: refuse_unfollowable_fork refused a gloas anchor, nothing subscribed to or fetched execution payload envelopes, and a fork-digest boundary needed a restart.

This PR makes the live follower follow gloas: ethlambda beacon --network plataberget checkpoint-syncs from a gloas anchor, imports gloas blocks and their payload envelopes from gossip and req/resp, tracks head and finality, judges payloads through the execution client, and crosses a fork-digest boundary at runtime. The node stays a follower (no gloas validator duties).

The runtime rollover is what Sepolia needs: it forks to gloas on 2026-10-06 13:53 UTC.

What Changed

Envelope import (crates/blockchain/src/beacon_envelope.rs, beacon_payloads.rs)

  • An envelope waits for its block's post-state, then for every sampled column when the bid has blob commitments (data availability moves from the block to the envelope), then goes to on_execution_payload_envelope.
  • A block whose parent is FULL with an unverified payload is held until that envelope verifies. A missing parent envelope is asked for at once (whatever the block's source), then re-asked once a slot.
  • An envelope for a block that is stored but not imported yet is checked against that block's bid and held outside the per-arrival-slot cap (awaiting_import). Range catch-up depends on this.
  • Every queue is capped, aged out and swept at finality.
  • Fork-choice events are timed at arrival, not at the slot tick. The head's payload status is kept with its root (Store::head_payload_status) and recomputed after a restart.

Gossip (state_transition::beacon::gossip, p2p/src/beacon/)

  • Gloas rules for beacon_block, data_column_sidecar, aggregates and attestations (progressive containers, aggregation-bits length bounded before expansion, verify_attestation_payload_status).
  • New topics: execution_payload and payload_attestation_message. They are joined only for gloas digests (BeaconTopics::for_fork).
  • Gloas column sidecars carry no signature, so their parking is bounded (beacon_columns.rs).

Req/resp (p2p/src/req_resp/envelopes.rs, envelope_client.rs)

  • Serves execution_payload_envelopes_by_{range,root}/1: canonical FULL envelopes only, and the head's envelope only when the head node is FULL.
  • Fetches by root on needs_envelope (same dedup and retry ladder as block fetches) and by range after a gloas span's blocks reach the actor. Range sync holds the next batch until the envelope answer is forwarded.

Fork-digest rollover (p2p/src/beacon/fork_schedule.rs, transition.rs)

  • ForkSchedule joins the next digest's topics SUBSCRIBE_LEAD_EPOCHS before its boundary and leaves the old ones UNSUBSCRIBE_LAG_EPOCHS after. Blob-parameter (BPO) boundaries are included.

Execution client (crates/net/engine, beacon_engine.rs)

  • engine_newPayloadV5 judges an envelope that passed the consensus checks:
    • INVALID with a null latestValidHash is a hash mismatch, so only that envelope is refused.
    • INVALID with a latestValidHash condemns the payload through the execution-chain walk.
    • An engine error keeps the envelope and retries once a slot, probing with the oldest.
  • engine_forkchoiceUpdatedV4 for a gloas head, with the custody columns. The head hash follows the head node's FULL or EMPTY status; finalized and safe come from the checkpoint blocks' bid.parent_block_hash.

Storage (crates/storage/src/store.rs)

  • New tables: ExecutionPayloadEnvelopes (verified envelopes) and BlockTimeliness (gloas timeliness). Both are reloaded on resume for the unfinalized window.
  • PTC vote vectors are reseeded empty on resume, since absent vectors fail a spec assert.

Checkpoint sync and API

  • A gloas anchor is accepted. Its envelope is fetched best-effort from the provider's execution_payload_envelopes endpoint, and the by-root request covers the rest.
  • The Beacon API serves gloas envelopes (crates/net/rpc/src/beacon/envelopes.rs). Optimism for gloas is judged read-only (rpc/src/shared/optimistic.rs).
  • refuse_validator_duties_from_gloas keeps the validator-client endpoints off gloas epochs.

Docs: CLAUDE.md, README.md, docs/beacon_engine.md, docs/beacon_stf.md, docs/beacon_wire.md, docs/checkpoint_sync.md, docs/spec_deviations.md.

Correctness / Behavior Guarantees

Deliberate departures from the gloas gossip rules (docs/beacon_wire.md), each for want of state this node does not keep. The engine-side choices are in docs/spec_deviations.md.

Where Spec Here Why
Block/column/envelope gossip A block seen without a post-state: REJECT Queue or IGNORE No bad-block cache yet
Envelope whose block state is not cached in p2p IGNORE Queue and forward to the actor The actor verifies every envelope; dropping it stalled range catch-up
payload_attestation_message on_payload_attestation_message checks the signature again Applied after p2p verified the signature; the actor re-checks only store-state conditions (known block, current slot, committee seat) Avoids a second signature check per vote

Limitations

  • ENR eth2 after a boundary: ethrex's DiscoveryServer cannot replace the served ENR at runtime, so the eth2 entry keeps the startup digest until a restart. Peers that discover the node after a boundary reject the record. Existing connections and gossip are unaffected.
  • Restarting a beacon data directory hits SpecAssert("block_root in store.unrealized_justifications") until fix(beacon): persist unrealized justifications so get_head survives a restart #628 lands. That is not gloas-specific.

Tests Added / Run

  • Fixture runners: gloas cases of gossip_beacon_block, gossip_data_column_sidecar, gossip_beacon_aggregate_and_proof, gossip_beacon_attestation, plus the new gossip_execution_payload_envelope and gossip_payload_attestation_message. The reject_block_failed_validation-style vectors are skipped by name, for the bad-block-cache reason above.
  • Fixture-driven envelope import tests in ethlambda-blockchain, behind its new beacon-spec-tests feature (run by make test-beacon-mainnet; reported as ignored by a plain cargo test).
  • Unit tests for the fork schedule and digest switch, envelope queues and caps, req/resp codec and handlers, engine V5/V4 wire shapes (crates/net/engine/tests/wire_smoke.rs), and the resumed-store head walk.
  • Live smoke against Platåberget (local, tip c184e736): caught up from the checkpoint anchor to the network head in under 30 s, then 0-slot lag with a root match against beacon.plataberget.ethpandaops.io on every sample for 6 minutes.

Related Issues / PRs

✅ Verification Checklist

  • Ran make fmt — clean
  • Ran make lint (clippy with -D warnings) — clean
  • Ran make test (test-consensus plus test-node, at release-fast) — all passing (left to CI)

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

🤖 Codex Code Review

⚠️ Warning: Diff was truncated from 21634 to 10000 lines. Some changes were not reviewed.

I found one concrete correctness bug; the rest of the reviewed paths look thoughtfully implemented.

  • crates/blockchain/src/beacon_envelope.rs:222 — trim_awaiting_import() says it drops “highest slot first”, but held.pop() removes the last inserted envelope for the chosen root, not necessarily the highest-slot one. If envelopes for a root arrive out of order, this can evict a newer sync-critical envelope while keeping an older one. That can stall or skew by-range envelope catch-up under pressure. Either maintain each per-root Vec<HeldEnvelope> sorted by slot or remove the max-slot element explicitly before decrementing total.

  • crates/blockchain/state_transition/src/beacon/gossip/payload_attestation.rs:97 — worth double-checking against the spec intent: store.has_state(&data.beacon_block_root) gates acceptance on the voted block’s post-state existing locally, but the committee is then read from head_root at crates/blockchain/state_transition/src/beacon/gossip/payload_attestation.rs:113. If the voted block is known and valid but its post-state is uncached/unavailable, the vote is ignored even though validation only seems to need the head state plus block slot/root metadata. If this is intentional, the comment should justify why the voted block’s post-state is required here; otherwise this is an avoidable liveness drop.

  • crates/blockchain/state_transition/src/beacon/gossip/attestation.rs:245 and crates/blockchain/state_transition/src/beacon/gossip/aggregate.rs:502 — the new payload-status consistency check is a good consensus-hardening addition. I did not spot an obvious fork-choice/finality regression in the touched validation flow.

  • crates/blockchain/state_transition/src/beacon/gossip/payload_attestation.rs:42 — the SeenPayloadAttestations API is clean and the “record only on accept” contract is explicit. That part looks good.

If you want, I can do a second pass focused only on fork-choice / envelope import interactions in crates/blockchain/src/beacon_envelope.rs and crates/blockchain/state_transition/src/beacon/fork_choice.rs.


Automated review by OpenAI Codex · gpt-5.4 · custom prompt

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

🤖 Claude Code Review

Review of PR 639: follow gloas live

This is a very large PR (about 15.6k lines added). I could only read part of it: the CLAUDE.md, README and Makefile changes, bin/ethlambda/src/{beacon,checkpoint_sync,main}.rs, and the first part of the diff. I did not read the following, so this is not a review of them:

  • beacon_envelope.rs (3.1k lines)
  • the p2p envelope client and server
  • the fork-digest transition code
  • the gloas gossip rules
  • the engine API V5/V4 changes

Findings

  1. No size cap on the anchor envelope download (checkpoint_sync.rs, fetch_envelope). response.error_for_status()?.bytes().await? buffers the whole body before SSZ decoding. The URL is operator-configured, so the risk is low. A misbehaving or compromised checkpoint server could still send a very large body. Other fetches in this file may already share a limit. If they do, apply it here. If they don't, a bounded read costs little for something that runs once at startup.

  2. The anchor envelope fetch only checks beacon_block_root (fetch_envelope). That is acceptable because the PR hands it to new_execution_payload_envelope, where the bid, column and signature checks apply. Please confirm that path has no shortcut for BlockArrival::now() that would treat a checkpoint-served envelope as timely or pre-verified. I could not check this in beacon_envelope.rs.

  3. Comment formatting in main.rs. The comment "After both Init* sends, … Off the startup path: the fetch is best-effort…" runs past the line width after the wrapped first sentence. cargo fmt doesn't reflow comments, so rewrap it by hand.

  4. Test-only tokio features. bin/ethlambda/Cargo.toml adds net, io-util, macros and rt to the dev-dependency feature list. Cargo unifies dev-dependency features into the normal build only for some commands, such as cargo test. The comment above that line still describes only test-util, so extend it.

  5. Doc and runtime consistency. The CLAUDE.md addition says the DB restart of a beacon directory hits SpecAssert("block_root in store.unrealized_justifications") until fix(beacon): persist unrealized justifications so get_head survives a restart #628 lands. That is a known restart-time panic. It is worth confirming it can't now be triggered by gloas restart paths, because the PR also reloads ExecutionPayloadEnvelopes and BlockTimeliness on resume. Also confirm the new tables don't need a DB_VERSION bump: the PR says no, and that holds only if older data directories simply have those tables empty.

  6. Dead-guard drift. refuse_unfollowable_fork and warn_if_unfollowed_fork_scheduled are now inert, since every fork is followed. Keeping them as guards for the next fork is reasonable, and the test was updated to match. Nothing is wrong here.

What looks good

  • The new tests (anchor_envelope_tests, anchor_envelope_root_tests) cover the 404 and wrong-block cases and the "already holds the envelope" case.
  • Reusing ForkSchedule for both the startup ENR and the runtime rollover keeps the two from drifting.
  • The anchor fetch is best-effort and runs off the startup path, after both Init* messages.
  • The Makefile runs the new beacon-spec-tests feature of ethlambda-blockchain under test-beacon-mainnet. Without the feature those tests report as ignored, so a plain cargo test needs no download.

Recommendation

Nothing I read is blocking. The consensus-critical parts (envelope queueing and aging, FULL/EMPTY head handling, the engine INVALID / latestValidHash semantics, p2p envelope range/root serving) need a reviewer who can go through those files in full before merge.


Automated review by Claude (Anthropic) · sonnet · custom prompt

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

🤖 Kimi Code Review

⚠️ Warning: Diff was truncated from 21634 to 10000 lines. Some changes were not reviewed.

I'll review this PR implementing Gloas (EIP-7732) ePBS support in ethlambda. This is a substantial consensus-layer change affecting fork choice, gossip validation, state transition, and networking.

Critical Issues

1. Fork Choice: Missing store.has_state check before store.get_state in block_payload_status (Line ~2150)

pub fn block_payload_status(store: &Store, root: Root) -> PayloadStatusEnum {
    if is_known_pre_gloas_block(store, root) {
        if store.is_beacon_optimistic(root) {
            PayloadStatusEnum::Syncing
        } else {
            PayloadStatusEnum::Valid
        }
    } else {
        store.beacon_block_payload_status(root)  // No state existence check!
    }
}

Issue: For an unknown root, store.beacon_block_payload_status(root) likely returns a default/invalid status rather than NOT_VALIDATED. The function should explicitly handle unknown roots:

} else if store.has_block(&root).unwrap_or(false) {
    store.beacon_block_payload_status(root)
} else {
    PayloadStatusEnum::Syncing  // or a dedicated "Unknown" variant
}

2. apply_or_hold_envelope Missing Signature Check Before Engine Call (Line ~1030 in beacon_envelope.rs)

// Consensus checks come first...
if let Err(err) = fork_choice::check_execution_payload_envelope(...) {
    // ...
}

// Then execution client...
let status = match self.engine.clone() {
    // ...
};

Issue: The signature verification verify_execution_payload_envelope_signature is only performed in hold_envelope_awaiting_columns (line ~1240), NOT in the direct import path. A malicious envelope that passes check_execution_payload_envelope (which uses ExecutionEngine::valid() — a no-op!) but has an invalid signature could reach the engine if columns are already present.

Fix: Add signature verification in apply_or_hold_envelope before the engine call, or ensure check_execution_payload_envelope verifies the signature when engine is ExecutionEngine::valid().

3. envelope_matches_bid Missing Critical Bid Fields (Line ~280 in beacon_envelope.rs)

fn envelope_matches_bid(...) -> bool {
    envelope.builder_index == bid.builder_index
        && envelope.payload.block_hash == bid.block_hash
        && envelope.payload.prev_randao == bid.prev_randao
        && envelope.payload.gas_limit == bid.gas_limit
        && envelope.execution_requests.hash_tree_root() == bid.execution_requests_root
        && envelope.payload.slot_number == block_slot
}

Issue: Missing checks for:

  • fee_recipient (bid commits to this)
  • timestamp (must match slot)
  • parent_hash / parent_beacon_block_root consistency

These are checked in verify_execution_payload_envelope eventually, but the early bid-matching should be comprehensive or documented as intentionally partial.

Security Vulnerabilities

4. gloas_root_fits Race Condition on Slot Bound (Line ~340 in beacon_columns.rs)

fn gloas_root_fits(&self, root: H256, slot: u64) -> bool {
    if self.column_parking.gloas_root_slots.get(&root).is_some_and(|&parked_slot| parked_slot != slot) {
        return false;
    }
    self.column_parking.gloas_roots.get(&slot).is_none_or(|roots| {
        roots.contains(&root)
            || roots.len() < MAX_PARKED_GLOAS_ROOTS_PER_SLOT
            || self.holds_block(&root)
    })
}

Issue: No atomicity between the gloas_root_slots check and the gloas_roots insertion. Two concurrent calls (though single-threaded actor, so perhaps OK — verify) could both pass the length check and exceed the cap. Actually fine for actor model, but document this invariant.

5. mock_engine Test Server: HTTP Request Smuggling Risk (Line ~2600 in beacon_envelope.rs)

async fn mock_engine(...) {
    // ...
    let (head_end, body_len) = loop {
        if let Some(end) = pending.windows(4).position(|window| window == b"\r\n\r\n") {
            // Parses Content-Length manually, no validation of header format

Issue: The hand-rolled HTTP parser doesn't validate that Content-Length is the only body framing mechanism, doesn't handle chunked encoding, and could be confused by malformed requests. Acceptable for tests only, but mark unsafe or document test-only use.

Correctness Issues

6. missing_columns_for_envelope Incorrect Logic (Line ~240 in beacon_envelope.rs)

pub(crate) fn missing_columns_for_envelope(
    commitment_count: usize,
    custody_columns: &[u64],
    present: &[u64],
) -> Option<Vec<u64>> {
    if commitment_count == 0 {
        return None;  // No commitments = no columns needed, but also no DA check?
    }
    // ...
}

Issue: When commitment_count == 0, returns None meaning "nothing outstanding". But the spec's is_data_available is "vacuous over an empty list" — this is correct per spec, but dangerous: a block with no commitments has no data availability requirement, which is correct for empty blocks but could be exploited if the bid claims zero commitments.

Verify: Does the bid validation ensure commitment_count matches the actual KZG commitments? Yes, in verify_execution_payload_envelope, so this is OK.

7. advance_beacon_clock_to Clamping Logic (Line ~3650 in lib.rs)

fn advance_beacon_clock_to(&mut self, event_ms: u64) {
    // ...
    let slot_end_ms = config.genesis_time_ms()
        .saturating_add(current_slot.saturating_add(1).saturating_mul(config.slot_duration_ms))
        .saturating_sub(1);
    let time = event_ms.min(slot_end_ms) / 1000;

Issue: Integer division / 1000 truncates toward zero. For events at slot_end_ms = 11999 (12s slot), this becomes 11, not 12. The spec uses whole seconds, but this could cause off-by-one in edge cases. Use proper second conversion.

8. apply_payload_attestation_message Ignores Return Value (Line ~3700 in lib.rs)

let _ = fork_choice::on_payload_attestation_message(...)
    .inspect_err(|err| debug!(...));

Issue: The let _ = discards Ok(()) and Err(...) equally. More importantly, the store clock is advanced but if the attestation fails, the clock stays advanced — this could cause subsequent attestations for the same slot to be judged with a later clock, potentially affecting timeliness calculations.

Performance Concerns

9. trim_awaiting_import O(n²) in Worst Case (Line ~180 in beacon_envelope.rs)

fn trim_awaiting_import(&mut self) {
    let mut total: usize = self.awaiting_import.values().map(Vec::len).sum();
    while total > MAX_ENVELOPES_AWAITING_IMPORT {
        let Some(furthest) = self.awaiting_import.iter()
            .filter_map(|(root, held)| held.iter().map(|h| h.slot).max().map(|s| (s, *root)))
            .max()
            .map(|(_, root)| root)
        else { return; };
        // ...
    }
}

Issue: Each iteration does a full scan to find max slot. With MAX_ENVELOPES_AWAITING_IMPORT = 512 and potential overfill of thousands, this is O(n²). Use a BTreeMap<Slot, Vec<Root>> or similar for O(log n) eviction.

10. el_parent Linear Walk Unbounded in Pathological Cases (Line ~280 in beacon_payloads.rs)

fn el_parent(store: &Store, root: H256) -> Option<H256> {
    // ...
    loop {
        if payload_hash_of(store, cursor) == Some(bid.parent_block_hash) {
            return Some(cursor);
        }
        // ...
        cursor = next;  // Could walk entire unfinalized chain!
    }
}

Issue: In a chain with many EMPTY nodes, this walks the entire unfinalized history. The finality_floor check helps but only at the boundary. Add a step limit or document the worst-case bound.

Rust Best Practices

11. Excessive clone() in settle_envelopes (Line ~1350 in beacon_envelope.rs)

let roots: Vec<H256> = self.envelopes.awaiting_engine.keys().copied().collect();
self.envelopes.work.extend(roots);

Issue: Collects all keys just to extend a VecDeque. Use self.envelopes.work.extend(self.envelopes.awaiting_engine.keys().copied()) directly.

12. HashMap with H256 Keys — Consider FxHashMap (Multiple Files)

The code uses std::collections::HashMap extensively. For consensus-critical code with many small lookups, rustc_hash::FxHashMap or hashbrown with a fast hasher would improve performance. Not critical but worth noting.

13. Missing #[must_use] on apply_or_hold_envelope (Line ~90 in beacon_envelope.rs)

async fn apply_or_hold_envelope(&mut self, held: HeldEnvelope) -> bool {

This returns whether the envelope was verified. Callers in settle_envelopes use it, but it's easy to miss. Add #[must_use].

Memory Safety

14. unsafe Block in beacon_envelope.rs Tests — Actually Safe (Line ~2600)

The mock_engine function uses raw TCP but no unsafe blocks. The concern is logical safety, not memory safety. No issue here.

Documentation/Clarity

15. ExecutionEngine::valid() Misleading Name (Line ~1040 in beacon_envelope.rs)

fork_choice::check_execution_payload_envelope(
    &self.store,
    &held.envelope,
    &config,
    &sidecars,
    &ExecutionEngine::valid(),  // This is a no-op engine!
)

Issue: ExecutionEngine::valid() suggests it validates; it actually skips validation. Rename to noop() or skip_validation() for clarity.

Minor Issues

16. Typo in CLAUDE.md (Line ~48)

-    ├─ p2p/                 # libp2p: gossipsub + req-resp (Status, BlocksByRoot, BlocksByRange, gloas envelopes)
+    ├─ p2p/                 # libp2p: gossipsub + req-resp (Status, BlocksByRoot, BlocksByRange, gloas envelopes)

"gloas" → "gloas" is correct (the fork name), but "gloas envelopes" should perhaps be "Gloas envelopes" for consistency.

17. tokio Feature Creep in bin/ethlambda/Cargo.toml (Line ~67)

tokio = { workspace = true, features = ["test-util", "net", "io-util", "macros", "rt"] }

The "net", "io-util", "macros", "rt" features are added for tests. Verify these don't bloat the release binary — they shouldn't if dev-dependencies are properly gated, but "rt" in particular might affect runtime behavior.

Consensus-Critical Verification Needed

18. get_head_node Payload Status Computation (Not fully visible in diff)

The PR mentions get_head_node is "one bottom-up walk for every fork, with the current slot's fork selecting the payload rules and the boost gate." Verify that the PayloadStatus returned correctly handles the case where a block has both FULL and EMPTY nodes but no verified payload — the EMPTY node should be selected, not an error.

19. condemn_gloas_payload Finality Check (Line ~380 in beacon_payloads.rs)

if root == finalized.root || slot <= compute_start_slot_at_epoch(finalized.epoch) {
    error!(...);
    return false;
}

Issue: Uses <= for slot comparison but == for root. If root != finalized.root but slot == compute_start_slot_at_epoch(finalized.epoch), this refuses to condemn. Is this correct? The finalized block could be at a later slot within the epoch if the epoch boundary was skipped.

Summary

Priority Count Categories
Critical 3 Missing signature check, incorrect payload status for unknown roots, bid matching incompleteness
Security 2 Race condition (documented), test HTTP parser (test-only)
Correctness 4 Clock clamping, attestation clock advance, missing_columns semantics, finality check
Performance 2 O(n²) eviction, unbounded EL walk
Rust/Style 4 Clones, #[must_use], naming, features

The most critical fix needed is Item 2: signature verification must occur before the engine call in the main import path, not just in the column-waiting path.


Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt

@MegaRedHand MegaRedHand added the beacon Ethereum Beacon Chain client label Oct 1, 2026
MegaRedHand added a commit that referenced this pull request Oct 5, 2026
…8-gloas-live

# Conflicts:
#	crates/net/p2p/src/beacon/transition.rs
MegaRedHand added a commit that referenced this pull request Oct 19, 2026
## 🗒️ Description / Motivation

A beacon follower that has caught up keeps downloading blocks it already
has.

`P2PServer::beacon_fetched_through` is the watermark range sync starts
from, and only `handle_beacon_blocks_by_range_response` advances it.
Once the follower is at the tip it imports from gossip, so the watermark
stays at its last range batch while the head moves on. Every peer that
connects later advertises a head above the watermark, and its `Status`
answer starts a range session from there:

```
watermark (last range batch)           our head           peer head
          |                                |                  |
          [ already imported from gossip ][ owed            ]
          ^ range session starts here      ^ should start here
```

Each re-fetched block also brings its custody columns
(`request_beacon_data_columns_by_range` runs for the same span), and
each one costs the chain actor an import attempt that ends at "already
in the store", followed by a head recompute.

Observed on the Platåberget follower (build `c184e736`) between 14:08Z
and 17:02Z on 2026-10-01:

| | |
|---|---|
| Bursts of arrivals that imported nothing | 66 |
| Blocks in them | 748 |
| Burst size | the slots since the burst before it |
| Cost per arrival | ~65 ms (`cascade_ms` ~30-47 ms + `get_head_ms` ~20
ms) |
| Largest burst | 67 blocks at 16:52:36Z; the gossip block for slot
354263 waited 4.2 s behind it in the queue |

## What Changed

- `crates/net/p2p/src/req_resp/handlers.rs`: `beacon_sync_target` takes
the store's head slot and measures the peer against
`max(fetched_through, head_slot)`. `handle_status_response` reads the
head with `Store::beacon_head`, the same whole-block decode
`build_status` already does for every handshake we send, and logs it
beside `fetched_through`.
- `crates/net/p2p/src/lib.rs`: the `beacon_fetched_through` doc says
only range answers advance it.
- `docs/beacon_wire.md`, `CLAUDE.md`: describe the two floors.

## Correctness / Behavior Guarantees

- The watermark still wins while a delivered batch is draining through
the actor mailbox: there the head is below it. That is the case it was
introduced for (11,213 blocks fetched to import 100 when range sync was
driven off the head alone), and it is unchanged.
- The head is a safe floor because a head block's whole ancestry is
already imported, so nothing at or below it can be missing from the
canonical chain.
- A peer on another fork whose head is above ours gets a session from
our head onward. Blocks of its branch at or below our head slot come
through the by-root parent lookup, as they already did whenever the
watermark was current.
- A peer exactly one block ahead (its block still in gossip flight)
still costs a one-block request. I left that in rather than adding a
tolerance, since a tolerance would delay real catch-up.
- The startup seed is unchanged: it was already the anchor head.

## Tests Added / Run

-
`beacon_sync_target_starts_after_a_head_gossip_moved_past_fetched_through`
(new): a head above the watermark moves the start to `head + 1`; a peer
at or behind the head gets `None`.
- `beacon_sync_target_keys_off_fetched_through_while_the_head_lags` (was
`..._not_store_head`): same assertions, now with a head below the
watermark passed in explicitly.
- `beacon_sync_target_is_bounded_by_max_sync_range`: updated for the new
argument.

```
cargo test -p ethlambda-p2p --lib --profile release-fast
cargo clippy -p ethlambda-p2p --all-targets --profile release-fast -- -D warnings
```

Not deployed to a follower yet.

## Related Issues / PRs

- Related to #639 (the build deployed on the follower).

## ✅ Verification Checklist

- [x] Ran `make fmt`: clean
- [x] Ran clippy on `ethlambda-p2p` with `-D warnings`: clean
- [x] Ran `ethlambda-p2p` unit tests: passing
Gloas sidecars carry no signed header or inclusion proof: they name their
block by root and slot, and the commitments their cells are checked against
live in that block's bid. The fulu rules cannot judge them, so add gloas's
modified data_column_sidecar rules beside them, keyed on (block root, index)
per the modified Seen, with a fork-agnostic DataColumnSidecar enum in the
types crate for the later plumbing to carry.

A sidecar whose block has not been seen is queued (the spec's "MAY be
queued"). A block seen without a post-state is queued rather than rejected,
the same deviation the fulu parent rule makes until a bad-block cache
exists, so that one vector is listed in SKIPPED.

The spec-test runner now keeps a per-handler list of validated forks
instead of a blanket fulu filter, and reads the per-message
current_time_ms gloas's column vectors use in place of offset_ms.
Fulu's column rule is split into a cheap half and a stateful half so a
duplicate is dropped without a blocking-pool permit and the KZG batch stays
off the swarm loop; the p2p wiring will call the halves separately. Give the
gloas rule the same shape, with validate_gloas as the combined form.

The stateful half now checks for a post-state before decoding the block, so
a block without one costs no read of it. A sidecar naming a block of an
earlier fork gets its own reject reason rather than reusing Malformed.

Tests pin the deliberate deviations: a block without a post-state is queued,
an already stored sidecar is ignored, and a non-gloas block is rejected by
name. The spec runner now fails a message that names no clock reading, and
the pre-fulu decode test uses bytes valid for fulu so it can only fail on
the fork.
The beacon wire computed its fork digest once at startup, so a follower
was stranded on topics nobody publishes to the moment a fork activated.
Blob-parameter-only (BPO) boundaries move the digest too, and Sepolia's
fulu to gloas boundary is close, so a long-running node has to cross
them on its own.

ForkSchedule is a pure function of the config and genesis validators
root: every digest from fork and blob-schedule epochs, digest to fork,
and the spec's window ("Transitioning the gossip", altair
p2p-interface.md). Post-fork topics are joined one epoch ahead of the
boundary and the pre-fork topics left two epochs after it, so a node
started inside a window holds both, same as one that crossed into it.

The P2P actor applies the schedule idempotently on a self-rescheduling
timer (and once at startup): it joins and leaves topic sets, swaps the
published digest and fork, bumps the metadata sequence number, accepts
peer Status at any held digest, and moves the discovery admission
filter, which is now shared with the peer table's copy. Attestation
decode takes its fork from the incoming topic's digest instead of the
current one.

The served ENR eth2 entry cannot be updated yet: ethrex's
DiscoveryServer::spawn returns no handle and has no message that
replaces its record. update_served_enr marks the gap.
Review follow-ups to crossing fork digests at runtime.

The fork digest gauge is reset before it is set, since the digest now
changes while running and the old label would otherwise keep reading 1.
Range sync starts only from a peer on the current digest (or the next
one within the gossip clock disparity of its boundary); a Status on any
held digest is still answered. Publishing picks the digest from the
message's own slot, so a slot-B message cannot leave on the old topics
before the switch runs, and data column decode failures take their fork
from the topic's digest instead of the wall clock.

ForkSchedule derives the eth2 entry's next_fork_* from its merged
entries and no longer clones the Config. Only a blob entry before fulu
is merged away; a post-fulu entry always moves the digest since it
hashes the entry's own epoch. BeaconTopics::for_fork matches on every fork,
apply() checks the current digest is held before joining anything, and
returns the topics it joined and left so the aggregator-subnet handling
is tested. Stale docs about a frozen-at-startup digest are rewritten.
The gauge test wrote the global fork-digest gauge while transition tests
in the same binary wrote it too, so it could fail under parallelism; the
reset-then-set logic is now a helper tested on its own vector. The
pre-fulu blob entry test put its entry on deneb's own fork epoch, where
the epoch dedup dropped it before the digest comparison ran; it now uses
an epoch no fork has.

Publishing now refuses a slot whose digest this node does not hold
instead of fanning out to a topic nobody joined. next_fork_boundary had
no callers left after the schedule took over, so it goes with its tests,
and the wire docs no longer say the digest is computed once.
The beacon follower ticks once per slot, so the store clock sat at the
slot's start for the whole slot. on_block read it for block timeliness
(attestation deadline and, from gloas, the payload timeliness committee
deadline), found zero milliseconds into the slot, and called every
current-slot block timely, late ones included. A late block therefore
got proposer boost and was exempt from the late-head reorg rules.

Advance the store clock to the block's wire receipt (decode_start, else
hand-off) before on_block. The spec's clock is whole seconds and its
on_tick runs whenever time > store.time, so the deadlines are judged on
the second the block fell in. The advance never goes backwards and never
crosses a slot boundary, which stays the tick's job since it also drains
deferred aggregates and recomputes the head.

Why the tick was per slot: the follower has no sub-slot duties (see
ChainDuties::Beacon, introduced with the chain actor in #9). That is a
statement about duties, not about fork choice needing a coarse clock, so
arrival-time advances are safe. The envelope and payload-attestation
handlers are not called live yet; they get the same hook when wired.

Pre-gloas blocks now also get spec-correct proposer boost: a late block
is no longer boosted.
A restarted gloas follower lost `verified_payloads`, which silently
dropped every FULL fork-choice node, and `block_timeliness`, which made
the head walk fail with `Error::SpecAssert("head_root in
store.block_timeliness")`. Both were in-memory only.

Two new tables hold them: `ExecutionPayloadEnvelopes` (slot || root, the
verified envelope, so a later change can serve envelopes by slot range)
and `BlockTimeliness` (root, one byte per deadline). Writes go through
to the table and the in-memory set; `from_db_state` rebuilds both for
every LiveChain block at or above the finalized block's slot, the
finalized block included since its payload status decides which branch
its children build on. The PTC vote vectors stay in memory on purpose:
after a restart only the current and previous slot's blocks read them.

`insert_verified_payload` now takes the envelope and errors on an
unknown block, since the row key needs the block's slot.

DB_VERSION is not bumped: new column families change no existing layout,
and no v5 directory holds gloas data since the previous binary refused
gloas.
Restoring verified_payloads on resume let the head walk reach
payload_timeliness, which reads the payload committee's vote vectors.
Those are never persisted, so the lookup failed with a SpecAssert and
get_head (and a timely on_block) errored; the first child of the
pre-restart tip failed the same way in notify_ptc_messages. Resume now
seeds both vectors empty for every gloas block in the restored window,
which is what a freshly imported block holds and is true of a process
that has seen no votes.

Also: persist block timeliness only for gloas blocks (nothing reads
pre-gloas entries after a restart, so the rows would only grow); fall
back to the anchor slot rather than the epoch start when the finalized
block has no row, since a finalized block can sit below its epoch start;
pass the block's slot to insert_verified_payload instead of re-decoding
the block; and correct the docs that called a missing timeliness entry
fatal or the scratch in-memory only.
The resume test only exercised the spec-literal gloas_get_head. The
actor uses get_head_node, whose walk_head re-derives payload links that
do not survive a restart and reads the boost and weight helpers, so
assert it succeeds and agrees with gloas_get_head.

Also make the resume fork check an exhaustive match so a new fork forces
a decision, state the real cost of the LiveChain scan in its doc, and
debug-assert that insert_verified_payload is given its block's own slot.
Every gloas block was ignored as an unsupported fork, so the node could not
relay one even once the actor follows gloas. Gloas now runs its own modified
`validate_beacon_block_gossip`: the operation and parent-request limits
(gloas lists are progressive, so decoding enforces none of them), bid checks in
place of execution-payload checks, and the parent payload status read through
the existing `is_parent_node_full` and `is_payload_verified`. Earlier forks keep
their rules unchanged.

`fixed_proposer` answers from a gloas parent's lookahead, which gloas keeps
unchanged from fulu. The spec runner now validates gloas `gossip_beacon_block`
and delivers a listed block's payload envelope before the case runs.
Gossip rejected a block with more deposit requests than the preset bound,
but gloas deposit requests are a progressive list that neither the spec nor
the state transition bounds, so import would have accepted what gossip
refused. The parent payload status is also read once rather than decoding
the parent block twice on the validation pool, and a failed read is now an
Ignore(Internal) with a log instead of a mislabelled queue.

Docs no longer claim every gloas block is ignored as an unsupported fork.
Gloas redefines DataColumnSidecar (no header or inclusion proof; a slot and a
block root instead), and the bytes carry no fork tag. Thread the
fork-agnostic DataColumnSidecar enum through p2p, the actor API, storage reads
and the chain actor so a gloas column takes the same routes a fulu one does.

The fork comes from context: the gossip topic's digest, a req/resp chunk's
context bytes (then checked against the digest at the sidecar's own slot), and
for stored rows the fork of the slot in the key, so the table layout and
DB_VERSION are unchanged.

A gloas column is verified against its own block's bid, so the chain actor
parks it keyed on its block root (a fulu one stays keyed on its parent) and
replays it through the p2p checks once that block has a post-state. Tests that
stored a fulu sidecar at a slot mainnet still schedules as phase0 now use a
store with fulu active from genesis, since rows decode by their slot's fork.
The parking lot for data column sidecars the p2p checks could not judge yet
is about to grow bounds of its own, so give it a module beside
beacon_aggregates instead of growing lib.rs. Pure move: no behavior change.
A gloas sidecar carries no signature, so a peer can invent (slot, root, index)
triples as fast as it can send them and each one parked costs a disk row.

- Park only while the node follows the fork; gloas blocks cannot import yet, so
  nothing would ever drain a parked row.
- Check shape before anything is written: slot in gloas, index in range, column
  and proofs agree within the blob limit, index one this node samples.
- Cap distinct parked gloas roots per slot and pin each root to one slot.
- Keep the parked count as a running counter instead of summing every root.
- Share one "can be judged now" predicate between the actor and the p2p check,
  so a root with a state but no block cannot bounce between them.
- Drop by-root column answers that were not asked for.

Also refreshes docs that still said a column waits for its parent.
A gloas follower dropped every aggregate, subnet attestation and attester
slashing as an unsupported fork. They now decode and validate under gloas's
modified rules: data.index is the payload-present flag (0 or 1), and a vote
for the full payload is ignored until the block's envelope is verified and
its payload validated (verify_attestation_payload_status).

Gloas shares electra's wire bytes but its Attestation is a progressive
container, so the aggregator's signed root differs. The aggregate therefore
gets its own SignedAggregateAndProof::Gloas variant rather than reusing
electra's, and attester slashings get a matching AttesterSlashing::Gloas.
Subnet attestations keep the SingleAttestation shape and are told apart by
slot.

apply_verified_aggregate took PreGloas rules unconditionally; it now reads
the vote's fork from its slot so payload_present is recorded for gloas votes.

Payload verdicts live in a new beacon-root-keyed scratch map, since a gloas
block carries no execution hash in el_block_hashes; an absent root reads as
NOT_VALIDATED, as the specification's default does.
… votes

Gloas aggregation bits are an unbounded progressive bitlist, and the seen-set
verdict expands them into a Vec<bool> before any other check, so a large
message forced a large allocation. A gloas aggregate longer than the electra
type bound is now rejected first, reading the length without expanding it.

An honest gloas-slot vote for the last pre-gloas block (FULL under the
boundary rule) was ignored as optimistic because block_payload_statuses never
holds a pre-gloas root. Such a root now reads its own block validity.

Also drops the now-unreachable UnsupportedFork arms on the aggregate and
attestation topics, fixes stale docs, tightens an assertion, and unit-tests
the payload-status branches and the status map.
Gloas moves the execution payload out of the block into a revealed
envelope, and the Amsterdam execution fork asks about it with
engine_newPayloadV5 and an ExecutionPayloadV4 (V3's fields plus
blockAccessList and slotNumber). Add the JSON view of that payload, the
client call (with the six second timeout amsterdam.md gives the method),
the capability entry, and a request builder that takes the payload,
parent root and requests from the envelope and the versioned hashes from
the block's bid, as verify_execution_payload_envelope does.

Nothing calls the new pieces yet; wiring them into envelope import is a
separate change.
ask_envelope returned Ok(None) for a non-gloas block, borrowing the
"nothing to ask, import it" meaning from ask. Here it could only mean a
block and envelope that do not belong together, which a caller copying
ask would have accepted unchecked. It now returns an EnvelopeMismatch
error, and the request builder checks the block is gloas, the envelope's
beacon_block_root is the block's root, and the bid's block_hash is the
payload's before building anything.

The missing-V5 handshake warning now fires only when the network
schedules gloas, so mainnet and Hoodi with a pre-Amsterdam execution
client do not warn on every start. Also refresh docs that still said
nothing calls the gloas request list or that V5 is absent, and use a
well-formed empty block access list in the wire test.
… gossip rules

The two new gloas topics need validators before p2p can subscribe to them.
Both follow the cheap/stateful split the other topics use, so the swarm loop
only runs the seen-cache, clock and finalized-slot checks inline; store reads,
the payload timeliness committee and signatures run on the blocking pool.

A block seen without a post-state is queued (envelope) or ignored (payload
attestation) where the spec rejects it, the deviation every other topic
already makes until a bad-block cache exists; the two matching
reject_block_failed_validation vectors are skipped for that reason. The
envelope's finalized-slot IGNORE runs before the block lookups, which only
changes the metric label between two IGNORE outcomes.

verify_execution_requests_limits moves to gossip/mod.rs so the block and
envelope rules share it.
Withdrawals and the execution request lists are progressive, so only the
gossip size cap bounded an envelope that could be queued for an unknown
block. The withdrawal and request count limits do not depend on the block,
so they now run in cheap_checks, before anything can be queued.

The payload attestation path also decoded the whole head block just to get
its root; head() returns the root alone. A committee-window miss now has its
own ignore reason instead of sharing StateUnavailable, and the deviation
paths (no post-state, uncached head, window miss, non-gloas block) are
pinned by unit tests since the fixtures treat Queue as ignore.
Gloas peers recover missed payloads through execution_payload_envelopes_by_range/1
and by_root/1. Register both protocols (bidirectional at the libp2p level) and
answer them from the store: the codec frames each envelope under the digest of
its block's epoch and rejects any non-gloas or unscheduled digest, by-range
serves only envelopes the next canonical block builds on (the head's when held,
since fork choice does not persist which payload branch the head picked), and
both are capped at MAX_REQUEST_PAYLOADS. No requests are issued yet; a response
nothing asked for is logged and dropped.
handlers.rs is already past three thousand lines and the client side of the
envelope protocols lands next, so give the server handlers, their pure
functions and tests a module of their own. Pure move.
@MegaRedHand
MegaRedHand enabled stacked auto-merge October 26, 2026 15:53
stacked auto-merge was automatically disabled October 26, 2026 16:09

Auto-merge was disabled for another pull request in the stack

@MegaRedHand
MegaRedHand added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit 433cd7a Oct 26, 2026
13 checks passed
@MegaRedHand
MegaRedHand deleted the feat/beacon-gloas-live branch October 26, 2026 16:46
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
- require_validated and the syncing endpoint's head check read optimism
  through block_is_optimistic, which knows a gloas block's payload link.
- payload_attestation_data follows attestation data's rule: a 503 without
  an execution client, or when the voted block's payload is unvalidated.
- The gloas validator-client tests serve through an idle execution client,
  and the optimistic-head test names the fork attestation_data now takes.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
- Gloas block production (produceBlockV4) appends the client versions to
  the graffiti the way fulu's does.
- The validator client's gloas proposal and duties tests take their
  graffiti and fee recipient from ProposerSettings, and a validator's own
  graffiti is checked to reach a gloas block request.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
- The BLS fixture tests follow main's cryptography-specs layout, and main's
  Makefile (gossip vectors in the pinned tree) replaces the separate gossip
  tree this branch extended.
- Operations under a gloas head are IGNOREd and a gloas-shaped attester
  slashing has no consumer; gloas operation vectors are not run.
- Published columns and operations go out under the slot's own digest, and
  column subnets use compute_subnet_for_data_column_sidecar.
- Gloas block production packs from the store's attestation pool, and a
  published gloas block gossips no sidecars (its columns follow the
  envelope). The payload attestation pool stays a shared handle.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
the write cursor also drives gloas's progressive registry. A cursor pass
crosses a ProgressiveList's subtrees with global indices, and
try_update_validators_with_balances and try_update_balances work on any
BeaconState, so electra's effective-balance step and the reward deltas
write through them too.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
gloas block processing (its attestations and sync aggregate) reads the
total active balance from the same cache, which gloas block production
passes in, and the cache's registry scan walks iter_validators so it
covers gloas's progressive registry.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
compute_node_weights' vote loop reads the justified-balances snapshot too,
and the fork-choice fixture runner compares every gloas node's weight
with the specification's gloas_get_weight at each checks step.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
gloas's progressive inactivity scores and ring buffers go through the same
element accessors (inactivity_score_mut, InactivityScoresRef), and
ProgressiveList is Buffered. Main's new tests write block_roots by index
rather than through iter_mut, which a tree-backed vector has not got.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663); no conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
the precompute covers gloas under the same (epoch, root) key, since a
gloas block's post-state never holds its own payload; a gloas test checks
the worker's state against checkpoint_state and an import's own advance.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to
#639's runtime fork-digest switch: the scoring module doc says how the
scored topics follow the current digest and what the switch leaves
unscored.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663); no conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack this branch sat on (#634, #635, #639) plus
#662 and #663. Merged against #639's squash as the base, since this branch
already carries feat/beacon-gloas-live, whose tree #639's squash matches:
what comes in is #662 and #663 only, without conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack this branch sat on (#634, #635, #639) plus
#662 and #663. Merged against #639's squash as the base, since this branch
already carries feat/beacon-gloas-live, whose tree #639's squash matches:
what comes in is #662 and #663 only, without conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack this branch sat on (#634, #635, #639, #662)
plus #663. Merged against #662's squash as the base, since this branch
already carries feat/beacon-gloas-validator-duties, whose tree #662's
squash matches: what comes in is #663 only, without conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack this branch sat on (#634, #635, #639, #662)
plus #663. Merged against #662's squash as the base, since this branch
already carries feat/beacon-gloas-validator-duties, whose tree #662's
squash matches: what comes in is #663 only, without conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to
#663: the sync duties share its state_for_epoch and duties_response, and
the tests its beacon_store_at_clock, rather than carrying copies.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663); no conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663); no conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
- The key-layout bump is DB_VERSION 7, since main spends 6 on #634's gloas
  config keys.
- A gloas fork-choice test stores a state whose header slot is its block's,
  since states are keyed by that slot, and the slot-peek test covers a
  gloas block, now the newest fork.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663); no conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663); no conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663); no conflicts.
MegaRedHand added a commit that referenced this pull request Oct 26, 2026
Main gained the gloas stack (#634, #635, #639, #662, #663). Adapted to it:
gloas aggregates are checked under the schedule's attester domain too
(is_valid_indexed_attestation_with_domain), main's attestation_for test
helper signs under the schedule's domain, and the test store helpers are
main's beacon_store_with_config.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

beacon Ethereum Beacon Chain client

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants