Repository navigation
feat(discovery): hunt attestation subnet peers with discv5 predicate queries - #732
Draft
MegaRedHand wants to merge 10 commits into
Draft
MegaRedHand wants to merge 10 commits into
MegaRedHand wants to merge 10 commits into
Conversation
ethrex used to pace its discv5 lookups off its peer table's connected count, which only RLPx could fill. A libp2p client registered nothing, so the count was permanently 0, completion was permanently 0, and lookups never eased off the 500ms startup rate: roughly 20x the intended steady-state FindNode traffic, for the life of the process. ethrex PR #7217 moves the contact table into the discovery server and gives a consumer `DiscoveryHandle::record_peer_event`, so any consumer can report its own connections whatever transport they run over. Pin the three ethrex crates to that PR's head and report every libp2p connection opened and closed. `--discovery.target-peers` now feeds the pacing it was deliberately kept out of, and `PEER_TABLE_TARGET_PEERS`, the placeholder that only existed to keep the old division off `NaN`, is gone. Reporting needs the discv5 node id behind a libp2p `PeerId`. Both names derive from the same secp256k1 key, and libp2p inlines a key that short into the peer id, so it is recoverable: `node_id` does that round trip, and pins it against `Node::from_enr` so the two halves cannot drift. The peer table is gone from this crate with it. Two consequences: - The filter is what discovery now takes, and `next_dial_candidate` hands back a `Node` rather than a `Contact`, so the ENR the dial loop used to read is no longer in reach. Judging a record is the one moment we hold it, so `LeanFilter` files each admitted peer's multiaddr and `attnets` in `DialTargets` as it goes, and the dial loop looks the node id back up there. Discovery still decides *whether* to dial; we only supply *how*. - `--discovery.advertise-ip` maps onto ethrex's new `nat_extip_set`, so an operator who names an external address keeps it: discv5's PONG voting no longer overwrites what was explicitly asked for. The pin is a commit rather than a branch, for the reason the release tag it replaces was chosen over `main`. It goes back to a tag once #7217 is in a release.
Brings in the Beacon Chain support (#661). Main moved the dial loop's peer-table draws onto a bounded contact-poll task, so that task now asks the discovery handle for nodes and looks their dial targets up instead. The filter takes main's fork id, and #605's node-id recovery from a PeerId reuses the decode main added for custody rather than a second copy of it. Cargo.lock is main's with only the ethrex sources moved to the pinned revision.
…dicate-queries Folds #605's move onto ethrex's contact-table discovery API into this branch, since ethrex's predicate queries are built on that API. Conflicts resolved in discovery/admission.rs and discovery/dial.rs: main had made LeanFilter cloneable with a shared fork-boundary Policy, and #605 had made it non-Clone because discovery takes it by value. Both hold now: clones share the Policy and the DialTargets map, and SpawnedDiscovery carries a clone, so set_fork_id still reaches the copy discovery runs.
The branch adds DiscoveryHandle::find_nodes and records_matching, discv5 predicate queries: discovery runs a consumer's test on every record a walk sees and answers with the matches whose nodes replied. That is what lets the dial loop look for peers on a given attestation subnet instead of waiting for the crawl to stumble on one. It is built on ethrex #7217, which the previous pin was, so the API this crate uses is unchanged and it builds as is. The lockfile moves only the ethrex crates (v25.0.0 -> v30.0.1 at 8f847524); make cooldown-check passes. A commit rather than the branch, for the same reason as before: what this crate builds against must not move without the manifest saying so.
…live ENR The dial loop ranks what the crawl happened to find, which only reorders random draws. About one mainnet record in 32 advertises a given subnet, so a subscribed subnet nobody connected covers could stay uncovered for as long as the crawl took to stumble on a peer for it. The subnet hunt (discovery/subnets.rs) searches for them instead. Each dial tick publishes the subscribed subnets with fewer than 3 connected peers advertising them (Lighthouse's TARGET_SUBNET_PEERS). A task off the P2P actor takes the matches discovery already holds, then walks discv5 with find_nodes for the rest, one hunt at a time, at most 6 peers each, 30 s apart. The predicate is admission plus one wanted attnets bit, as in Lighthouse's subnet predicate. Finds join the dial loop's candidate queue and rank after custody gaps, ahead of other subnets. Nothing is hunted while the loop has no budget to dial, and a peer handed over is left out of hunts for 10 minutes, since discovery keeps offering a mainnet peer that connected and said goodbye. /eth/v1/node/identity now reports the ENR discv5 serves rather than the one built at startup. ethrex mirrors every record it re-signs (on discv5 IP voting) into the SharedLocalNode it was spawned with; LocalEnr reads that mirror, and the RPC crate gets it as an EnrSource closure, since it does not depend on ethlambda-p2p. The other direction stays unsupported, now documented against the new API: ethrex has no message that edits its local record. Its one consumer-driven edit writes the EIP-2124 `eth` entry, not `eth2`, and a write into the mirror is overwritten by discovery's next edit. So the served eth2 still keeps the startup digest across a fork boundary.
The endpoint reads discovery's own copy of the record since the previous commit, so after IP voting it reports the re-signed record, IP included, rather than the startup one the paragraph described.
This was referenced Oct 29, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft: pins ethrex to
8f847524, the head of ethrex'sfeat/discv5-predicate-queries. That branch has no ethrex PR yet and is built on the open lambdaclass/ethrex#7217. Not mergeable until both are in an ethrex release and the pin goes back to a tag.Includes #605. This branch merges
feat/ethrex-discovery-contact-table(#605), since predicate queries are built on the contact-table API that PR migrates to. #605 can be closed in favor of this one, or merged first; the diff here then shrinks to the last three commits.Motivation
The dial loop ranks what discv5's crawl happens to find, which only reorders random draws. About one mainnet record in 32 advertises a given attestation subnet, so a subnet this node subscribes to can stay without peers for as long as the crawl takes to stumble on one. ethrex's new
DiscoveryHandle::find_nodesandrecords_matchinglet the search be aimed instead, the way Lighthouse uses sigp/discv5'sfind_node_predicate.What this does
1. ethrex pin bump
v25.0.0-based #7217 head (4462253e) to8f847524(v30.0.1+ the branch). Builds with no API changes: the contact-table API is the same. The lockfile moves only ethrex crates;make cooldown-checkpasses.2. Subnet hunt (
discovery/subnets.rs)attnetsadvertises) with fewer thanTARGET_SUBNET_PEERS= 3 connected peers advertising it (Lighthouse's number)LeanFilter) plus at least one wantedattnetsbit; the bit test runs first so most records are turned away before anything is decodedrank_candidatesgains a tier: custody gaps, then wanted subnets, then any uncovered subnetlean_discovery_subnet_hunt_peers_total3. ENR sync between discv5 and the API
SharedLocalNodeit is spawned with.discovery::LocalEnrreads that mirror, and/eth/v1/node/identityreads it per request throughethlambda_rpc::EnrSource(a closure, since the RPC crate does not depend onethlambda-p2p).DiscoveryServerProtocolhas no message that edits the local record. Its only consumer-driven edit, thefork_idwatch, writes the EIP-2124ethentry, noteth2. Writing into the mirror does nothing: discovery serves its own copy and overwrites the mirror on its next edit. So the servedeth2still keeps the startup digest across a fork boundary (update_served_enrstill only warns; its doc now says this). What ethrex would need: a protocol message that applies an edit throughNodeRecord::edit(oneseqbump, re-signed) and mirrors the result, asrefresh_fork_idalready does foreth.Merge resolution with #605
Main had made
LeanFilterClonewith a shared fork-boundaryPolicy; #605 had made it non-Clonebecause discovery takes it by value. Both hold now: clones share thePolicyand theDialTargetsmap, andSpawnedDiscoverycarries a clone, soset_fork_idstill reaches discovery's copy. #605's behavior change is inherited:--discovery.advertise-ipsetsnat_extip_set, so IP voting no longer overwrites an address the operator named.Assumptions
beacon_committee_subscriptions) are not fed to the hunt; that stays a separate gap.attnetsof peers dialed from discovery. Inbound peers never show us an ENR, so they count for nothing: the hunt errs toward hunting.Testing
make lint,cargo check --workspace --all-targets --locked,make cooldown-checkcargo test --workspace --profile release-fast --lib --bins: all pass. leanSpec spec suites not run.offer's limit and dedup,short_subnets, the no-budget gate, and the new ranking tier.discv5_server_testscoverfind_nodes; this side tests what it feeds it and what it does with the answer.Follow-ups
eth2at fork boundaries.records_matchingdoes not exclude ethrex's tried set, so a hunt can re-offer a peer the contact poll already handed out within the same 10 minutes; the dial loop absorbs it (AlreadyInProgress, skip if connected).