Skip to content

fix(evm): skip lagging upstreams for eth_call(latest) and mirror head lag to */All - #25

Open
1marcghannam wants to merge 3 commits into
feat/websocket-supportfrom
fix/eth-call-latest-skip-lagging-upstreams
Open

1marcghannam wants to merge 3 commits into
feat/websocket-supportfrom
fix/eth-call-latest-skip-lagging-upstreams

Conversation

@1marcghannam

@1marcghannam 1marcghannam commented Sep 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Incident: Priority Pool totalQueued via eRPC returned 16,355 instead of ~692 because a stalled WS peer (internal-eth-mainnet-reth-ws-0) still answered eth_call("latest"). Tip-pin image only covers getBlock.
  • Root cause: eth_call("latest") resolves to bn=0 and fail-opens availability gating; {*,All} BlockHeadLag was not mirrored for peer upstreams so network-scope lag predicates can no-op (upstream #1019).
  • Fix (on tip-pin lineage):
    1. Hard gate: skip upstreams lagging TipHW by >16 for latest-state methods + latest/pending/empty tag.
    2. Tip partition: prefer near-tip peers first for those reads (failover still allowed).
    3. Tracker fix(health): lag predicates silently no-op for CB-open upstreams under evalScope:network erpc/erpc#1019 port: mirror BlockHeadLag into {ups,"*",All}.

Base: fix/pin-near-tip-getblock-to-evm-leader (LinkPool tip-pin / WS lineage — not upstream main).

Test plan

  • TestNetwork_EthCallLatest_SkipsLaggingUpstream
  • TestNetwork_EthCallConcreteBlock_AllowsLaggingArchive
  • TestWildcardLagMirroredForPeerUpstreams
  • Near-tip getBlock pin still green

Rollout

  1. Build/publish image from this tip-pin + eth_call lineage.
  2. Bump internal-erpc image tag.
  3. Verify Priority Pool totalQueued via eRPC ≈ 692; no stalled reth-0 while behind TipHW.

@github-actions

github-actions Bot commented Sep 15, 2026

Copy link
Copy Markdown
File Lines Key changes Risk
🟠 networks.go +110/-0 maxLatestStateLagBlocks, isLatestStateReadMethod, isMovingLatestOrPendingTag, ... ⚠ ErrUpstreamBlockUnavailable
🔵 tracker.go +19/-0 loadOrStoreUpsMetrics
3 test files +381

xray — see through AI slop with deterministic architecture PR diff reviews

@1marcghannam
1marcghannam changed the base branch from main to fix/pin-near-tip-getblock-to-evm-leader September 15, 2026 09:40
@1marcghannam
1marcghannam force-pushed the fix/eth-call-latest-skip-lagging-upstreams branch from d9a3a20 to ac2e98e Compare September 15, 2026 09:41
… lag to */All

Stale eth_call("latest") could be served by a WS peer thousands of blocks behind TipHW because tip integrity only covered getBlock and bn=0 failed open. Skip peers lagging >16 for moving-tag state reads, prefer near-tip order, and mirror BlockHeadLag into {*,All} so network-scope lag predicates see quiet/stalled peers.
…skip

Use the stalled-vs-tip totalQueued values (16355 vs 692), ~8k block lag,
and the real selector so the regression mirrors the production failure mode.
@1marcghannam
1marcghannam force-pushed the fix/eth-call-latest-skip-lagging-upstreams branch from c82e538 to 0c96ef1 Compare September 15, 2026 09:50
@1marcghannam
1marcghannam changed the base branch from fix/pin-near-tip-getblock-to-evm-leader to feat/websocket-support September 15, 2026 09:51
…tateLagBlocks

The 16-block hard gate mirrored the default selection policy's
blockNumberLagAbove(16) but was not operator-tunable, unlike the policy
threshold. Add evm.maxLatestStateLagBlocks (nil -> 16, <=0 disables),
resolved at the read site like MaxRetryableBlockDistance. Also correct
the lag-test comments: lag comes from the standard poller mocks
(0x11118888 vs 0x22228888), not the no-op SuggestLatestBlock calls.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant