Repository navigation
[simplex]: recover a submitted BRIDGE top-up placement from on-chain evidence and retry transient tracking errors - #1412
Conversation
Wizdave97
left a comment
There was a problem hiding this comment.
Verdict: changes needed before merge. I reviewed only this PR's commit on top of #1408.
These parts look right:
- The typed
SubmittedUnconfirmedError. - The EntryPoint-nonce rule: it uses the same key and compares the full nonce.
- The deadline-bounded retry on
FAILED.
But several of the "settled only on evidence" paths aren't actually proof, or can never trigger, and one of them can lock a reclaim for good.
Must fix
-
An unconfirmed cancel hash is handed back forever (BridgeTopUp.ts:1332-1345).
- A native
cancelOrderthat gets no receipt within 60 s now storesreclaimTxHash. - Suppose that tx is later dropped (its nonce reused) or mines and reverts. Every later attempt hands the hash back to the SDK, which either times out or throws
GetRequest missing. - After 5 failures the top-up goes back to
unfilled.reclaim()keepsreclaimTxHash, so no new cancel is ever sent, and the escrow stays locked until someone edits the state by hand. - Before this PR, only a mined, successful cancel was stored.
- Suggestion: store the hash as unconfirmed and check its receipt before handing it back. If it succeeded, hand it back. If it reverted, or it isn't found and the wallet nonce has moved past it, clear it and send a new cancel.
- A native
-
A record without
placeSubmittedAtnever ages out ofplacing(BridgeTopUp.ts:797, :1575).submittedAtfalls back toupdatedAtwhenplaceSubmittedAtis missing. Butupdate()rewritesupdatedAton every write, including the cursor write at :728 just beforesettleUnplaceable.- So
Date.now() - submittedAtis always about 0, and neitherPLACEMENT_DROPPED_MSnorPLACEMENT_WINDOW_MScan ever pass. - The success path (:586) writes only
placeTxHash. IfrecordPlacementthen can't re-read the receipt (for example a bundle reorged out and never re-bundled), the record staysplacingforever and blocksstart(). Oldfailed/placingrecords have the same problem. - The success path also never records
placeUserOpHashfor a sponsored send. Once the timing is fixed, a bundle hash that vanished would be settled as a dropped native tx. - Every test sets
placeSubmittedAt, so the fallback is never exercised. - Suggestion: write
placeSubmittedAtand the UserOp details on the success path too. For old records, fall back tostageTimes.placing ?? createdAt.
-
A recovery still running for an old record writes into the new one (BridgeTopUp.ts:377, :1569).
restore()re-runsrecoverPlacementfor everyfailedrecord that has a hash and a session key. That now includes every record this PR settles as failed.faileddoesn't count as pending, sostart()lets the operator begin a new top-up while that re-check is still running (at least 120 s of receipt wait, plus a scan).- When the re-check finishes, its
update({placeScannedTo})andupdate({stage: "failed"})land on the new record. A live new top-up can then showfailed, and a third top-up can replace it while its order still holds escrow (c03 again). - Only the retry timers are tied to their record; these writes aren't.
- Suggestion: tie
update()to a record id, so it does nothing whenthis.record.id !== id, and/or havestart()refuse while a recovery is in flight.
-
"A native tx the node has forgotten" is not proof it can never be mined (BridgeTopUp.ts:808-817).
- A pending tx can be missing from one load-balanced backend, or from one node's mempool, and still be valid elsewhere until its nonce is used.
- Writing
failed/ "Nothing was escrowed" at that point means a later inclusion places an order that nothing tracks. - Suggestion: record the tx nonce when submitting, and settle only when
getTransactionCount(scannedTo) > nonceand the scan found nothing.
Should fix
-
Evidence is read at the chain tip (:868).
- viem's
fallbackcan send agetLogspage the primary endpoint refuses (for example a 2000-block range) to the next endpoint. If that endpoint is behind, it silently returns an empty page. - Nonces are read with no margin for reorgs.
- The Permit2 expiry rule compares against the host clock, not
getBlock(scannedTo).timestamp. placeFromBlockhas no margin below it.- Suggestion: read nonces a few blocks below
scannedTo, compare the deadline with the block timestamp, start the scan a few blocks belowplaceFromBlock, and save the cursor after each chunk.
- viem's
-
recordPlacementdoesn't filterOrderPlacedlogs by gateway address (:609).- A bundle receipt carries other ops' logs, and our session address is visible in the mempool. An earlier op in the same bundle could therefore emit a forged
OrderPlacedfor our session, and we would track a fake order. - The scan path already filters by address. Add the same
log.address === gatewaycheck here.
- A bundle receipt carries other ops' logs, and our session address is visible in the mempool. An earlier op in the same bundle could therefore emit a forged
Cross-PR:
- A native
sendTransactionthat throws after the node accepted it (an RPC timeout, then "already known" on retry) is still a plain error. The placement then endsfailedwith no hash. #1413 only covers bundler sends. - #1411 adds the same "refuse a native batch on an undelegated wallet" rule for vault batches as
TokenSender's guard here. One shared helper would keep the two consistent.
Docs: the changelog note says old records are "dated by updatedAt" and recovered "the same way", which (2) shows doesn't work. At 125 lines it also mostly restates the code and could be much shorter.
- Store a cancel sent without a receipt as reclaimSubmission (tx hash and nonce, or UserOp hash, nonce and key) and check it before handing it to the SDK: a success is handed back, a revert or a consumed nonce clears it and sends a new cancel, anything else waits. A timed-out UserOp cancel is no longer resent while it can land. - Write placeSubmittedAt, and for a sponsored send the UserOp's hash, nonce, key and expiry, on the success path too; date a legacy record by stageTimes.placing then createdAt, never updatedAt. - Bind every background flow's writes to its record id, so a recovery for a replaced record cannot write into the next top-up. - Send native placements with an explicit pending nonce and carry it in the submission; settle a native placement only when the wallet's nonce is past it, and drop the "forgotten for 30 minutes" rule. - Read nonces and the permit clock below the scanned tip (chain timestamp, not the host clock), start the scan a reorg margin below placeFromBlock, and save the scan cursor after each chunk. - Filter OrderPlaced logs by the gateway address in recordPlacement. - Shorten the changelog note to the final behaviour and evidence rules.
21e1c9d to
f44b236
Compare
- Store a cancel sent without a receipt as reclaimSubmission (tx hash and nonce, or UserOp hash, nonce and key) and check it before handing it to the SDK: a success is handed back, a revert or a consumed nonce clears it and sends a new cancel, anything else waits. A timed-out UserOp cancel is no longer resent while it can land. - Write placeSubmittedAt, and for a sponsored send the UserOp's hash, nonce, key and expiry, on the success path too; date a legacy record by stageTimes.placing then createdAt, never updatedAt. - Bind every background flow's writes to its record id, so a recovery for a replaced record cannot write into the next top-up. - Send native placements with an explicit pending nonce and carry it in the submission; settle a native placement only when the wallet's nonce is past it, and drop the "forgotten for 30 minutes" rule. - Read nonces and the permit clock below the scanned tip (chain timestamp, not the host clock), start the scan a reorg margin below placeFromBlock, and save the scan cursor after each chunk. - Filter OrderPlaced logs by the gateway address in recordPlacement. - Shorten the changelog note to the final behaviour and evidence rules.
724278b to
4560cc4
Compare
Wizdave97
left a comment
There was a problem hiding this comment.
Requesting changes for one new issue. Of the six round-1 findings, five are fixed and (5) is partially fixed.
Must fix
- A sponsored cancel waits forever when the bundler can't be asked (BridgeTopUp.ts:1543-1556).
- Scenario: the bundler accepted the cancel op, then lost it or went down, or its URL was removed.
userOpReceiptthen throws on every attempt, before the nonce and Permit2 checks run. - After 6 attempts the top-up returns to
unfilled, and every reclaim repeats the same loop. - The op never landed, so the escrow stays locked.
- Suggested fix:
- When the lookup fails, still apply the rule "nonce unchanged at the evidence block and Permit2 deadline passed". That proves the op can never land, and it needs no bundler.
- The comment says nothing independent backs the cancel, but
cancelOrderemitsOrderCancelled(commitment, canceller)on every successful cancel (IntentGatewayV2.sol:491). Scanning for it withrecord.commitment, from a block stored with the submission, settles the cancel the same way the placement scan does.
- Scenario: the bundler accepted the cancel op, then lost it or went down, or its URL was removed.
Should fix
-
A duplicate cross-chain cancel doesn't revert; it pays the relayer fee again.
_cancelFromSource(ExtrinsicIntents.sol:143) checks only the creator and the height. A second cancel therefore dispatches another GET, anddispatchWithFeeTokenpulls the fee a second time.- The comments at :1547 and :1579-1580, and line 116 of the note, say a duplicate "can only revert".
settleLegacyCancelclears a sponsored cancel's bundle-tx hash when a reorg re-bundles the op, which triggers that second cancel.- It also compares the bundle tx's nonce, which is the bundler EOA's, with the filler wallet's nonce (:1600).
- Suggested fix: keep
reclaimSubmissionuntilORDER_CANCELEDis recorded, and checktx.from === solverbefore using its nonce.
-
The 64-block evidence margin is the same on every chain (:137). That's about 16 s on Arbitrum.
- A fallback endpoint lagging by more than that serves an empty
getLogspage, and the saved cursor then skips that range for good. - Suggested fix: size the margin in seconds, using the block time measured at startup.
- A fallback endpoint lagging by more than that serves an empty
Non-blocking:
- Silent drop. A re-check of a replaced record that finds a live order drops it without logging (:616). Log the commitment at warn level.
- Iteration history in docs. Note lines 112-117, and the comments at :1386-1388 and :1576-1580, describe "a
reclaimTxHashthat older code stored from an unconfirmed send". No released code did that; it was this PR's own earlier commit. - PR description. It still says a forgotten native tx is settled, that an unconfirmed cancel is stored as
reclaimTxHash, and that a UserOp cancel is "still re-sent".
- Store a cancel sent without a receipt as reclaimSubmission (tx hash and nonce, or UserOp hash, nonce and key) and check it before handing it to the SDK: a success is handed back, a revert or a consumed nonce clears it and sends a new cancel, anything else waits. A timed-out UserOp cancel is no longer resent while it can land. - Write placeSubmittedAt, and for a sponsored send the UserOp's hash, nonce, key and expiry, on the success path too; date a legacy record by stageTimes.placing then createdAt, never updatedAt. - Bind every background flow's writes to its record id, so a recovery for a replaced record cannot write into the next top-up. - Send native placements with an explicit pending nonce and carry it in the submission; settle a native placement only when the wallet's nonce is past it, and drop the "forgotten for 30 minutes" rule. - Read nonces and the permit clock below the scanned tip (chain timestamp, not the host clock), start the scan a reorg margin below placeFromBlock, and save the scan cursor after each chunk. - Filter OrderPlaced logs by the gateway address in recordPlacement. - Shorten the changelog note to the final behaviour and evidence rules.
4560cc4 to
26ad6c9
Compare
…evidence and retry transient tracking errors
- Store a cancel sent without a receipt as reclaimSubmission (tx hash and nonce, or UserOp hash, nonce and key) and check it before handing it to the SDK: a success is handed back, a revert or a consumed nonce clears it and sends a new cancel, anything else waits. A timed-out UserOp cancel is no longer resent while it can land. - Write placeSubmittedAt, and for a sponsored send the UserOp's hash, nonce, key and expiry, on the success path too; date a legacy record by stageTimes.placing then createdAt, never updatedAt. - Bind every background flow's writes to its record id, so a recovery for a replaced record cannot write into the next top-up. - Send native placements with an explicit pending nonce and carry it in the submission; settle a native placement only when the wallet's nonce is past it, and drop the "forgotten for 30 minutes" rule. - Read nonces and the permit clock below the scanned tip (chain timestamp, not the host clock), start the scan a reorg margin below placeFromBlock, and save the scan cursor after each chunk. - Filter OrderPlaced logs by the gateway address in recordPlacement. - Shorten the changelog note to the final behaviour and evidence rules.
26ad6c9 to
cd10f6c
Compare
Stacked on #1408. Review only the commits above it; the base will move to
mainonce #1408 merges.This PR fixes BRIDGE top-up placement, tracking and reclaim-cancel recovery, from the 2026-10-15 simplex audit: c03 and c15 (medium), c58 and c44 (low), a stale-retry bug, and the native-batch counterpart of c14.
Why:
placeTxHashwas stored. The top-up was then markedfailed("nothing escrowed"), a non-pending stage. A new top-up could overwrite the record and its session key while the order had in fact landed and was holding escrow.FAILED, marked the orderunfilledfor good._cancelFromSourcechecks only the creator and the height, so it dispatches another GET request anddispatchWithFeeTokenpulls the relayer fee again.Fix (
src/services/BridgeTopUp.ts,UserOpSender.ts,TokenSender.ts, newsubmission.ts,src/data/types.ts):UserOpSender(no receipt) andTokenSender(any receipt-wait error after broadcast; an explicit revert is still thrown as such) throwSubmittedUnconfirmedError. It carries either the UserOp hash, nonce, key, permit expiry and submission time, or the tx hash, nonce and submission time.TokenSender.submitCallsalso returns thatsubmissionon success.UserOpSender.lookupUserOpReceipttells "unknown to the bundler" apart from "couldn't ask".place()storesplaceSession(the order's session-key address) andplaceFromBlock. After the send, successful or not, it stores the submission (placeTxHashandplaceTxNonce, orplaceUserOpHash,placeUserOpNonce,placeUserOpNonceKey,placeUserOpExpires) andplaceSubmittedAt.OrderPlacedlogs for our session, with a persisted cursor (placeScannedTo);OrderPlaced;placeTxNonceat the scanned height (a node that has merely forgotten the tx proves nothing, so a record with a nonce keeps waiting; only a legacy record with no nonce and an unknown tx is settled, unverified);max(64, ceil(EVIDENCE_MARGIN_SECONDS / blockTimeSeconds))blocks, withEVIDENCE_MARGIN_SECONDS = 300. The block time is the chain's viemchain.blockTime, else 0.25 s (a larger, safer margin), so the margin is 1200 blocks on a 0.25 s chain and 64 on a 12 s one. Every scan restarts that margin below its saved cursor, so a page a lagging backend served empty is read again. The cursor is still saved per chunk.recordPlacementpicks theOrderPlacedwhosesessionis ours (userfor legacy records), not the first one in the bundle. When a recovery for a replaced record finds a live order, it logs its commitment atwarnand writes nothing.awaiting_bids/fillingcan becomeunfilled. A stale timer can therefore no longer mark a never-placed orderunfilledand thenreclaimed.EXPIRED, orFAILEDafter the deadline, meansunfilled. A dashboard sync moves anunfilledtop-up whose order did fill todelivering.deliveryRequesttakes the fill receipt'sSentlog from the BRIDGE token to our Hyperbridge key, using the amount to break ties. The indexer is used only as a fallback, and only when it returns exactly one node.reclaimSubmission(tx hash and nonce, or UserOp hash, nonce, key,expires,submittedAt) withfromBlock, the paying chain's height read before the send, andscannedTo, the scan cursor. A confirmed send also storesreclaimTxHash. The submission is kept until theORDER_CANCELEDstep is recorded, in the same write, so a bundle tx that is reorged out and re-bundled is still found.OrderCancelled(commitment, canceller)logs for the order's commitment, from an evidence margin belowfromBlock, where a log's transaction is the cancel and a scan failure waits; (3) with no log up toscannedTo, the nonce rules at an evidence margin below it: the wallet's nonce past a tx's, the EntryPoint's past an op's, or an unchanged nonce withexpiresset and the evidence block's timestamp at or pastsubmittedAtplus the permit window, each of which means a new cancel is sent; (4) anything else waits.reclaimSubmissionwithoutfromBlock(no release wrote one) waits withno recorded start block. AreclaimTxHashwith no submission is a confirmed cancel, since it was only ever written after a confirmed send, and is handed back as it is.TokenSendernative batch refuses an undelegated wallet, where it would succeed and do nothing.boot.tsshares one read-onlyDelegationServicebetweenTokenSenderandBridgeTopUp.Validation
All run this round, from
sdk/orsdk/packages/simplex:src/tests/bridge-top-up.test.ts91 passed,token-sender.test.ts26,services/UserOpSender.test.ts13,ui/src/operator104; 234 passed in total. New cases cover: a bundler that cannot be asked, with a log found (its tx is handed back), with a moved nonce (resend), with an unchanged nonce and an expired permit (resend), with an unchanged nonce and a live permit (wait), and with a failed scan (wait); the margin on a 0.25 s, undeclared and 12 s chain; a rescan starting below its cursor for both placement and cancel; and the warning for a replaced record's live order.pnpm --filter="@hyperbridge/simplex" test:data146 passed;test:unit263 passed;test:filler70 passed, 5 skipped.pnpm --filter="@hyperbridge/simplex" lint: no errors; Biome is clean on the changed files.npx tsc --noEmit -p .: the same 14 errors as main (3 of them in theoxdependency), none in changed files.Not covered: