Skip to content

[simplex]: recover a submitted BRIDGE top-up placement from on-chain evidence and retry transient tracking errors - #1412

Open
seunlanlege wants to merge 5 commits into
fix/simplex-top-up-reclaim-fee-tokenfrom
fix/simplex-top-up-placement-recovery
Open

seunlanlege wants to merge 5 commits into
fix/simplex-top-up-reclaim-fee-tokenfrom
fix/simplex-top-up-placement-recovery

Conversation

@seunlanlege

@seunlanlege seunlanlege commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Stacked on #1408. Review only the commits above it; the base will move to main once #1408 merges.

This PR fixes BRIDGE top-up placement, tracking and reclaim-cancel recovery, from the 2026-10-15 simplex audit: c03 and c15 (medium), c58 and c44 (low), a stale-retry bug, and the native-batch counterpart of c14.

Why:

  • A placement whose UserOp or tx was broadcast but got no receipt in time threw before placeTxHash was stored. The top-up was then marked failed ("nothing escrowed"), a non-pending stage. A new top-up could overwrite the record and its session key while the order had in fact landed and was holding escrow.
  • A transient error while tracking a live order, or the SDK's FAILED, marked the order unfilled for good.
  • A reclaim cancel with no receipt was either re-sent, or waited on forever when the bundler could not be asked about it. A second cross-chain cancel does not revert: _cancelFromSource checks only the creator and the height, so it dispatches another GET request and dispatchWithFeeToken pulls the relayer fee again.

Fix (src/services/BridgeTopUp.ts, UserOpSender.ts, TokenSender.ts, new submission.ts, src/data/types.ts):

  • Typed unconfirmed submissions.
    • UserOpSender (no receipt) and TokenSender (any receipt-wait error after broadcast; an explicit revert is still thrown as such) throw SubmittedUnconfirmedError. It carries either the UserOp hash, nonce, key, permit expiry and submission time, or the tx hash, nonce and submission time.
    • TokenSender.submitCalls also returns that submission on success.
    • The message text is unchanged, so other callers behave as before.
    • UserOpSender.lookupUserOpReceipt tells "unknown to the bundler" apart from "couldn't ask".
  • The placement stays pending and is recovered in-process.
    • Before submitting, place() stores placeSession (the order's session-key address) and placeFromBlock. After the send, successful or not, it stores the submission (placeTxHash and placeTxNonce, or placeUserOpHash, placeUserOpNonce, placeUserOpNonceKey, placeUserOpExpires) and placeSubmittedAt.
    • Recovery runs every 60 s and tries, in order:
      1. the tx receipt;
      2. the bundler receipt;
      3. a chunked scan of the paying chain's OrderPlaced logs for our session, with a persisted cursor (placeScannedTo);
      4. settling.
    • "Nothing was escrowed" is written only on evidence:
      • a mined receipt with no matching OrderPlaced;
      • a reverted batch;
      • a native tx whose wallet nonce has moved past placeTxNonce at the scanned height (a node that has merely forgotten the tx proves nothing, so a record with a nonce keeps waiting; only a legacy record with no nonce and an unknown tx is settled, unverified);
      • a UserOp whose EntryPoint nonce moved, read at the scanned height, or whose Permit2 permit expired by the chain's own clock at that block.
    • Inconclusive cases retry. A bundler-reported tx that is not found falls back to the UserOp rules, because a reorged-out op can be re-bundled under a new hash.
    • An EIP-2612 bootstrap op has no expiry. It stays pending until its nonce moves or its log appears.
  • Evidence margin sized in seconds. Evidence is read, and a scan starts, an evidence margin below the height it reached: max(64, ceil(EVIDENCE_MARGIN_SECONDS / blockTimeSeconds)) blocks, with EVIDENCE_MARGIN_SECONDS = 300. The block time is the chain's viem chain.blockTime, else 0.25 s (a larger, safer margin), so the margin is 1200 blocks on a 0.25 s chain and 64 on a 12 s one. Every scan restarts that margin below its saved cursor, so a page a lagging backend served empty is read again. The cursor is still saved per chunk.
  • recordPlacement picks the OrderPlaced whose session is ours (user for legacy records), not the first one in the bundle. When a recovery for a replaced record finds a live order, it logs its commitment at warn and writes nothing.
  • Retry timers are bound to their record, and only awaiting_bids/filling can become unfilled. A stale timer can therefore no longer mark a never-placed order unfilled and then reclaimed.
  • Tracking retries until the order's deadline (BSC height, strict comparison). EXPIRED, or FAILED after the deadline, means unfilled. A dashboard sync moves an unfilled top-up whose order did fill to delivering.
  • deliveryRequest takes the fill receipt's Sent log from the BRIDGE token to our Hyperbridge key, using the amount to break ties. The indexer is used only as a fallback, and only when it returns exactly one node.
  • Reclaim cancel settled from on-chain evidence.
    • Every cancel the filler sends is stored as reclaimSubmission (tx hash and nonce, or UserOp hash, nonce, key, expires, submittedAt) with fromBlock, the paying chain's height read before the send, and scannedTo, the scan cursor. A confirmed send also stores reclaimTxHash. The submission is kept until the ORDER_CANCELED step is recorded, in the same write, so a bundle tx that is reorged out and re-bundled is still found.
    • Before a stored cancel is handed to the SDK it is settled in this order: (1) a tx's receipt by hash, or a UserOp's bundler receipt if the bundler answers (a failed lookup is not fatal); (2) a chunked scan of the paying chain's IntentGateway OrderCancelled(commitment, canceller) logs for the order's commitment, from an evidence margin below fromBlock, where a log's transaction is the cancel and a scan failure waits; (3) with no log up to scannedTo, the nonce rules at an evidence margin below it: the wallet's nonce past a tx's, the EntryPoint's past an op's, or an unchanged nonce with expires set and the evidence block's timestamp at or past submittedAt plus the permit window, each of which means a new cancel is sent; (4) anything else waits.
    • A reclaimSubmission without fromBlock (no release wrote one) waits with no recorded start block. A reclaimTxHash with no submission is a confirmed cancel, since it was only ever written after a confirmed send, and is handed back as it is.
  • TokenSender native batch refuses an undelegated wallet, where it would succeed and do nothing. boot.ts shares one read-only DelegationService between TokenSender and BridgeTopUp.

Validation

All run this round, from sdk/ or sdk/packages/simplex:

  • src/tests/bridge-top-up.test.ts 91 passed, token-sender.test.ts 26, services/UserOpSender.test.ts 13, ui/src/operator 104; 234 passed in total. New cases cover: a bundler that cannot be asked, with a log found (its tx is handed back), with a moved nonce (resend), with an unchanged nonce and an expired permit (resend), with an unchanged nonce and a live permit (wait), and with a failed scan (wait); the margin on a 0.25 s, undeclared and 12 s chain; a rescan starting below its cursor for both placement and cancel; and the warning for a replaced record's live order.
  • pnpm --filter="@hyperbridge/simplex" test:data 146 passed; test:unit 263 passed; test:filler 70 passed, 5 skipped.
  • pnpm --filter="@hyperbridge/simplex" lint: no errors; Biome is clean on the changed files.
  • npx tsc --noEmit -p .: the same 14 errors as main (3 of them in the ox dependency), none in changed files.

Not covered:

  • No fork or mainnet run.
  • A bootstrap op that never surfaces keeps the top-up pending, which blocks a new one, until the next sponsored op moves the nonce. A bootstrap cancel with an unchanged nonce and no log likewise waits.

@Wizdave97 Wizdave97 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: changes needed before merge. I reviewed only this PR's commit on top of #1408.

These parts look right:

  • The typed SubmittedUnconfirmedError.
  • The EntryPoint-nonce rule: it uses the same key and compares the full nonce.
  • The deadline-bounded retry on FAILED.

But several of the "settled only on evidence" paths aren't actually proof, or can never trigger, and one of them can lock a reclaim for good.

Must fix

  1. An unconfirmed cancel hash is handed back forever (BridgeTopUp.ts:1332-1345).

    • A native cancelOrder that gets no receipt within 60 s now stores reclaimTxHash.
    • Suppose that tx is later dropped (its nonce reused) or mines and reverts. Every later attempt hands the hash back to the SDK, which either times out or throws GetRequest missing.
    • After 5 failures the top-up goes back to unfilled. reclaim() keeps reclaimTxHash, so no new cancel is ever sent, and the escrow stays locked until someone edits the state by hand.
    • Before this PR, only a mined, successful cancel was stored.
    • Suggestion: store the hash as unconfirmed and check its receipt before handing it back. If it succeeded, hand it back. If it reverted, or it isn't found and the wallet nonce has moved past it, clear it and send a new cancel.
  2. A record without placeSubmittedAt never ages out of placing (BridgeTopUp.ts:797, :1575).

    • submittedAt falls back to updatedAt when placeSubmittedAt is missing. But update() rewrites updatedAt on every write, including the cursor write at :728 just before settleUnplaceable.
    • So Date.now() - submittedAt is always about 0, and neither PLACEMENT_DROPPED_MS nor PLACEMENT_WINDOW_MS can ever pass.
    • The success path (:586) writes only placeTxHash. If recordPlacement then can't re-read the receipt (for example a bundle reorged out and never re-bundled), the record stays placing forever and blocks start(). Old failed/placing records have the same problem.
    • The success path also never records placeUserOpHash for a sponsored send. Once the timing is fixed, a bundle hash that vanished would be settled as a dropped native tx.
    • Every test sets placeSubmittedAt, so the fallback is never exercised.
    • Suggestion: write placeSubmittedAt and the UserOp details on the success path too. For old records, fall back to stageTimes.placing ?? createdAt.
  3. A recovery still running for an old record writes into the new one (BridgeTopUp.ts:377, :1569).

    • restore() re-runs recoverPlacement for every failed record that has a hash and a session key. That now includes every record this PR settles as failed.
    • failed doesn't count as pending, so start() lets the operator begin a new top-up while that re-check is still running (at least 120 s of receipt wait, plus a scan).
    • When the re-check finishes, its update({placeScannedTo}) and update({stage: "failed"}) land on the new record. A live new top-up can then show failed, and a third top-up can replace it while its order still holds escrow (c03 again).
    • Only the retry timers are tied to their record; these writes aren't.
    • Suggestion: tie update() to a record id, so it does nothing when this.record.id !== id, and/or have start() refuse while a recovery is in flight.
  4. "A native tx the node has forgotten" is not proof it can never be mined (BridgeTopUp.ts:808-817).

    • A pending tx can be missing from one load-balanced backend, or from one node's mempool, and still be valid elsewhere until its nonce is used.
    • Writing failed / "Nothing was escrowed" at that point means a later inclusion places an order that nothing tracks.
    • Suggestion: record the tx nonce when submitting, and settle only when getTransactionCount(scannedTo) > nonce and the scan found nothing.

Should fix

  1. Evidence is read at the chain tip (:868).

    • viem's fallback can send a getLogs page the primary endpoint refuses (for example a 2000-block range) to the next endpoint. If that endpoint is behind, it silently returns an empty page.
    • Nonces are read with no margin for reorgs.
    • The Permit2 expiry rule compares against the host clock, not getBlock(scannedTo).timestamp.
    • placeFromBlock has no margin below it.
    • Suggestion: read nonces a few blocks below scannedTo, compare the deadline with the block timestamp, start the scan a few blocks below placeFromBlock, and save the cursor after each chunk.
  2. recordPlacement doesn't filter OrderPlaced logs by gateway address (:609).

    • A bundle receipt carries other ops' logs, and our session address is visible in the mempool. An earlier op in the same bundle could therefore emit a forged OrderPlaced for our session, and we would track a fake order.
    • The scan path already filters by address. Add the same log.address === gateway check here.

Cross-PR:

  • A native sendTransaction that throws after the node accepted it (an RPC timeout, then "already known" on retry) is still a plain error. The placement then ends failed with no hash. #1413 only covers bundler sends.
  • #1411 adds the same "refuse a native batch on an undelegated wallet" rule for vault batches as TokenSender's guard here. One shared helper would keep the two consistent.

Docs: the changelog note says old records are "dated by updatedAt" and recovered "the same way", which (2) shows doesn't work. At 125 lines it also mostly restates the code and could be much shorter.

seunlanlege added a commit that referenced this pull request Oct 19, 2026
- Store a cancel sent without a receipt as reclaimSubmission (tx hash and
  nonce, or UserOp hash, nonce and key) and check it before handing it to the
  SDK: a success is handed back, a revert or a consumed nonce clears it and
  sends a new cancel, anything else waits. A timed-out UserOp cancel is no
  longer resent while it can land.
- Write placeSubmittedAt, and for a sponsored send the UserOp's hash, nonce,
  key and expiry, on the success path too; date a legacy record by
  stageTimes.placing then createdAt, never updatedAt.
- Bind every background flow's writes to its record id, so a recovery for a
  replaced record cannot write into the next top-up.
- Send native placements with an explicit pending nonce and carry it in the
  submission; settle a native placement only when the wallet's nonce is past
  it, and drop the "forgotten for 30 minutes" rule.
- Read nonces and the permit clock below the scanned tip (chain timestamp, not
  the host clock), start the scan a reorg margin below placeFromBlock, and
  save the scan cursor after each chunk.
- Filter OrderPlaced logs by the gateway address in recordPlacement.
- Shorten the changelog note to the final behaviour and evidence rules.
seunlanlege added a commit that referenced this pull request Oct 19, 2026
@seunlanlege
seunlanlege force-pushed the fix/simplex-top-up-placement-recovery branch from 21e1c9d to f44b236 Compare October 19, 2026 13:45
seunlanlege added a commit that referenced this pull request Oct 19, 2026
- Store a cancel sent without a receipt as reclaimSubmission (tx hash and
  nonce, or UserOp hash, nonce and key) and check it before handing it to the
  SDK: a success is handed back, a revert or a consumed nonce clears it and
  sends a new cancel, anything else waits. A timed-out UserOp cancel is no
  longer resent while it can land.
- Write placeSubmittedAt, and for a sponsored send the UserOp's hash, nonce,
  key and expiry, on the success path too; date a legacy record by
  stageTimes.placing then createdAt, never updatedAt.
- Bind every background flow's writes to its record id, so a recovery for a
  replaced record cannot write into the next top-up.
- Send native placements with an explicit pending nonce and carry it in the
  submission; settle a native placement only when the wallet's nonce is past
  it, and drop the "forgotten for 30 minutes" rule.
- Read nonces and the permit clock below the scanned tip (chain timestamp, not
  the host clock), start the scan a reorg margin below placeFromBlock, and
  save the scan cursor after each chunk.
- Filter OrderPlaced logs by the gateway address in recordPlacement.
- Shorten the changelog note to the final behaviour and evidence rules.
seunlanlege added a commit that referenced this pull request Oct 19, 2026
@seunlanlege
seunlanlege force-pushed the fix/simplex-top-up-placement-recovery branch from 724278b to 4560cc4 Compare October 19, 2026 15:08

@Wizdave97 Wizdave97 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for one new issue. Of the six round-1 findings, five are fixed and (5) is partially fixed.

Must fix

  1. A sponsored cancel waits forever when the bundler can't be asked (BridgeTopUp.ts:1543-1556).
    • Scenario: the bundler accepted the cancel op, then lost it or went down, or its URL was removed. userOpReceipt then throws on every attempt, before the nonce and Permit2 checks run.
    • After 6 attempts the top-up returns to unfilled, and every reclaim repeats the same loop.
    • The op never landed, so the escrow stays locked.
    • Suggested fix:
      • When the lookup fails, still apply the rule "nonce unchanged at the evidence block and Permit2 deadline passed". That proves the op can never land, and it needs no bundler.
      • The comment says nothing independent backs the cancel, but cancelOrder emits OrderCancelled(commitment, canceller) on every successful cancel (IntentGatewayV2.sol:491). Scanning for it with record.commitment, from a block stored with the submission, settles the cancel the same way the placement scan does.

Should fix

  1. A duplicate cross-chain cancel doesn't revert; it pays the relayer fee again.

    • _cancelFromSource (ExtrinsicIntents.sol:143) checks only the creator and the height. A second cancel therefore dispatches another GET, and dispatchWithFeeToken pulls the fee a second time.
    • The comments at :1547 and :1579-1580, and line 116 of the note, say a duplicate "can only revert".
    • settleLegacyCancel clears a sponsored cancel's bundle-tx hash when a reorg re-bundles the op, which triggers that second cancel.
    • It also compares the bundle tx's nonce, which is the bundler EOA's, with the filler wallet's nonce (:1600).
    • Suggested fix: keep reclaimSubmission until ORDER_CANCELED is recorded, and check tx.from === solver before using its nonce.
  2. The 64-block evidence margin is the same on every chain (:137). That's about 16 s on Arbitrum.

    • A fallback endpoint lagging by more than that serves an empty getLogs page, and the saved cursor then skips that range for good.
    • Suggested fix: size the margin in seconds, using the block time measured at startup.

Non-blocking:

  • Silent drop. A re-check of a replaced record that finds a live order drops it without logging (:616). Log the commitment at warn level.
  • Iteration history in docs. Note lines 112-117, and the comments at :1386-1388 and :1576-1580, describe "a reclaimTxHash that older code stored from an unconfirmed send". No released code did that; it was this PR's own earlier commit.
  • PR description. It still says a forgotten native tx is settled, that an unconfirmed cancel is stored as reclaimTxHash, and that a UserOp cancel is "still re-sent".

seunlanlege added a commit that referenced this pull request Oct 20, 2026
- Store a cancel sent without a receipt as reclaimSubmission (tx hash and
  nonce, or UserOp hash, nonce and key) and check it before handing it to the
  SDK: a success is handed back, a revert or a consumed nonce clears it and
  sends a new cancel, anything else waits. A timed-out UserOp cancel is no
  longer resent while it can land.
- Write placeSubmittedAt, and for a sponsored send the UserOp's hash, nonce,
  key and expiry, on the success path too; date a legacy record by
  stageTimes.placing then createdAt, never updatedAt.
- Bind every background flow's writes to its record id, so a recovery for a
  replaced record cannot write into the next top-up.
- Send native placements with an explicit pending nonce and carry it in the
  submission; settle a native placement only when the wallet's nonce is past
  it, and drop the "forgotten for 30 minutes" rule.
- Read nonces and the permit clock below the scanned tip (chain timestamp, not
  the host clock), start the scan a reorg margin below placeFromBlock, and
  save the scan cursor after each chunk.
- Filter OrderPlaced logs by the gateway address in recordPlacement.
- Shorten the changelog note to the final behaviour and evidence rules.
seunlanlege added a commit that referenced this pull request Oct 20, 2026
@seunlanlege
seunlanlege force-pushed the fix/simplex-top-up-placement-recovery branch from 4560cc4 to 26ad6c9 Compare October 20, 2026 11:54
seunlanlege added a commit that referenced this pull request Oct 20, 2026
…evidence and retry transient tracking errors
- Store a cancel sent without a receipt as reclaimSubmission (tx hash and
  nonce, or UserOp hash, nonce and key) and check it before handing it to the
  SDK: a success is handed back, a revert or a consumed nonce clears it and
  sends a new cancel, anything else waits. A timed-out UserOp cancel is no
  longer resent while it can land.
- Write placeSubmittedAt, and for a sponsored send the UserOp's hash, nonce,
  key and expiry, on the success path too; date a legacy record by
  stageTimes.placing then createdAt, never updatedAt.
- Bind every background flow's writes to its record id, so a recovery for a
  replaced record cannot write into the next top-up.
- Send native placements with an explicit pending nonce and carry it in the
  submission; settle a native placement only when the wallet's nonce is past
  it, and drop the "forgotten for 30 minutes" rule.
- Read nonces and the permit clock below the scanned tip (chain timestamp, not
  the host clock), start the scan a reorg margin below placeFromBlock, and
  save the scan cursor after each chunk.
- Filter OrderPlaced logs by the gateway address in recordPlacement.
- Shorten the changelog note to the final behaviour and evidence rules.
@seunlanlege
seunlanlege force-pushed the fix/simplex-top-up-placement-recovery branch from 26ad6c9 to cd10f6c Compare October 20, 2026 12:35
@seunlanlege
seunlanlege requested a review from Wizdave97 October 21, 2026 09:57

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants