Skip to content

feat(messaging): persist message nonces across restarts (Stellar Wave) - #1251

Merged
mijinummi merged 1 commit into
MDTechLabs:mainfrom
barry01-hash:test/1161-message-nonce-persistence
Sep 28, 2026
Merged

mijinummi merged 1 commit into
MDTechLabs:mainfrom
barry01-hash:test/1161-message-nonce-persistence

Conversation

@barry01-hash

Copy link
Copy Markdown
Contributor

Closes #1161

What

Every chain now keeps a persisted high-water mark (next nonce) and every message a persisted pin of the nonce it was first delivered with, so a restart, replica failover, or queue redelivery can never re-issue a nonce — and a re-executed message reuses its original nonce instead of double-broadcasting on the destination chain.

Historically the nonce lived only in process memory; restarting the relayer mid-delivery could mint the same nonce twice.

How it works

  • New NonceStore contract (types.ts) + InMemoryNonceStore (default, this is today's behavior) and FileNonceStore (src/nonce/).
    • FileNonceStore writes one JSON file atomically (temp sibling + fsync + rename), 0600, corrupt/unsupported files fail fast instead of silently resetting the counter to zero.
  • New NonceManager (src/nonce/nonce-manager.ts) hands out per-chain nonces with serialised acquire/release and the key crash-safety order: a nonce is persisted before it may be broadcast. Persist failure rolls the in-memory counter back and fails the delivery rather than burning an unrecorded nonce.
  • Both EvmExecutor and SorobanExecutor wire the manager, reserve once per delivery, reuse the reserved nonce on repricing retries, and release the pin after on-chain confirmation.
  • executor.updateNonce()/getNonce() now delegate to the manager; raiseOnly semantics mean a stale operator input can never lower the counter.
  • New events: nonce-loaded, nonce-assigned, nonce-reused, nonce-error {phase: load|persist|release}, nonce-declined, plus Soroban nonce-bookkeeping.

Files

  • apps/relayer-service/src/nonce/{in-memory-nonce-store,file-nonce-store,nonce-manager}.ts (new)
  • apps/relayer-service/src/executors/{evm,soroban}-executor.ts (wired)
  • apps/relayer-service/src/types.ts, src/index.ts
  • Tests: src/nonce/__tests__/* incl. a cross-component pipeline spec (executor + file store + mocked EVM RPC)
  • Docs: docs/MESSAGE_NONCE_PERSISTENCE.md

Verification

  • tsc --noEmit: clean
  • jest --ci: 342 tests / 15 suites, all passing (43 new)
  • Coverage (feature module): src/nonce 96.66% stmts / 91.37% branch (file-nonce-store.ts 98.57% stmts, in-memory-nonce-store.ts 100%, nonce-manager.ts 93.1%)
  • Executors: evm-executor.ts 96.38% stmts / 100% funcs (Soroban's low coverage is pre-existing carried-over untested RPC paths, not from this change)
  • npm run build: clean

Safety rationale

  • Reserve-before-broadcast: crash after reserve → next start reads persisted value and skips past it.
  • Per-message pin: redelivery reuses the exact nonce, so a retried message cannot mint a second one.
  • FileNonceStore never silently resets to zero on a corrupt file (the one situation that could re-issue consumed nonces) — it fails fast and asks the operator to restore from backup.

Note: apps/relayer-service/node_modules is tracked in the repo (pre-existing); it is intentionally not touched in this PR. The CI workflow runs the relayer-service suite.

Assignees: @barry01-hash

Add a durable NonceStore (InMemory / FileNonceStore) and a per-chain
NonceManager. Executors reserve a nonce and persist the chain high-water
mark and per-message pin before broadcasting, so a crash, failover, or
queue redelivery never re-issues a nonce and a re-executed message
reuses its original nonce instead of double-broadcasting. Repricing
retries resubmit with the reserved nonce. Corrupt state files fail fast
rather than resetting to zero.
@drips-wave

drips-wave Bot commented Sep 28, 2026

Copy link
Copy Markdown

@barry01-hash Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@mijinummi
mijinummi merged commit 1d67d06 into MDTechLabs:main Sep 28, 2026
1 check failed
@grantfox-oss grantfox-oss Bot mentioned this pull request Sep 28, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add message nonce persistence

2 participants