Skip to content

feat(messaging): support message status reconciliation - #1249

Merged
mijinummi merged 3 commits into
MDTechLabs:mainfrom
barry01-hash:test/1164-message-status-reconciliation
Sep 28, 2026
Merged

mijinummi merged 3 commits into
MDTechLabs:mainfrom
barry01-hash:test/1164-message-status-reconciliation

Conversation

@barry01-hash

@barry01-hash barry01-hash commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Reconcile the relayer's tracked message status against the destination chain so a hung executor, a dropped completion callback, or a receipt that never finalised cannot strand a message in flight forever.

Closes #1164

What changed

  • MessageQueue lifecycle API for reconciliation (backward compatible, purely additive):
    • markSubmitted(messageId, txHash) — records a broadcast transaction; the message stays in flight as submitted until completed, failed, or reconciled.
    • fail(messageId, error?) — force-fails an in-flight message with no further delivery attempt (used for on-chain reverts).
    • getMessageStatus() / getInflightSnapshot() — read-only tracked state for monitoring and the reconciler.
    • complete() now advances the message to confirmed on success, and a late duplicate completion (e.g. reconciler racing the executor) is recorded once so the real outcome is never overwritten.
    • Staleness is measured from the current delivery attempt (inflightAt, set at dequeue), so normal retries are never flagged stale.
  • New MessageStatusReconciler — repairs drift in in-flight messages:
    • processing past staleAfterMs → re-queued for another attempt; failed once the retry budget is spent.
    • submitted + final receipt → confirmed; reverted receipt → failed immediately (deterministic); no receipt after staleAfterMs → re-queued for resubmission.
    • Fresh deliveries and chains without a provider are never touched. Provider faults leave messages in flight and count as errors.
    • Events: message-reconciled, stale-message-detected, reconcile-error, reconciler-degraded/reconciler-recovered, per-run reconciliation-summary; getStats() metrics.
  • EvmExecutorStatusProvider — reference provider that reuses EvmExecutor.getTransactionStatus, so reconciliation shares the live confirmation policy rather than defining a second notion of finality. Other chain adapters (Soroban/Solana) build to the same MessageStatusProvider shape.
  • Docs: new docs/MESSAGE_STATUS_RECONCILIATION.md (rules table, events, metrics, configuration, dependencies, troubleshooting) plus a cross-link from docs/CROSS_CHAIN_DUPLICATE_MESSAGE_DETECTION.md.

Verification evidence

  • 295 tests passing (11 suites) after merging current upstream main into the branch: the 270 upstream tests (including PR test(messaging): reject malformed cross-chain messages at intake #1241's malformed-message validation) untouched plus our 26 reconciler, 11 queue-lifecycle, and 4 pipeline tests.
  • message-status-reconciler.spec.ts covers normal/boundary/failure paths: fresh vs stale, retries exhausted, confirm/revert/not-found, provider throws/unknown/unavailable, degraded/recovered alerts, timers, run coalescing. 100% statement/function/line coverage (98% branches; one defensive formatting fallback).
  • reconciliation-pipeline.spec.ts integrates queue → reconciler → executor → mocked RPC: executor-backed confirmation, lost-submission recovery (requeue then confirm), revert → permanent fail, and healthy deliveries untouched.
  • tsc --noEmit clean; npm run build succeeds.
  • Works alongside the merged validation (PR test(messaging): reject malformed cross-chain messages at intake #1241): intake validation and status reconciliation compose cleanly. The reconciler/pipeline fixtures now use a distinct source chain because the merged validator rejects sourceChainId === destinationChainId.

Notes for reviewers

  • Pre-existing repo state (committed apps/relayer-service/node_modules is a broken install) required a local pnpm install --ignore-workspace to run tests; the tracked node_modules is committed untouched.
  • The pre-existing jest "worker process failed to exit gracefully" warning reproduces on the baseline suites alone and is unrelated to this change.
  • Labels ("Stellar Wave" etc.) require a maintainer; fork contributors cannot set them.

Reconcile the relayer's tracked message status against the destination
chain so a hung executor, a dropped completion callback, or a receipt
that never finalised cannot strand a message in flight forever.

- MessageQueue lifecycle API for reconciliation: markSubmitted() records
  a broadcast transaction hash, fail() force-fails an in-flight message,
  getMessageStatus()/getInflightSnapshot() expose tracked state, and
  complete() now advances the message to confirmed and records a late
  duplicate result only once. Staleness is measured from the current
  delivery attempt (inflightAt, set at dequeue) so normal retries are
  never flagged stale.
- New MessageStatusReconciler repairs drift: stale processing messages
  are re-queued (or failed once the retry budget is spent); submitted
  messages are confirmed when the receipt is final, failed on revert, and
  re-queued when no receipt appears after the stale threshold. Fresh
  deliveries and chains without a provider are left untouched. Emits
  message-reconciled, stale-message-detected, reconcile-error,
  reconciler-degraded/recovered and a per-run reconciliation-summary, and
  exports getStats().
- Ships EvmExecutorStatusProvider, which reuses EvmExecutor's own
  getTransactionStatus so reconciliation shares the live confirmation
  policy instead of defining a second notion of finality.
- 42 new tests (121 total): queue lifecycle, reconciler unit coverage at
  100% statements/functions/lines, and an integration pipeline across
  queue -> reconciler -> executor -> mocked RPC.
- Docs: MESSAGE_STATUS_RECONCILIATION.md plus a cross-link from the
  duplicate-detection doc; CI now runs relayer-service tests.

Closes MDTechLabs#1164
…tatus-reconciliation

# Conflicts:
#	docs/CROSS_CHAIN_DUPLICATE_MESSAGE_DETECTION.md
…ion merge

Merging upstream main brought in malformed-message validation (PR MDTechLabs#1241),
which rejects messages whose sourceChainId equals destinationChainId. Our
reconciler and pipeline fixtures used 'ethereum' for both. Point the
fixtures at an 'arbitrum' source so they stay valid under the new intake
validation while still exercising the EVM destination path.

Full suite: 295 tests passing (11 suites).
@drips-wave

drips-wave Bot commented Sep 28, 2026

Copy link
Copy Markdown

@barry01-hash Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@mijinummi
mijinummi merged commit 9872887 into MDTechLabs:main Sep 28, 2026
1 check failed
@grantfox-oss grantfox-oss Bot mentioned this pull request Sep 28, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support message status reconciliation

2 participants