Skip to content

core: background job queue for chain submission and long-running work - #71

Merged
joelpeace48-cell merged 1 commit into
FinesseStudioLab:mainfrom
Codeinx:issue-54-job-queue
Aug 25, 2026
Merged

joelpeace48-cell merged 1 commit into
FinesseStudioLab:mainfrom
Codeinx:issue-54-job-queue

Conversation

@Codeinx

@Codeinx Codeinx commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Problem

Submitting a Soroban transaction and waiting for confirmation takes several seconds. Doing that inside a request handler ties up a connection, times out under load, and loses the work entirely if the process restarts mid-flight.

What this adds

New module: src/queue/

  • A durable job queue (JobQueue + JobStore) backed by a JSON snapshot file, written with write-to-temp-then-rename on every mutation so a crash mid-write never leaves a half-written file behind. Chosen over BullMQ/Redis to keep this repo's dependency footprint and CI/test setup unchanged — the acceptance criteria call for jobs to survive a restart, not for a specific backend.
  • Designed for jobs like transaction submission/confirmation polling, export generation, indexer catch-up, and webhook fan-out (handler registration is by job type, so callers plug in their own).
  • Idempotent by construction: a job id that the store has already observed complete is never re-dispatched, including across a simulated crash-and-restart against the same durable file — a job may run twice after a crash, and for a transaction submission that's a real financial event, not a free retry.
  • Retry with bounded, jittered exponential backoff, moving to a queryable dead-letter status once maxAttempts is exhausted (deadLetterQueue()).
  • metrics() exposes queue depth, oldest-pending age, dead-letter count, and failure rate.

Testing

src/queue/queue.test.ts covers:

  • a handler invoked twice via run() only executes once
  • the double-run-after-crash case explicitly: two separate JobQueue instances against the same durable file, verifying the handler does not re-run on the second instance
  • retry-until-dead-letter, with the last error preserved
  • metrics reflecting depth, dead-letter count, and failure rate

npm run lint, npm run build, npm test, and npm audit --audit-level=high are all clean on this branch.

Scope note

This PR only adds src/queue/** (new files) — it does not touch package.json, src/routes/v1/index.ts, or any other file shared with the other two PRs open against this issue tracker (#44, #58), so it should apply cleanly regardless of merge order.

Closes #54.

…ing work

Adds a small file-backed durable job queue (src/queue) for work that
takes too long to run inline in a request handler: transaction
submission/confirmation polling, export generation, indexer catch-up,
and webhook fan-out.

- Durable: every state transition is written to disk (write-temp then
  rename) so pending/active jobs survive a process restart.
- Idempotent handlers: a job id that has already been observed to
  complete is never re-dispatched, including across a simulated
  crash-and-restart against the same durable file — covered by a test
  that runs the same job through two separate JobQueue instances.
- Retry with exponential backoff (jittered, bounded) up to a
  per-job maxAttempts, after which the job moves to a dead-letter
  status that is queryable via deadLetterQueue().
- metrics() exposes queue depth, oldest-pending age, dead-letter count,
  and failure rate.

Closes FinesseStudioLab#54.
@joelpeace48-cell

Copy link
Copy Markdown
Contributor

@Codeinx Nice job

@joelpeace48-cell
joelpeace48-cell merged commit dcc669a into FinesseStudioLab:main Aug 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

core: background job queue for chain submission and long-running work

2 participants