Repository navigation
feat(queue): priority lists and * wildcard for queue:work - #15
Merged
Merged
Conversation
`ark queue:work` could only work one queue, and a bare worker silently ignores jobs dispatched to any other queue. `--queue` now accepts: - a comma-separated priority list (`--queue=webhook,ocr,default`): each loop takes the first job found walking the list; - `*` for every queue on the connection, combinable with names to prioritize some (`--queue=webhook,*`). A bare `queue:work` still works only the default queue. `QueueContract.popAny()` backs the wildcard. The database driver takes the oldest due job across all queues (FIFO, so no queue starves); the redis driver discovers queues with SCAN (never KEYS) and works them round-robin. The base implementation falls back to the default queue, so drivers without it behave as before.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ark queue:workcan only work one queue, and a bare worker silently ignores jobs dispatched to any other queue. This caught Arkyc in production: its single PM2 worker drained onlydefaultwhile every job went to a named queue, so sessions stalled inprocessingand no webhook was ever delivered.--queuenow accepts:A bare
queue:workis deliberately unchanged. Making it work every queue would break split deployments where a worker must not take jobs it can't run.Changes
runNextJob()anddaemon({ queue })take a string or a priority-ordered list;ANY_QUEUE('*') is exported.QueueWorkCommand: parses--queueinto a deduplicated list; the startup line says what it drains.QueueContract.popAny()backs the wildcard. The base implementation works the default queue, so custom drivers behave as before until they override it.SCAN(never the blockingKEYS), including queues holding only delayed jobs, and works them round-robin.RedisClientgainsscan, which ioredis already has.--queueforms and the "every queue needs a worker" rule. The jobs guide warns that a bare worker ignoresonQueue(...)jobs. The CHANGELOG has an Unreleased entry.style:commit holds unrelatedpnpm formatoutput (6 files, formatting only).Tests
12 new tests in
packages/queue/tests/multi-queue.test.ts:*draining all, andwebhook,*ordering.popAny: run against a fake table that applies the real filters. Checks there's no queue filter and the reserved job keeps its own queue.popAny: run against a fake client. Checks discovery across twoSCANpages, round-robin order, and a queue holding only delayed jobs being found once due.tests/MemoryQueue.tsnow keeps jobs per queue (unnamed pushes and pops still usedefault), so the existing worker tests pass unchanged. All 24 queue tests pass.