[DEV-104] Resume in-progress tickets left behind by an interrupted worker before picking up new work - #95
Merged
Merged
Conversation
…n interrupted worker before picking up new work
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task: DEV-104
Summary: Resume in-progress tickets left behind by an interrupted worker before picking up new work
Implementation Details
I'll start by exploring the codebase to understand the existing architecture and the files mentioned in the task.
Now let me read the key files mentioned in the task:
Now let me look at the worker entry points:
Now let me look at the relevant sections in index.ts:
Let me examine the supporting modules:
Let me check the docs and existing tests:
Let me look at how the
workercommand is dispatched in index.ts, the tracker client interface, and the failure-feedback test patterns:Let me examine the repo-manager and state modules to understand worktree layout:
Let me check the settings types and env-integer helper:
I now have a full picture. Let me set up a todo list and verify one dependency before implementing:
Now let me create the orphan-recovery module:
Now wire it into the workspace worker:
Now add the recovery call in
runWorkspaceWorkerbefore acquirers are built and started:Let me clean up the now-redundant dynamic import in
buildFleetEventAcquirers:Let me verify the modified sections:
Dependencies appear missing in this worktree. Let me install:
Typecheck passes. Now the tests:
The test helper's default re-applied when passing
undefinedexplicitly. Fix with anullsentinel:All pass. Let me check the RepoConfig type so I can add a wiring test for
recoverOrphanedWorkspaceRuns:The Jira client throws without creds — that gives a deterministic offline degradation test. Let me add a wiring test:
Both pass. Now update the docs. First
worker.md— add a startup-recovery subsection after "Re-running a task":Now let me check the relevant part of
automated-task-processing.md:Now run the full package test suite to check for regressions:
All 1010 tests pass. Now lint and format:
All green. Let me do a final review of the workspace-worker changes in context:
DEV-104 is implemented. Summary of the changes:
New:
packages/code/src/lib/orphan-recovery.tsrecoverOrphanedTaskRuns()— runs at worker startup before acquirers pick new work. It keeps the existingreapOrphanedRuns()bulk-fail (all origins), then for each orphanedtask-origin run with a ticket key it approximates the graceful-shutdown UX: posts the same processing-failure comment via the existingreportTaskFailure()path and moves the ticket back to its To Do status (src/lib/orphan-recovery.ts:117).reportTaskFailurerecords the incomplete attempt for the retry gate, so a requeued-but-unchanged ticket is skipped on the next pickup exactly like any reported failure;--force/edit/comment unlock semantics are unchanged.WORKER_ORPHAN_MAX_AGE_HOURS, default 7 days) are reaped silently; markdown/scheduled runs recover via automation leases and get no tracker feedback; missing tracker creds degrade to reap-only with a warning.loadProjectSettingsFrom()(merges repo base-worktree settings) andresolveStatusName()for status-name resolution in fleet mode.Wiring:
workspace-worker.ts—recoverOrphanedWorkspaceRuns()is called inrunWorkspaceWorkerright after state opens and before any acquirer starts (workspace-worker.ts:450); the old reap-only block insidebuildFleetEventAcquirers(which only ran with GitHub creds) is removed.Tests — 20 new tests: a crash-mid-run simulation (in-progress run in a "dead" store → reopen → recovery asserts failed run, comment, To Do transition, retry-gate state), clobber/staleness/markdown/no-creds/fetch-error guards, and workspace wiring degradation. Full suite: 1010 pass, 0 fail; typecheck and lint (0 errors) clean.
Docs — new "Interrupted runs are recovered on startup" section in
docs/code/worker.md, a mention inautomated-task-processing.md, and a bullet under "How events are handled".This PR was automatically created by @devintern/code