Skip to content

fix(dev-fleet): re-resolve the main checkout without a gateway restart - #11972

Closed
chenmingwei23 wants to merge 1 commit into
kirodotdev:mainfrom
chenmingwei23:fix/dev-fleet-rere-solve-main-checkout-3470
Closed

chenmingwei23 wants to merge 1 commit into
kirodotdev:mainfrom
chenmingwei23:fix/dev-fleet-rere-solve-main-checkout-3470

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

A user who answers Dev Fleet's setup card keeps seeing the setup card. ensure_main_repo_discovered() set _DISCOVERY_DONE = True at the end of every run, including a run that found nothing, so a process that started before the operator pointed Dev Fleet at a checkout answered "no checkout found" for the rest of its life.

repo_path was the odd knob out. auto_prune is re-read each reaper cycle and ticket_url_template is read per resolution, so both take effect while the gateway runs.

Why it matters

This lands in the lowest-context moment the product has: a first run, on the one screen whose only job is to get the user to a working fleet. The old copy did name the restart, so it was not a trap, but "do this, then restart the gateway" is a worse loop than "do this" when nothing about the state actually requires a restart.

What changed

Symptom: the setup card never goes away. Root cause: the latch recorded a negative result as final. Change: _DISCOVERY_DONE = bool(discovered), so only a resolved checkout latches, and /fleet calls the new worktree_ops._ensure_repo_resolved() per poll. That returns at a truthiness guard before any await, so a resolved install pays nothing and only the state with no fleet re-runs discovery.

Only one of the two documented remedies can self-heal, and the copy now says which. This corrects a premise in the issue, which lists setting KIROCREW_DEVFLEET_REPO and adding dev_fleet.repo_path as equivalent. _load_dev_fleet_cfg re-reads config.json on every call, so a running process sees a newly written repo_path. KIROCREW_DEVFLEET_REPO is read with os.environ.get off this process's own environment, which no outside shell can change, so that route needs a restart and always will. no_checkout_found_action previously ended "then restart the gateway" for both; it now leads with the config route and says plainly that the variable needs a restart.

The refresher restart is not bookkeeping. _status_refresher returns when nothing is resolved rather than idling, on the previously correct reasoning that a startup-resolved global never becomes resolved later. A late resolution that left it stopped would serve a fleet whose rows never refresh again: the setup card disappears, the page looks alive, and nothing fetches. That is worse than the honest "restart the gateway" this replaces, so _ensure_repo_resolved starts it on the attempt that resolves. It lives in worktree_ops because the task handle is owned at that level and repository sits below it in the component DAG.

No verdict outlives the attempt that produced it. _REPO_INVALID_MSG is now assigned on both branches, so an attempt that finds nothing clears it instead of inheriting it. This is the hazard the triage proposal identified and the issue's own inventory missed: MAIN_REPO from a later attempt sitting beside an earlier attempt's validation verdict would hand _repo() a path whose markers were never checked, and worktree remove, update-ref -d, pull --ff-only and pip install -e run inside whatever that is.

The retry spawns no subprocess. _load_fallback_repos and _upstream_remote both return before reaching git while _repo() raises, and _resolve_primary_checkout runs only on an attempt that found a path. The credential-helper warm keeps its own None sentinel because it reads --system and --global scope only and is repo-independent, so its two git config calls stay once-per-process rather than once-per-poll. What an unresolved poll costs is tiers 2 and 5 on the subprocess executor, every 12 seconds, only while the page is open.

A resolved path that fails the marker test still latches. Tiers 1 and 2 are taken verbatim, so re-running returns the same path every time, and that state renders the RepoUnreadable banner naming the path and the remedy.

Two of the four blockers the issue lists needed no work, which the triage proposal established and I confirmed against the code. The AST ratchet checks MAIN_REPO loads, not writers, so a second writer needs no exemption; both new guards are bare truthiness tests, which it permits. _UPSTREAM_REMOTE and _FALLBACK_REPOS decline to cache when _repo() raises, so neither was ever poisoned. Main also already carried ensure_main_repo_discovered() as a single-flight seam, so no new lock or entry point was needed.

The decision to make this lifecycle change is recorded in #3470, raised by Design Review on #3406 and routed by triage. No user-facing default of a command, loop or agent changes: an unconfigured install that previously required a restart now resolves on its own, and a configured install behaves identically.

Backwards compatibility

Compatible. Nothing is rejected, renamed or removed. _DISCOVERY_DONE goes from "always true after the first run" to "true once a checkout resolved", which only ever adds attempts in the state that had no fleet; a resolved install runs discovery exactly once, as before. RepoNotConfigured stops being terminal for the process lifetime, and every existing catch site already treats it as a per-call condition and recomputes: _upstream_remote degrades to origin without caching, _load_fallback_repos leaves the list None which consumers read as or [], and _auto_prune_reaper re-asks _repo() every cycle. GET /api/crons and every other payload shape is unchanged; the only wire-visible difference is that /fleet can stop reporting needs_setup without a restart.

Tests

test/test_dev_fleet_repo_reresolution.py, 9 cases: an attempt that finds nothing does not latch, the first attempt that resolves does, a resolved-but-invalid path latches, a stale invalid-path message is cleared, the helper warm is paid once across three attempts, a resolved install runs no discovery, a late resolution starts the refresher, a still-unresolved attempt does not, and a disabled-background-tasks mode resolves without starting a task.

Mutation-verified: 7 mutations applied to real source, 7 caught, both sources restored byte-identical. Each breaks one property, including restoring the original unconditional latch and letting the invalid-path verdict survive.

Also green locally: 84 backend cases across the accessor ratchet, component boundaries, gateway routes and the new file; 96 DevFleetPage cases; all 19 i18n checks including changed-passthrough and changed-values; all 11 locale style suites, 95 cases. The gates check_sync_io_in_async, check_loop_bound_locks, check_feature_map, check_black_formatting, check_subprocess_encoding, check_focus_cue, check_comment_history and docs_lint pass, plus black and flake8 on the four changed Python files.

The zh-CN style suite caught a real defect in my first draft: it stacked four possessive particles in one value where its rule permits two.

Catalog edits are textual, 1 line changed and 0 added per file, so the reserialisation that duplicateKeys.test.ts documents dropping eleven Japanese translations is avoided.

Manual verification

N/A for the mechanism, and stated rather than skipped. The behaviour is a module-global lifecycle over three call sites, and the 9 cases plus 7 mutations cover each transition directly, including the two that need an event loop (refresher start, and no start while unresolved). Reproducing it by hand needs a gateway that genuinely cannot find a checkout, which on this host means pointing a pod at a path with no checkout anywhere in its discovery tiers, and the observable result is the same boolean the tests assert.

The copy change is verified by the gates that own it rather than by eye: changed-passthrough at zero for all 11 non-English catalogs, the 11 per-locale register suites, and the existing DevFleetPage assertion that the card still renders its env example.

Related Issues

Fixes #3470

Pattern harvest

Rule candidate: review-prompt

Pattern: a single-flight latch records "done" after an attempt that produced no answer, freezing a state the operator can still change from outside the process. The generalization is worth a prompt line because the latch reads as correct at the call site (it is idempotent, it is single-flight, it does cache) and only the negative case is wrong. The same shape shipped in config-default materialization, where a stored value always beat a changed dataclass default. Worth asking of any _DONE / _LOADED / is None guard: does it cache an ANSWER, or the absence of one?

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@chenmingwei23
chenmingwei23 requested review from a team and patrigao September 19, 2026 07:13
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 19, 2026
ensure_main_repo_discovered() latched unconditionally, so a gateway that
started before the operator pointed Dev Fleet at a checkout kept answering
"no checkout found" until it was restarted. It now records "done" only once
a checkout RESOLVED, and /fleet re-runs it per poll while nothing is.

Only dev_fleet.repo_path can self-heal: _load_dev_fleet_cfg re-reads
config.json on every call, whereas KIROCREW_DEVFLEET_REPO is read off this
process's own environment, which no outside shell can change. The setup
card's copy now names the two routes separately instead of telling every
user to restart.

A resolved install returns at a truthiness guard before any await, so the
retry costs nothing once there is a fleet. An unresolved one spawns no
subprocess, and the credential-helper warm keeps its own once-per-process
sentinel. A late resolution restarts the status refresher, which returns
rather than idles when nothing is resolved.

Fixes kirodotdev#3470
@chenmingwei23
chenmingwei23 force-pushed the fix/dev-fleet-rere-solve-main-checkout-3470 branch from 3a6f3a6 to 89cb4e2 Compare September 19, 2026 08:00
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Opus 4.8 Review (fork) — ⚠️ review incomplete

Reviewed 89cb4e23671496853ad46a252a639478ca56f687 via the fork AI-review pipeline; updated in place on each push.

No completed Opus verdict for this commit; see the Fork Opus 4.8 Review job logs.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — ⚠️ could not complete

The first-principles review did not produce a verdict for 89cb4e23671496853ad46a252a639478ca56f687 (the review step failed). See the Fork First Principles Review job logs. Advisory — does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — ⚠️ could not complete

The design review did not produce a verdict for 89cb4e23671496853ad46a252a639478ca56f687 (the review step failed). See the Fork Design Review job logs. Advisory — does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ⚠️ review incomplete

Reviewed 89cb4e23671496853ad46a252a639478ca56f687 via the fork AI-review pipeline; updated in place on each push.

No completed GPT verdict for this commit; see the Fork GPT 5.6 Review job logs.

Captured review output -- no verdict stamp for this commit, so this is NOT a verdict

Incomplete review: pass(es) 1 2 did not complete.

The pass-2 verdict was not published because both calls are required. Re-run the workflow for a complete review.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 19, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 19, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Superseded by #12463, which carries this exact commit (89cb4e2) on a same-repository branch.

This PR cannot reach a verdict. fork-opus-review.yml runs harden-runner with egress-policy: block and an allowed-endpoints list that omits the Claude Code installer host, so every Fork Opus run since 2026-09-20 07:43Z fails at Failed to connect to claude.ai port 443 after 1 ms and both required review lanes publish a "review incomplete" notice instead of a finding. Measured over 46 polls across roughly 22 hours: every non-skipped Fork Opus run concluded failure, and the four review comment bodies never changed length.

That outage is tracked as #11911. #11976 allowed the bun release-asset host, which did clear the earlier bun: command not found and ECONNREFUSED failures, but it did not add the installer host; #12081 and #12201 propose the remaining entry and are both still open.

An override cannot resolve this. An override disposes of a finding, and there is no verdict here to dispose of. claude-review.yml, the same-repository lane, declares no egress policy at all, so the review lanes on #12463 are already running rather than failing closed.

Closing in favour of #12463. Nothing about the change itself is in question: this head measured 77 lanes success, 3 advisory neutral, mergeable true, with the two review lanes as the only outstanding items. The fork branch is left intact.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dev Fleet: re-resolve the main checkout without a gateway restart

1 participant