Repository navigation
fix(dev-fleet): re-resolve the main checkout without a gateway restart - #12463
Conversation
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify the declared trade: an unparseable What this change shipsInventory (9 items) — 8 justifiedIntent: FIX — a user who answers Dev Fleet's setup card keeps seeing it, because a failed discovery latched as final; discovery now retries until a checkout resolves.
[FIRST-PRINCIPLES-REVIEWED] 9e75f24 |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the declared trade: a permanently unparseable config now yields a setup card blaming a missing checkout, not the unreadable file (#12513). [DESIGN-REVIEWED] 9e75f24 |
UX Review (Fable 5) — ✅ PASSUX-level review of UX-Verdict: PASS The card's only copy-able snippet ( Suggestions
[UX-REVIEWED] 9e75f24 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Review round 1 dispositionHead Design CONCERNS -- fixedThe finding is correct and the defect is mine. Taken the first of the two remedies the verdict names, the string compare with no subprocess.
Five tests cover this, each mutation-verified: correcting the path re-runs discovery; an UX suggestion -- kept out of this change, with the reasonThe mechanism is real: the card's only copyable snippet is The consequence the suggestion names is already covered by the adjacent sentence. Closing that gap costs a new i18n key in 12 locale catalogs, plus the generated CIOne failure on the previous head, |
4ecb2f6 to
83317ed
Compare
Disposition for head
|
9846e35 to
7b41f1b
Compare
Advisory dispositions for
|
Dev Fleet resolved its main checkout once per process and latched the answer unconditionally, so a gateway that started before the operator pointed it at a checkout kept answering 'no checkout found' until it was restarted. A found-but-invalid path latched the same way, so an operator who named a readable directory that is not a Kiro Crew checkout met a RepoUnreadable banner that survived every correction they made, while the banner's own remedy was the edit it ignored. ensure_main_repo_discovered records 'done' only once a checkout RESOLVED, and /fleet re-runs it per poll while nothing is. _invalid_resolution_is_stale reopens an invalid latch once the operator's configured string changes, on that string alone: no git and no stats. It compares against the string the latching attempt read rather than against MAIN_REPO, which is the _resolve_primary_checkout form of it, so a configured path pointing into a linked worktree does not read as changed on every poll. A resolved and valid install returns at its first guard with no await. Only dev_fleet.repo_path can self-heal. _load_dev_fleet_cfg re-reads config.json on every call, whereas KIROCREW_DEVFLEET_REPO is read off this process's own environment, which no outside shell can change, so that route still needs a restart and the setup card names the two routes separately rather than telling every user to restart. All 13 locales carry that caveat. Reopening also requires the config read itself to have succeeded. _load_dev_fleet_cfg answers an empty section for a file it cannot parse, so the configured path resolves to the same empty string a config naming no path gives. Treating that difference as a change would reopen the latch on evidence nobody read: discovery would find no configured path, fall through to the INFERRED tiers, and latch a checkout the operator never named while their own setting sat in a file the process merely failed to read, and every later git call, including the destructive ones, would target it. _load_dev_fleet_cfg_checked reports whether every file present parsed and _configured_main_repo_checked carries that flag beside the path, so a partial read holds the latch while a parseable file carrying no repo_path still reopens. _load_dev_fleet_cfg keeps its plain signature for the callers that read a single setting and have no use for the flag. _ensure_repo_resolved asks through _repo() rather than testing MAIN_REPO for truthiness, because _status_refresher returns on its first line for an unusable checkout, so starting it would mint a task per poll that dies immediately. An unresolved install spawns no subprocess, and the credential-helper warm keeps its own once-per-process sentinel. A late resolution restarts the status refresher, which returns rather than idles when nothing is resolved. Tests live in test/test_dev_fleet_repo_reresolution.py and are mutation-verified in both directions, including against the narrower fix that would freeze a genuinely cleared path. Fixes #3470
05fdbde to
06d621c
Compare
|
Rebased: main moved in docs/system-specs/modules/aws-control.md, src/kiro_crew/acp/client.py, src/kiro_crew/apps/builtins/aws_control/backend/backup.py, src/kiro_crew/apps/builtins/aws_control/backend/routes.py, src/kiro_crew/dashboard/handlers/agents.py and test/test_aws_control_routes.py since this head went green (3017e6a -> 1fa7f5b); CI re-running. Scoped tests for those files, re-run on the new head 06d621c: test_acp_client.py, test_aws_control_routes.py and test_aws_control_backup.py, 925 passed. This change's own affected files plus the agents-handler tests: 816 passed, 1 skipped. All 14 local gates green against the new base. The diff is byte-identical to the pre-rebase head, same 18 files and same counts, so the rebase changed history only. |
The staleness test and discovery each needed the configured path, and each read config.json for itself. Two reads of one file can disagree: the first could see a whole, corrected path and reopen the latch while the second returned the empty string from a torn read, sending discovery to the INFERRED tiers. That latch is the dangerous one. An inferred checkout passes the marker test, so the resolution is VALID and therefore final: nothing re-resolves it, the staleness test only reopens a latched-INVALID resolution, and Pull + Build meanwhile mutates a checkout the operator never named. Only a gateway restart clears it. The attempt takes one checked read and hands it to _discover_main_repo, which accepts it as an optional argument and reads tier 2 itself only for a caller holding no snapshot, so dev_fleet_startup keeps its existing call. A partial read publishes nothing at all and the next poll retries against a settled file. Publishing nothing means publishing the UNRESOLVED state rather than leaving the previous one standing. _repo() gates on MAIN_REPO alone and never consults _DISCOVERY_DONE, so an attempt that returned with the provisional import-time hint in place would let every consumer operate on a checkout it could not confirm. That hint is what _default_main_repo_state picks before any config is read, and what dev_fleet_startup exists to replace and normalize, so an attempt that cannot read tier 2 has no basis for endorsing it. Cleared, _repo() raises RepoNotConfigured and the page shows the setup card. Clearing the published pair is not enough by itself. The reopen path arrives holding _DISCOVERY_DONE True, and the pre-lock gate returns early once MAIN_REPO and the message are both empty, so a torn read there cleared the pair and left the flag standing: the retry this branch promises was unreachable and the page kept the setup card until a gateway restart, the same failure this change exists to end. The branch clears the flag as well. The German and French cards read in one register. The new action string uses the informal address both style guides mandate, while the adjacent help and env-example strings carried the formal one, so a single card showed both. Those two neighbors move to the informal address in each locale; Spanish already reads that way throughout. Seven mutations, each reddening its own assertion: dropping the partial-read return, not handing the snapshot to discovery, having discovery ignore the snapshot and re-read, dropping each of the three parts of the clearing, and dropping the _DISCOVERY_DONE clear, which strands the next poll on the reopen path.
06d621c to
9e75f24
Compare
|
|
|
buluoray
left a comment
There was a problem hiding this comment.
Verified at 9e75f24a. The fix is real, not a narrowed window — the latch no longer records a negative as final.
_DISCOVERY_DONE has exactly three assignments in repository.py: the module default at :318, the torn-read clear at :443, and _DISCOVERY_DONE = bool(discovered) at :496. That last one is the whole fix: a not-found attempt latches False, so the pre-lock gate at :402 proceeds on the next poll instead of freezing the negative for the process lifetime.
Per-call re-resolution is genuine rather than import-frozen — the class of bug that has shipped here before. The config tier is re-read every attempt through _load_dev_fleet_cfg_checked -> config_dir(), and the import-time MAIN_REPO = _default_main_repo_state() (:313) is stat/env-only and gets overwritten under the lock on each attempt.
The stale GPT BLOCK (posted against head 1805abcd) asked for _DISCOVERY_DONE to be cleared in the partial-read branch; that is present at :442-:443, where the branch also clears MAIN_REPO and _REPO_INVALID_MSG and returns. So the reopen-plus-torn-read path is a retry, not a terminal state.
Caches invalidate on the right key. _LATCHED_CONFIGURED stores the configured string the latching attempt read (:485) and _invalid_resolution_is_stale compares against that, not against the _resolve_primary_checkout-rewritten MAIN_REPO. No cached-None dead branch is introduced: _upstream_remote returns "origin" without assigning _UPSTREAM_REMOTE, and _load_fallback_repos returns leaving _FALLBACK_REPOS as None, so an unresolved poll poisons neither.
MAIN_REPO and its validation verdict are published with no interposed await (:473 then :479), so _repo() — which checks both — can never hand out a path whose markers were not checked. That is what keeps a destructive git op off an unvalidated path.
The sync guard is unaffected: its python -c probe lives in worktree_ops.py and already passes -I to drop cwd from sys.path, and this diff adds no cwd-sensitive probe — discovery is config read, stats, and git -C <repo>.
Concurrency is double-checked locking on _DISCOVERY_LOCK with every safety-critical global written under it; a pre-lock read race self-corrects on the next poll and cannot trigger a git op because _repo() re-validates at each call site.
One behaviour worth knowing rather than fixing: a VALID resolved install returns at the pre-lock guard and will not re-target if the operator later edits dev_fleet.repo_path to a different valid checkout without a restart. That matches the PR's stated scope and is arguably the safer default (no silent mid-run target swap), but the setup-card copy only distinguishes the env-var route, so the restart requirement is not stated anywhere the operator would see it.
kirodotdev#12463) * fix(dev-fleet): re-resolve the main checkout without a gateway restart Dev Fleet resolved its main checkout once per process and latched the answer unconditionally, so a gateway that started before the operator pointed it at a checkout kept answering 'no checkout found' until it was restarted. A found-but-invalid path latched the same way, so an operator who named a readable directory that is not a Kiro Crew checkout met a RepoUnreadable banner that survived every correction they made, while the banner's own remedy was the edit it ignored. ensure_main_repo_discovered records 'done' only once a checkout RESOLVED, and /fleet re-runs it per poll while nothing is. _invalid_resolution_is_stale reopens an invalid latch once the operator's configured string changes, on that string alone: no git and no stats. It compares against the string the latching attempt read rather than against MAIN_REPO, which is the _resolve_primary_checkout form of it, so a configured path pointing into a linked worktree does not read as changed on every poll. A resolved and valid install returns at its first guard with no await. Only dev_fleet.repo_path can self-heal. _load_dev_fleet_cfg re-reads config.json on every call, whereas KIROCREW_DEVFLEET_REPO is read off this process's own environment, which no outside shell can change, so that route still needs a restart and the setup card names the two routes separately rather than telling every user to restart. All 13 locales carry that caveat. Reopening also requires the config read itself to have succeeded. _load_dev_fleet_cfg answers an empty section for a file it cannot parse, so the configured path resolves to the same empty string a config naming no path gives. Treating that difference as a change would reopen the latch on evidence nobody read: discovery would find no configured path, fall through to the INFERRED tiers, and latch a checkout the operator never named while their own setting sat in a file the process merely failed to read, and every later git call, including the destructive ones, would target it. _load_dev_fleet_cfg_checked reports whether every file present parsed and _configured_main_repo_checked carries that flag beside the path, so a partial read holds the latch while a parseable file carrying no repo_path still reopens. _load_dev_fleet_cfg keeps its plain signature for the callers that read a single setting and have no use for the flag. _ensure_repo_resolved asks through _repo() rather than testing MAIN_REPO for truthiness, because _status_refresher returns on its first line for an unusable checkout, so starting it would mint a task per poll that dies immediately. An unresolved install spawns no subprocess, and the credential-helper warm keeps its own once-per-process sentinel. A late resolution restarts the status refresher, which returns rather than idles when nothing is resolved. Tests live in test/test_dev_fleet_repo_reresolution.py and are mutation-verified in both directions, including against the narrower fix that would freeze a genuinely cleared path. Fixes kirodotdev#3470 * fix(dev-fleet): carry one checked config snapshot through discovery The staleness test and discovery each needed the configured path, and each read config.json for itself. Two reads of one file can disagree: the first could see a whole, corrected path and reopen the latch while the second returned the empty string from a torn read, sending discovery to the INFERRED tiers. That latch is the dangerous one. An inferred checkout passes the marker test, so the resolution is VALID and therefore final: nothing re-resolves it, the staleness test only reopens a latched-INVALID resolution, and Pull + Build meanwhile mutates a checkout the operator never named. Only a gateway restart clears it. The attempt takes one checked read and hands it to _discover_main_repo, which accepts it as an optional argument and reads tier 2 itself only for a caller holding no snapshot, so dev_fleet_startup keeps its existing call. A partial read publishes nothing at all and the next poll retries against a settled file. Publishing nothing means publishing the UNRESOLVED state rather than leaving the previous one standing. _repo() gates on MAIN_REPO alone and never consults _DISCOVERY_DONE, so an attempt that returned with the provisional import-time hint in place would let every consumer operate on a checkout it could not confirm. That hint is what _default_main_repo_state picks before any config is read, and what dev_fleet_startup exists to replace and normalize, so an attempt that cannot read tier 2 has no basis for endorsing it. Cleared, _repo() raises RepoNotConfigured and the page shows the setup card. Clearing the published pair is not enough by itself. The reopen path arrives holding _DISCOVERY_DONE True, and the pre-lock gate returns early once MAIN_REPO and the message are both empty, so a torn read there cleared the pair and left the flag standing: the retry this branch promises was unreachable and the page kept the setup card until a gateway restart, the same failure this change exists to end. The branch clears the flag as well. The German and French cards read in one register. The new action string uses the informal address both style guides mandate, while the adjacent help and env-example strings carried the formal one, so a single card showed both. Those two neighbors move to the informal address in each locale; Spanish already reads that way throughout. Seven mutations, each reddening its own assertion: dropping the partial-read return, not handing the snapshot to discovery, having discovery ignore the snapshot and re-read, dropping each of the three parts of the clearing, and dropping the _DISCOVERY_DONE clear, which strands the next poll on the reopen path.
Problem / Motivation
A user who answers Dev Fleet's setup card keeps seeing the setup card.
ensure_main_repo_discovered()set_DISCOVERY_DONE = Trueat the end of every run, including a run that found nothing, so a process that started before the operator pointed Dev Fleet at a checkout answered "no checkout found" for the rest of its life.repo_pathwas the odd knob out.auto_pruneis re-read each reaper cycle andticket_url_templateis read per resolution, so both take effect while the gateway runs.Why it matters
This lands in the lowest-context moment the product has: a first run, on the one screen whose only job is to get the user to a working fleet. The old copy did name the restart, so it was not a trap, but "do this, then restart the gateway" is a worse loop than "do this" when nothing about the state actually requires a restart.
What changed
Symptom: the setup card never goes away. Root cause: the latch recorded a negative result as final. Change:
_DISCOVERY_DONE = bool(discovered), so only a resolved checkout latches, and/fleetcalls the newworktree_ops._ensure_repo_resolved()per poll. That returns at a truthiness guard before any await, so a resolved install pays nothing and only the state with no fleet re-runs discovery.Only one of the two documented remedies can self-heal, and the copy now says which. This corrects a premise in the issue, which lists setting
KIROCREW_DEVFLEET_REPOand addingdev_fleet.repo_pathas equivalent._load_dev_fleet_cfgre-readsconfig.jsonon every call, so a running process sees a newly writtenrepo_path.KIROCREW_DEVFLEET_REPOis read withos.environ.getoff this process's own environment, which no outside shell can change, so that route needs a restart and always will.no_checkout_found_actionpreviously ended "then restart the gateway" for both; it now leads with the config route and says plainly that the variable needs a restart.The refresher restart is not bookkeeping.
_status_refresherreturns when nothing is resolved rather than idling, on the previously correct reasoning that a startup-resolved global never becomes resolved later. A late resolution that left it stopped would serve a fleet whose rows never refresh again: the setup card disappears, the page looks alive, and nothing fetches. That is worse than the honest "restart the gateway" this replaces, so_ensure_repo_resolvedstarts it on the attempt that resolves. It lives inworktree_opsbecause the task handle is owned at that level andrepositorysits below it in the component DAG.No verdict outlives the attempt that produced it.
_REPO_INVALID_MSGis now assigned on both branches, so an attempt that finds nothing clears it instead of inheriting it. This is the hazard the triage proposal identified and the issue's own inventory missed:MAIN_REPOfrom a later attempt sitting beside an earlier attempt's validation verdict would hand_repo()a path whose markers were never checked, andworktree remove,update-ref -d,pull --ff-onlyandpip install -erun inside whatever that is.The retry spawns no subprocess.
_load_fallback_reposand_upstream_remoteboth return before reaching git while_repo()raises, and_resolve_primary_checkoutruns only on an attempt that found a path. The credential-helper warm keeps its ownNonesentinel because it reads--systemand--globalscope only and is repo-independent, so its twogit configcalls stay once-per-process rather than once-per-poll. What an unresolved poll costs is tiers 2 and 5 on the subprocess executor, every 12 seconds, only while the page is open.A resolved path that fails the marker test still latches, and that state renders the
RepoUnreadablebanner naming the path and the remedy. That latch is reopenable, and only tier 1 is fixed for the process: it is read off this process's own environment, which no outside shell can change, whereas tier 2 re-readsconfig.jsonon every call. So_invalid_resolution_is_stalereopens the latch once the configured string changes, which is what makes the card's "picks it up on its next refresh" promise true for the typo case as well as the not-found one. It compares against the string the latching attempt read rather than againstMAIN_REPO, which is the_resolve_primary_checkoutform of it, so a configured path pointing into a linked worktree does not read as changed on every poll. Both reads that decide where the fleet points go through_configured_main_repo_checked, and the attempt hands its one snapshot to_discover_main_reporather than letting it read tier 2 again, because two reads of one file can disagree and the inferred latch that disagreement produces would be valid, and therefore final.Two of the four blockers the issue lists needed no work, which the triage proposal established and I confirmed against the code. The AST ratchet checks
MAIN_REPOloads, not writers, so a second writer needs no exemption; both new guards are bare truthiness tests, which it permits._UPSTREAM_REMOTEand_FALLBACK_REPOSdecline to cache when_repo()raises, so neither was ever poisoned. Main also already carriedensure_main_repo_discovered()as a single-flight seam, so no new lock or entry point was needed.The decision to make this lifecycle change is recorded in #3470, raised by Design Review on #3406 and routed by triage. No user-facing default of a command, loop or agent changes: an unconfigured install that previously required a restart now resolves on its own, and a configured install behaves identically.
Backwards compatibility
Compatible on every wire shape, with one behaviour change declared below. Nothing is rejected, renamed or removed.
_DISCOVERY_DONEgoes from "always true after the first run" to "true once a checkout resolved", which only ever adds attempts in the state that had no fleet; a resolved install runs discovery exactly once, as before.RepoNotConfiguredstops being terminal for the process lifetime, and every existing catch site already treats it as a per-call condition and recomputes:_upstream_remotedegrades tooriginwithout caching,_load_fallback_reposleaves the listNonewhich consumers read asor [], and_auto_prune_reaperre-asks_repo()every cycle.GET /api/cronsand every other payload shape is unchanged; the only wire-visible difference is that/fleetcan stop reportingneeds_setupwithout a restart.The one behaviour change, stated plainly because it is a trade rather than a strict improvement: an attempt that cannot parse
config.jsonor its overlay publishes no checkout, where this base falls through to the inferred tiers. While a writer is mid-write that is a one-poll pause. If the file never parses, a stray comma left inconfig.local.jsonsay, the setup card stays where the base would have served an inferred fleet. That is intended. When the read fails there is no way to tell an operator who setdev_fleet.repo_pathfrom one who set nothing, so inferring serves the second correctly and silently runsPull + Build,worktree removeandupdate-ref -dinside a checkout the first never named; refusing costs the second a setup card until the file parses. Given which of those two errors is recoverable, refusing is the safer direction for a surface that runs destructive git. The residual is that the card attributes that state to a missing checkout rather than to the unreadable file, which is tracked in #12513 rather than folded in here.Tests
test/test_dev_fleet_repo_reresolution.py, 9 cases: an attempt that finds nothing does not latch, the first attempt that resolves does, a resolved-but-invalid path latches, a stale invalid-path message is cleared, the helper warm is paid once across three attempts, a resolved install runs no discovery, a late resolution starts the refresher, a still-unresolved attempt does not, and a disabled-background-tasks mode resolves without starting a task.Mutation-verified: 7 mutations applied to real source, 7 caught, both sources restored byte-identical. Each breaks one property, including restoring the original unconditional latch and letting the invalid-path verdict survive.
Also green locally: 84 backend cases across the accessor ratchet, component boundaries, gateway routes and the new file; 96
DevFleetPagecases; all 19 i18n checks includingchanged-passthroughandchanged-values; all 11 locale style suites, 95 cases. The gatescheck_sync_io_in_async,check_loop_bound_locks,check_feature_map,check_black_formatting,check_subprocess_encoding,check_focus_cue,check_comment_historyanddocs_lintpass, plusblackandflake8on the four changed Python files.The zh-CN style suite caught a real defect in my first draft: it stacked four possessive particles in one value where its rule permits two.
Catalog edits are textual, 1 line changed and 0 added per file, so the reserialisation that
duplicateKeys.test.tsdocuments dropping eleven Japanese translations is avoided.Manual verification
N/A for the mechanism, and stated rather than skipped. The behaviour is a module-global lifecycle over three call sites, and the 9 cases plus 7 mutations cover each transition directly, including the two that need an event loop (refresher start, and no start while unresolved). Reproducing it by hand needs a gateway that genuinely cannot find a checkout, which on this host means pointing a pod at a path with no checkout anywhere in its discovery tiers, and the observable result is the same boolean the tests assert.
The copy change is verified by the gates that own it rather than by eye:
changed-passthroughat zero for all 11 non-English catalogs, the 11 per-locale register suites, and the existingDevFleetPageassertion that the card still renders its env example.Related Issues
Fixes #3470
Supersedes #11972, which is this same commit raised from a fork. The fork
review lanes cannot complete:
fork-opus-review.ymlsetsegress-policy: blockwith an allowlist that omits the Claude Code installer host, so everyFork Opus run fails with
Failed to connect to claude.ai port 443 after 1 ms.That outage is tracked as #11911; #11976 allowed the bun release-asset host
but not the installer host, and #12081 / #12201 propose the remaining entry.
claude-review.yml, the same-repository lane, declares no egress policy, sothis branch can actually be reviewed. The tree here is byte-identical to
#11972 at 89cb4e2.
Pattern harvest
Rule candidate: review-prompt
Pattern: a single-flight latch records "done" after an attempt that produced no answer, freezing a state the operator can still change from outside the process. The generalization is worth a prompt line because the latch reads as correct at the call site (it is idempotent, it is single-flight, it does cache) and only the negative case is wrong. The same shape shipped in config-default materialization, where a stored value always beat a changed dataclass default. Worth asking of any
_DONE/_LOADED/is Noneguard: does it cache an ANSWER, or the absence of one?Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)