Repository navigation
fix(dev-fleet): re-resolve the main checkout without a gateway restart - #11972
chenmingwei23 wants to merge 1 commit into
Conversation
ensure_main_repo_discovered() latched unconditionally, so a gateway that started before the operator pointed Dev Fleet at a checkout kept answering "no checkout found" until it was restarted. It now records "done" only once a checkout RESOLVED, and /fleet re-runs it per poll while nothing is. Only dev_fleet.repo_path can self-heal: _load_dev_fleet_cfg re-reads config.json on every call, whereas KIROCREW_DEVFLEET_REPO is read off this process's own environment, which no outside shell can change. The setup card's copy now names the two routes separately instead of telling every user to restart. A resolved install returns at a truthiness guard before any await, so the retry costs nothing once there is a fleet. An unresolved one spawns no subprocess, and the credential-helper warm keeps its own once-per-process sentinel. A late resolution restarts the status refresher, which returns rather than idles when nothing is resolved. Fixes kirodotdev#3470
3a6f3a6 to
89cb4e2
Compare
Opus 4.8 Review (fork) —
|
First Principles Review (Fable 5, fork) —
|
Design Review (Fable 5, fork) —
|
GPT 5.6 Review (fork) —
|
|
Superseded by #12463, which carries this exact commit (89cb4e2) on a same-repository branch. This PR cannot reach a verdict. That outage is tracked as #11911. #11976 allowed the bun release-asset host, which did clear the earlier An override cannot resolve this. An override disposes of a finding, and there is no verdict here to dispose of. Closing in favour of #12463. Nothing about the change itself is in question: this head measured 77 lanes success, 3 advisory neutral, mergeable true, with the two review lanes as the only outstanding items. The fork branch is left intact. |
Problem / Motivation
A user who answers Dev Fleet's setup card keeps seeing the setup card.
ensure_main_repo_discovered()set_DISCOVERY_DONE = Trueat the end of every run, including a run that found nothing, so a process that started before the operator pointed Dev Fleet at a checkout answered "no checkout found" for the rest of its life.repo_pathwas the odd knob out.auto_pruneis re-read each reaper cycle andticket_url_templateis read per resolution, so both take effect while the gateway runs.Why it matters
This lands in the lowest-context moment the product has: a first run, on the one screen whose only job is to get the user to a working fleet. The old copy did name the restart, so it was not a trap, but "do this, then restart the gateway" is a worse loop than "do this" when nothing about the state actually requires a restart.
What changed
Symptom: the setup card never goes away. Root cause: the latch recorded a negative result as final. Change:
_DISCOVERY_DONE = bool(discovered), so only a resolved checkout latches, and/fleetcalls the newworktree_ops._ensure_repo_resolved()per poll. That returns at a truthiness guard before any await, so a resolved install pays nothing and only the state with no fleet re-runs discovery.Only one of the two documented remedies can self-heal, and the copy now says which. This corrects a premise in the issue, which lists setting
KIROCREW_DEVFLEET_REPOand addingdev_fleet.repo_pathas equivalent._load_dev_fleet_cfgre-readsconfig.jsonon every call, so a running process sees a newly writtenrepo_path.KIROCREW_DEVFLEET_REPOis read withos.environ.getoff this process's own environment, which no outside shell can change, so that route needs a restart and always will.no_checkout_found_actionpreviously ended "then restart the gateway" for both; it now leads with the config route and says plainly that the variable needs a restart.The refresher restart is not bookkeeping.
_status_refresherreturns when nothing is resolved rather than idling, on the previously correct reasoning that a startup-resolved global never becomes resolved later. A late resolution that left it stopped would serve a fleet whose rows never refresh again: the setup card disappears, the page looks alive, and nothing fetches. That is worse than the honest "restart the gateway" this replaces, so_ensure_repo_resolvedstarts it on the attempt that resolves. It lives inworktree_opsbecause the task handle is owned at that level andrepositorysits below it in the component DAG.No verdict outlives the attempt that produced it.
_REPO_INVALID_MSGis now assigned on both branches, so an attempt that finds nothing clears it instead of inheriting it. This is the hazard the triage proposal identified and the issue's own inventory missed:MAIN_REPOfrom a later attempt sitting beside an earlier attempt's validation verdict would hand_repo()a path whose markers were never checked, andworktree remove,update-ref -d,pull --ff-onlyandpip install -erun inside whatever that is.The retry spawns no subprocess.
_load_fallback_reposand_upstream_remoteboth return before reaching git while_repo()raises, and_resolve_primary_checkoutruns only on an attempt that found a path. The credential-helper warm keeps its ownNonesentinel because it reads--systemand--globalscope only and is repo-independent, so its twogit configcalls stay once-per-process rather than once-per-poll. What an unresolved poll costs is tiers 2 and 5 on the subprocess executor, every 12 seconds, only while the page is open.A resolved path that fails the marker test still latches. Tiers 1 and 2 are taken verbatim, so re-running returns the same path every time, and that state renders the
RepoUnreadablebanner naming the path and the remedy.Two of the four blockers the issue lists needed no work, which the triage proposal established and I confirmed against the code. The AST ratchet checks
MAIN_REPOloads, not writers, so a second writer needs no exemption; both new guards are bare truthiness tests, which it permits._UPSTREAM_REMOTEand_FALLBACK_REPOSdecline to cache when_repo()raises, so neither was ever poisoned. Main also already carriedensure_main_repo_discovered()as a single-flight seam, so no new lock or entry point was needed.The decision to make this lifecycle change is recorded in #3470, raised by Design Review on #3406 and routed by triage. No user-facing default of a command, loop or agent changes: an unconfigured install that previously required a restart now resolves on its own, and a configured install behaves identically.
Backwards compatibility
Compatible. Nothing is rejected, renamed or removed.
_DISCOVERY_DONEgoes from "always true after the first run" to "true once a checkout resolved", which only ever adds attempts in the state that had no fleet; a resolved install runs discovery exactly once, as before.RepoNotConfiguredstops being terminal for the process lifetime, and every existing catch site already treats it as a per-call condition and recomputes:_upstream_remotedegrades tooriginwithout caching,_load_fallback_reposleaves the listNonewhich consumers read asor [], and_auto_prune_reaperre-asks_repo()every cycle.GET /api/cronsand every other payload shape is unchanged; the only wire-visible difference is that/fleetcan stop reportingneeds_setupwithout a restart.Tests
test/test_dev_fleet_repo_reresolution.py, 9 cases: an attempt that finds nothing does not latch, the first attempt that resolves does, a resolved-but-invalid path latches, a stale invalid-path message is cleared, the helper warm is paid once across three attempts, a resolved install runs no discovery, a late resolution starts the refresher, a still-unresolved attempt does not, and a disabled-background-tasks mode resolves without starting a task.Mutation-verified: 7 mutations applied to real source, 7 caught, both sources restored byte-identical. Each breaks one property, including restoring the original unconditional latch and letting the invalid-path verdict survive.
Also green locally: 84 backend cases across the accessor ratchet, component boundaries, gateway routes and the new file; 96
DevFleetPagecases; all 19 i18n checks includingchanged-passthroughandchanged-values; all 11 locale style suites, 95 cases. The gatescheck_sync_io_in_async,check_loop_bound_locks,check_feature_map,check_black_formatting,check_subprocess_encoding,check_focus_cue,check_comment_historyanddocs_lintpass, plusblackandflake8on the four changed Python files.The zh-CN style suite caught a real defect in my first draft: it stacked four possessive particles in one value where its rule permits two.
Catalog edits are textual, 1 line changed and 0 added per file, so the reserialisation that
duplicateKeys.test.tsdocuments dropping eleven Japanese translations is avoided.Manual verification
N/A for the mechanism, and stated rather than skipped. The behaviour is a module-global lifecycle over three call sites, and the 9 cases plus 7 mutations cover each transition directly, including the two that need an event loop (refresher start, and no start while unresolved). Reproducing it by hand needs a gateway that genuinely cannot find a checkout, which on this host means pointing a pod at a path with no checkout anywhere in its discovery tiers, and the observable result is the same boolean the tests assert.
The copy change is verified by the gates that own it rather than by eye:
changed-passthroughat zero for all 11 non-English catalogs, the 11 per-locale register suites, and the existingDevFleetPageassertion that the card still renders its env example.Related Issues
Fixes #3470
Pattern harvest
Rule candidate: review-prompt
Pattern: a single-flight latch records "done" after an attempt that produced no answer, freezing a state the operator can still change from outside the process. The generalization is worth a prompt line because the latch reads as correct at the call site (it is idempotent, it is single-flight, it does cache) and only the negative case is wrong. The same shape shipped in config-default materialization, where a stored value always beat a changed dataclass default. Worth asking of any
_DONE/_LOADED/is Noneguard: does it cache an ANSWER, or the absence of one?Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)