Skip to content

fix(dev-fleet): stamp the revision an artifact was built from - #22686

Merged
bolichen97 merged 1 commit into
mainfrom
kc-wt-10459
Nov 11, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
kc-wt-10459

Conversation

@dodgeblaster

@dodgeblaster dodgeblaster commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: make Dev Fleet provisioning report an artifact as provisioned only when it was built from the revision now checked out, so pod up never boots on a stale venv / node_modules / dist.

Provisioning reported whether an artifact EXISTS, never which revision it was
built from: has_dist was dist_dir(checkout).is_dir(), has_venv a bare
binary-presence check. So any operation that rewrites a worktree's tracked
content (a rebase or sync pulling a package-lock.json bump, an Advance
re-run past its early-return) left the tree reporting itself provisioned while
its artifacts belonged to the prior revision — and a pod up booted on that
stale bundle.

Why it matters

A pod booted on a stale bundle serves one revision's frontend against another's
backend (or an editable install / node_modules from before a dependency bump),
which fails silently: the dashboard renders, so the gap only surfaces as a
confusing behavioural mismatch. The existing _rebase/sync and Advance
re-entry paths all reproduce it.

Not a goal

  • No change to pod/systemd security, the pod_up/pod_down surface, or the
    systemd unit.
  • No hand-maintained build-input map. (The _BUILD_INPUTS map the original issue
    proposed retiring is not on main — it lived only on the closed-not-merged
    feat(dev-fleet): pin a worktree to the stable release channel #10066 — so there is nothing to remove.)
  • No new cross-process lock for the rare, loud, self-recovering overlap race
    (pod up already tells the user to re-run --provision).

What changed

The presence checks now mean present AND built from the artifact's current
build input
, keyed on the input (not HEAD), so a commit that changes no input
leaves the artifact current and forces no needless rebuild.

artifact build input where the built-from id is recorded
venv pyproject.toml + setup.cfg blob oids (deps are dynamic; the real install_requires live in setup.cfg) per-checkout manifest
node_modules website/package-lock.json + website/package.json blob oids (npm ci installs the lockfile and refuses one out of step with package.json) per-checkout manifest
dist HEAD:website tree oid the EXISTING build-id.json the dist carries
  • The dist reuses the frontend's own fingerprint — no new record.
    website/vite.config.ts's buildIdPlugin already writes the HEAD:website
    tree id as websiteTree into the dist's build-id.json on every vite build
    (provision's own and the Dev Fleet sync's), and
    frontend._write_build_source_fingerprint writes the same id beside a synced
    dist. has_dist reads that record through
    read_dist_tree in dashboard/stale_bundle_guard.py — the one reader of those stamps,
    which takes the dist directory as an argument — and compares it to the current
    HEAD:website.
    A separate manifest key would have drifted — the sync path rebuilds the dist
    without touching a manifest.
  • For the venv and node_modules, each build step captures its input oid
    BEFORE building and stamps it on success into a per-checkout manifest; the
    check compares that stamp to the input's current oid. Capturing before the
    build means a revision that advances during the build leaves the artifact
    correctly stale, not certified against the post-build HEAD.
  • input unresolvable (non-git tree — tarball install, test fixture — or no
    trusted git) → presence-only, so a tree with no resolvable revision keeps
    reporting what it has.
  • The primary checkout's venv stays presence-only. Pull + Build rebuilds it
    without writing the stamp and pods never run there, so the Fleet poll calls
    has_venv(path, require_stamp=False) for the is_main row; every other
    worktree keeps the stamp requirement.
  • git is resolved to a trusted absolute executable
    (platform_compat.trusted_git_bin, under hardened_git_env), never the
    service PATH, which leads with agent-writable dirs; an untrusted/absent git
    makes the input unresolvable rather than spawning a possibly-planted shim.
  • The venv/node_modules manifest lives inside the git directory (git rev-parse --git-path kirocrew-provision.json), not the working tree, so it never dirties
    a rebase/prune and needs no ignore rule — a worktree whose revision predates
    this change is not dirtied by a provision.
  • Existing worktrees rebuild once. A git worktree provisioned before the
    manifest existed carries no stamp, so its venv and node_modules read "not
    built" until its next provision, which rebuilds them once and records the stamp.

This closes the rebase/sync invalidation gap and the Advance re-entry gap.
The checks live in pod/provision.py; Dev Fleet's fleet_state.py picks the
main checkout's probe, and its apps/builtins/dev_fleet/runtime.py comment and docs/system-specs/modules/dev-fleet.md
describe them.

Tests

test/test_pod_provision.py::TestProvisionProvenance (real git repos under
tmp_path, hermetic git identity): a doc-only commit does not invalidate venv or
dist; venv restales only on a pyproject.toml change; node_modules restales only
on a package-lock.json change, not a source edit; has_dist reads the vite
build-id.json / kirocrew-build-source.txt fingerprint through the bundle
guard's read_dist_tree (pinned by a test that swaps the reader) and recognises
a synced dist; the pre-build input oid (not completion HEAD) is what is stamped; the
manifest lives in the git dir and leaves git status clean; untrusted/absent git
→ presence-only; non-git tree → presence-only; corrupt manifest → not-provisioned;
an unreadable manifest is never rewritten by a stamp or clear;
the uv venv path does not fall through to pip; an unstamped venv reads built only with require_stamp=False, and the Fleet poll reports an unstamped main venv as built while an unstamped worktree venv reads not built (test/test_dev_fleet_app_context.py::test_unstamped_main_venv_reads_built_but_a_worktree_needs_the_stamp, red with the fleet_state.py change reverted; 10 repeats green, and green in the xdist-distributed suite run). Full test_pod_provision.py
passes; flake8 / isort / mypy / docs-lint clean on the changed files.

Fixes #10459

Pattern harvest

Not generalizable: the defect was a design gap — a presence check standing in for
a provenance check — not a mechanical code smell a grep/semgrep rule could catch.
A .is_dir() / .exists() probe is correct in most contexts; only "does this
prove the artifact matches the current source?" makes it wrong, which is a
semantic question about intent, not a syntactic pattern. The durable guard is the
behavioural TestProvisionProvenance suite, which pins that each has_* compares
the artifact to its build input rather than mere presence.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 21, 2026
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision f513d595eeaf4feded86a9c7cbbe2ed356f28cb5 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of f513d595eeaf4feded86a9c7cbbe2ed356f28cb5 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] f513d59

False positive or not applicable? A repository writer can comment:
/ai-review override gpt f513d595eeaf4feded86a9c7cbbe2ed356f28cb5 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of f513d595eeaf4feded86a9c7cbbe2ed356f28cb5 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Every item exists to remove the stale-bundle boot in #10459. The dist check reuses the existing build-id.json stamp and its one reader, so no second record was added.

What this change ships

Inventory (10 items) — 8 justified

Intent: make pod up refuse a worktree whose venv, node_modules or dist was built from an earlier revision. This is a FIX.

  1. A worktree venv reads "not built" once pyproject.toml/setup.cfg change after it was built — justified
  2. The npm install step re-runs once website/package-lock.json/package.json change — justified
  3. The dist reads "not built" when its build-id.json tree differs from HEAD:website — justified
  4. The stale-bundle guard's dist-stamp reader becomes public and takes a dist directory — justified
  5. The Fleet view keeps reporting the main checkout's venv as built without a stamp — justified
  6. Worktrees provisioned before this change rebuild their venv and node_modules once — justified
  7. The provenance stamp file lives in the git dir, so git status stays clean — justified
  8. Without a trusted git, or outside a git tree, the checks fall back to presence-only — justified
  9. A build's old stamp is cleared before it starts, so an interrupted rebuild is not vouched for — undeclared
  10. ensure_venv/ensure_node_modules return false if the build input moved during the build — undeclared

[FIRST-PRINCIPLES-REVIEWED] f513d59

Description read

[DESCRIPTION-READ] a35605b04e247dfde7bab8e4257b9905234adce72f81f11474574c2c08ed6ace

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of f513d595eeaf4feded86a9c7cbbe2ed356f28cb5 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks. The stricter has_venv/has_dist is read outside the patch only by pod/cli.py's pod up, which still works: a stale venv rebuilds, and a stale dist fails loudly.

Notes

  • A stale dist now hits pod up's "no built dist for X" error, which tells the user to build it as if it were missing; the real cause is a dist from an earlier website/ tree. Wording that says the bundle is stale would save a confused round-trip.
  • The description has no ## Backwards compatibility section, though has_venv/has_dist now reject trees they used to accept. It should list src/kiro_crew/pod/cli.py:_cmd_up (pod up path; still works: rebuilds the venv once, a stale dist fails loudly).

[DESIGN-REVIEWED] f513d59

Description read

[DESCRIPTION-READ] 8e2f4b5d918b80eab87b4e3168528dc126221e4fe6eb020aa2f265da530d6106

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed f513d595eeaf4feded86a9c7cbbe2ed356f28cb5 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] f513d59

Verdict parsed from the review's SHA-scoped output markers for commit f513d595eeaf4feded86a9c7cbbe2ed356f28cb5.

False positive or not applicable? A repository writer can comment:
/ai-review override fable f513d595eeaf4feded86a9c7cbbe2ed356f28cb5 false-positive: <reason, 20 to 500 characters>

@dodgeblaster
dodgeblaster force-pushed the kc-wt-10459 branch 2 times, most recently from 38f0613 to 7e5ed1f Compare October 21, 2026 07:23
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 21, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 21, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 21, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 21, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026
@github-actions github-actions Bot added the readiness: passed Eligible automated validation passed for the current revision label Nov 2, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Hold at 397b8dac12c0962c26039c9291686ef0badf6a3d for a human reviewer.

The r136 defect is fixed. _manifest_path now runs git with env=hardened_git_env(), matching _resolve_oid (provision.py L449-L456), and a regression test covers an inherited GIT_DIR. The provenance design still reads as sound, and checks are green on this head.

What still needs a human: this PR adds two gateway git spawns to the BENIGN_PACKAGE_SPAWNS spawn-audit allowlist (spawn_audit_helpers.py L1055-L1066). These spawns run on every Fleet poll against agent-writable worktrees. Widening a security allowlist is reserved for a maintainer, so a human reviewer's approval clears this hold.

AI findings judged non-blocking:

  • GPT 6.1 F1 (downgraded): other installers do not clear the stamps, so an exact revert of the build inputs can accept stale dependencies. This is no worse than the presence-only base, and the next Pull+Build reinstalls without reading the stamp (L574).
  • Design Review: about four git rev-parse spawns per worktree per poll. A performance suggestion; the poll runs them off the event loop.

— crew-pr-reviewer (r141)

@bolichen97

Copy link
Copy Markdown
Contributor

@iamwhatever, could you take another look when you have a moment? Your requested change looks addressed at 397b8da: _manifest_path in pod/provision.py now passes env=hardened_git_env() to its subprocess.run, matching _resolve_oid, and its docstring says an inherited GIT_DIR or GIT_* variable cannot redirect the manifest path. The PR is mergeable on this head. Your review is the remaining blocker. Thanks.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 7, 2026
@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=8f085147bfe0 Second copy of the dist-tree reader

fixed in 5ae8c59: _dist_build_tree is deleted from pod/provision.py; has_dist now calls read_dist_tree(dist) in dashboard/stale_bundle_guard.py, which takes the dist directory as an argument (None = the served dist), so one function decides which build stamp wins.
provision.py no longer parses build-id.json / websiteTree itself (the Clears-when condition).
Regression: test_dist_reads_its_stamp_through_the_bundle_guard_reader swaps the guard reader and asserts has_dist follows it; it fails on the prior head's private copy.
The two undeclared Notes (package.json / setup.cfg inputs; existing worktrees read not-built until re-provisioned) are now stated in the PR body and in docs/system-specs/modules/dev-fleet.md.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author

@iamwhatever — your request-changes review at 95d7ec8 is still answered as below, now on head bd9829f (a rebase onto current main, which had conflicted).

fixed (carried unchanged into bd9829f): _manifest_path runs git rev-parse --git-path with env=hardened_git_env(), matching _resolve_oid, so the BENIGN_PACKAGE_SPAWNS row's stated basis holds for both probes.

src/kiro_crew/pod/provision.py::_manifest_path passes env=hardened_git_env(); inherited GIT_* variables are dropped before the spawn.
Regression test test/test_pod_provision.py::TestProvisionProvenance::test_inherited_git_dir_cannot_redirect_the_manifest sets GIT_DIR to another repo's git dir and asserts the manifest still resolves inside the checkout's own git dir.

Rebase: still one commit, same six files. The only conflict was a comment in dev_fleet/runtime.py; main's new _POD_AVAILABLE wording (systemd, launchd or Task Scheduler) is kept and the PR's has_dist wording re-applied. test/test_pod_provision.py (67) and the related dev-fleet / stale-bundle / spawn-audit suites (262) pass locally on the new head.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 9, 2026
@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=28eaf1307ffd Main checkout's venv always reports as unbuilt

fixed in 586c1ee: has_venv takes require_stamp (default True); fleet_state._build_fleet passes require_stamp=not is_main, so the primary checkout, whose venv Pull + Build rebuilds without stamping, gets the presence-only probe. Every other worktree keeps the stamp requirement.
The sync runner is unchanged: pods never run on main, so a stamp there would certify nothing a pod reads.
Regression: test/test_dev_fleet_app_context.py::test_unstamped_main_venv_reads_built_but_a_worktree_needs_the_stamp (unstamped main venv reads built, unstamped worktree venv does not) fails with the fleet_state change reverted. TestProvisionProvenance::test_unstamped_venv_reads_built_only_without_the_stamp_requirement pins the probe itself.
Spec (dev-fleet.md Build-state provenance), docstrings, the fleet_state comment and the PR body describe the main-checkout case.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=8f085147bfe0 Second copy of the dist-tree reader

fixed in 5ae8c59: _dist_build_tree is deleted from pod/provision.py; has_dist now calls read_dist_tree(dist) in dashboard/stale_bundle_guard.py, which takes the dist directory as an argument (None = the served dist), so one function decides which build stamp wins.
provision.py no longer parses build-id.json / websiteTree itself (the Clears-when condition).
Regression: test_dist_reads_its_stamp_through_the_bundle_guard_reader swaps the guard reader and asserts has_dist follows it; it fails on the prior head's private copy.
The two undeclared Notes (package.json / setup.cfg inputs; existing worktrees read not-built until re-provisioned) are now stated in the PR body and in docs/system-specs/modules/dev-fleet.md.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=10f587d8f2ca Dependency stamps certify HEAD instead of the inputs actually installed

rebutted (disproportional): the case needs an uncommitted edit to pyproject.toml / setup.cfg / the website lockfile at the moment provisioning runs, followed by reverting that edit; the outcome is a dev pod venv or node_modules built from the developer's own discarded edit, recovered by re-running provision.
The base's has_venv was presence-only, so this case was already unchecked before this PR; the PR is a strict improvement over it.
The fix would add a second working-tree fingerprint scheme and a dirty-tree policy across every build step and freshness check, which outweighs a rare, self-inflicted, recoverable mismatch in dev tooling. The Opus adjudication downgraded it on the same grounds.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=10f587d8f2ca Completion-time dist stamps can certify an older bundle

rebutted (disproportional): it needs a clean, frontend-only rebase in the same checkout that lands inside one vite build's compile window. The worst case is that the dist reads as current when it is stale, which is exactly what the base's presence-only has_dist reported on every run. No data is lost, and the next website/ change mismatches the stamp and forces a rebuild.
The stamp is written when the build finishes, by two writers this PR does not change (website/vite.config.ts buildIdPlugin, frontend.py). Capturing the tree before compiling means a new contract across a TypeScript and a Python writer plus every reader. That costs more than a rare, self-correcting stale dev bundle. Opus adjudication downgraded it on the same grounds.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=10f587d8f2ca Failed invalidation leaves an old stamp certifying rebuilt dependencies

rebutted (disproportional): it needs a manifest inside the git dir that can be read but not written, and the operator then checking out B and back to A in that same worktree. git checkout itself writes HEAD and the index in that git dir, and nothing provision writes leaves the file read-only.
Outcome when it does happen: the venv or node_modules reads as current when it is not. That is exactly what the base's presence-only has_venv / _has_node_modules reported, so this is no regression. It also surfaces quickly as an import or tsc failure, and deleting the artifact fixes it.
Stamp and clear are deliberately best-effort bookkeeping. Aborting provisioning when the old stamp cannot be cleared would make every checkout with an unwritable manifest fail to provision, which costs more than this rare, recoverable mismatch. Opus adjudication downgraded it on the same grounds.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=28eaf1307ffd Main checkout's venv always reports as unbuilt

fixed in 586c1ee (carried unchanged into a4f7f1b, a rebase onto main): has_venv takes require_stamp (default True); fleet_state._build_fleet passes require_stamp=not is_main, so the primary checkout, whose venv Pull + Build rebuilds without stamping, gets the presence-only probe. Every other worktree keeps the stamp requirement.
The sync runner is unchanged: pods never run on main, so a stamp there would certify nothing a pod reads.
Regression: test/test_dev_fleet_app_context.py::test_unstamped_main_venv_reads_built_but_a_worktree_needs_the_stamp (unstamped main venv reads built, unstamped worktree venv does not) fails with the fleet_state change reverted. TestProvisionProvenance::test_unstamped_venv_reads_built_only_without_the_stamp_requirement pins the probe itself.
Spec (dev-fleet.md Build-state provenance), docstrings, the fleet_state comment and the PR body describe the main-checkout case.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=8f085147bfe0 Second copy of the dist-tree reader

fixed in 5ae8c59: _dist_build_tree is deleted from pod/provision.py; has_dist now calls read_dist_tree(dist) in dashboard/stale_bundle_guard.py, which takes the dist directory as an argument (None = the served dist), so one function decides which build stamp wins.
provision.py no longer parses build-id.json / websiteTree itself (the Clears-when condition).
Regression: test_dist_reads_its_stamp_through_the_bundle_guard_reader swaps the guard reader and asserts has_dist follows it; it fails on the prior head's private copy.
The two undeclared Notes (package.json / setup.cfg inputs; existing worktrees read not-built until re-provisioned) are now stated in the PR body and in docs/system-specs/modules/dev-fleet.md.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=10f587d8f2ca Dependency stamps certify HEAD instead of the inputs actually installed

rebutted (disproportional): the case needs an uncommitted edit to pyproject.toml / setup.cfg / the website lockfile at the moment provisioning runs, followed by reverting that edit; the outcome is a dev pod venv or node_modules built from the developer's own discarded edit, recovered by re-running provision.
The base's has_venv was presence-only, so this case was already unchecked before this PR; the PR is a strict improvement over it.
The fix would add a second working-tree fingerprint scheme and a dirty-tree policy across every build step and freshness check, which outweighs a rare, self-inflicted, recoverable mismatch in dev tooling. The Opus adjudication downgraded it on the same grounds.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=10f587d8f2ca Completion-time dist stamps can certify an older bundle

rebutted (disproportional): it needs a clean, frontend-only rebase in the same checkout that lands inside one vite build's compile window. The worst case is that the dist reads as current when it is stale, which is exactly what the base's presence-only has_dist reported on every run. No data is lost, and the next website/ change mismatches the stamp and forces a rebuild.
The stamp is written when the build finishes, by two writers this PR does not change (website/vite.config.ts buildIdPlugin, frontend.py). Capturing the tree before compiling means a new contract across a TypeScript and a Python writer plus every reader. That costs more than a rare, self-correcting stale dev bundle. Opus adjudication downgraded it on the same grounds.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=10f587d8f2ca Failed invalidation leaves an old stamp certifying rebuilt dependencies

rebutted (disproportional): it needs a manifest inside the git dir that can be read but not written, and the operator then checking out B and back to A in that same worktree. git checkout itself writes HEAD and the index in that git dir, and nothing provision writes leaves the file read-only.
Outcome when it does happen: the venv or node_modules reads as current when it is not. That is exactly what the base's presence-only has_venv / _has_node_modules reported, so this is no regression. It also surfaces quickly as an import or tsc failure, and deleting the artifact fixes it.
Stamp and clear are deliberately best-effort bookkeeping. Aborting provisioning when the old stamp cannot be cleared would make every checkout with an unwritable manifest fail to provision, which costs more than this rare, recoverable mismatch. Opus adjudication downgraded it on the same grounds.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at a4f7f1b.

has_venv / has_dist / _has_node_modules now mean "present and built from the current build input" instead of just "present":

  • venv: the inputs are the pyproject.toml and setup.cfg oids. node_modules: the inputs are website/package-lock.json and website/package.json. Both are stamped in a per-worktree manifest under git rev-parse --git-path, so git status stays clean.
  • dist: it gets no manifest key. has_dist compares the dist's own build-id.json / kirocrew-build-source.txt (read through the now-shared stale_bundle_guard.read_dist_tree) with HEAD:website.
  • Build order: inputs are captured before the build, and the old stamp is cleared before the first mutating step. Each build path revalidates after stamping, so a rebase mid-build reads stale instead of certified.
  • Fallbacks: a non-git tree, no trusted git, or an untracked input falls back to presence-only. The primary checkout's venv is probed with require_stamp=False, because Pull + Build rebuilds it without stamping.
  • Manifest writes: a manifest that exists but can't be read is never overwritten with one that has lost the other stamps.

The open request-changes point from iamwhatever is fixed. _manifest_path now runs git with env=hardened_git_env() like _resolve_oid, and test_inherited_git_dir_cannot_redirect_the_manifest covers it. Both new spawn-audit rows run a trusted absolute git with constant argv under the hardened env. There is no regex.

CI: all 104 checks are green on this head. macOS shard 2 was cancelled on a test timeout and passed on rerun.

… presence

Provisioning checked only whether a venv/node_modules/dist EXISTED, never
which revision it was built from, so any operation that rewrites a worktree's
tracked content (a rebase or sync pulling a package-lock.json bump, an
Advance re-run past its early-return) left the tree reporting itself
provisioned while its artifacts belonged to the prior revision -- and a
`pod up` booted on that stale bundle.

Record each artifact's BUILD INPUT oid in a per-checkout manifest and make
has_venv / has_dist / _has_node_modules mean "present AND built from the
current input". Staleness is keyed on the input, not HEAD, so a commit that
changes neither input (a doc edit) leaves the artifacts current and does not
force a needless multi-minute rebuild:

- venv          -> HEAD:pyproject.toml (an editable install only restales
                   when pyproject moves)
- node_modules  -> HEAD:website        (the tree the lockfile/deps live in)
- dist          -> HEAD:website        (the SPA source tree)

The input oid is captured BEFORE each build and stamped afterward, so a
revision that advances during a build leaves the artifact correctly stale
rather than certified against the post-build HEAD. An unresolvable input
(non-git tree, or no trusted git) defers to presence-only, so a tarball
install keeps reporting what it has.

- git is resolved to a trusted absolute executable
  (platform_compat.trusted_system_bin_quiet), never the service PATH, which
  leads with agent-writable dirs; an untrusted/absent git makes the input
  unresolvable rather than spawning a possibly-planted shim.
- The manifest lives inside the git directory (git rev-parse --git-path),
  not the working tree, so it never dirties a rebase/prune and needs no
  ignore rule -- a worktree whose revision predates this change is not
  dirtied by a provision.

This closes the rebase/sync invalidation gap and the Advance re-entry gap
without a hand-maintained input map. The _BUILD_INPUTS map the original
issue proposed retiring is not present on main (it lived only on the
closed-not-merged #10066), so there is nothing to remove.

Fixes #10459
@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=28eaf1307ffd Main checkout's venv always reports as unbuilt

fixed in 586c1ee (carried unchanged through rebases into f513d59): has_venv takes require_stamp (default True); fleet_state._build_fleet passes require_stamp=not is_main, so the primary checkout, whose venv Pull + Build rebuilds without stamping, gets the presence-only probe. Every other worktree keeps the stamp requirement.
The sync runner is unchanged: pods never run on main, so a stamp there would certify nothing a pod reads.
Regression: test/test_dev_fleet_app_context.py::test_unstamped_main_venv_reads_built_but_a_worktree_needs_the_stamp (unstamped main venv reads built, unstamped worktree venv does not) fails with the fleet_state change reverted. TestProvisionProvenance::test_unstamped_venv_reads_built_only_without_the_stamp_requirement pins the probe itself.
Spec (dev-fleet.md Build-state provenance), docstrings, the fleet_state comment and the PR body describe the main-checkout case.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=8f085147bfe0 Second copy of the dist-tree reader

fixed in 5ae8c59: _dist_build_tree is deleted from pod/provision.py; has_dist now calls read_dist_tree(dist) in dashboard/stale_bundle_guard.py, which takes the dist directory as an argument (None = the served dist), so one function decides which build stamp wins.
provision.py no longer parses build-id.json / websiteTree itself (the Clears-when condition).
Regression: test_dist_reads_its_stamp_through_the_bundle_guard_reader swaps the guard reader and asserts has_dist follows it; it fails on the prior head's private copy.
The two undeclared Notes (package.json / setup.cfg inputs; existing worktrees read not-built until re-provisioned) are now stated in the PR body and in docs/system-specs/modules/dev-fleet.md.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=10f587d8f2ca Dependency stamps certify HEAD instead of the inputs actually installed

rebutted (disproportional): the case needs an uncommitted edit to pyproject.toml / setup.cfg / the website lockfile at the moment provisioning runs, followed by reverting that edit; the outcome is a dev pod venv or node_modules built from the developer's own discarded edit, recovered by re-running provision.
The base's has_venv was presence-only, so this case was already unchecked before this PR; the PR is a strict improvement over it.
The fix would add a second working-tree fingerprint scheme and a dirty-tree policy across every build step and freshness check, which outweighs a rare, self-inflicted, recoverable mismatch in dev tooling. The Opus adjudication downgraded it on the same grounds.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=10f587d8f2ca Completion-time dist stamps can certify an older bundle

rebutted (disproportional): it needs a clean, frontend-only rebase in the same checkout that lands inside one vite build's compile window. The worst case is that the dist reads as current when it is stale, which is exactly what the base's presence-only has_dist reported on every run. No data is lost, and the next website/ change mismatches the stamp and forces a rebuild.
The stamp is written when the build finishes, by two writers this PR does not change (website/vite.config.ts buildIdPlugin, frontend.py). Capturing the tree before compiling means a new contract across a TypeScript and a Python writer plus every reader. That costs more than a rare, self-correcting stale dev bundle. Opus adjudication downgraded it on the same grounds.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author
  • span=10f587d8f2ca Failed invalidation leaves an old stamp certifying rebuilt dependencies

rebutted (disproportional): it needs a manifest inside the git dir that can be read but not written, and the operator then checking out B and back to A in that same worktree. git checkout itself writes HEAD and the index in that git dir, and nothing provision writes leaves the file read-only.
Outcome when it does happen: the venv or node_modules reads as current when it is not. That is exactly what the base's presence-only has_venv / _has_node_modules reported, so this is no regression. It also surfaces quickly as an import or tsc failure, and deleting the artifact fixes it.
Stamp and clear are deliberately best-effort bookkeeping. Aborting provisioning when the old stamp cannot be cleared would make every checkout with an unwritable manifest fail to provision, which costs more than this rare, recoverable mismatch. Opus adjudication downgraded it on the same grounds.

@dodgeblaster

Copy link
Copy Markdown
Contributor Author

@iamwhatever — your request-changes review at 95d7ec8 is still answered as below, now on head f513d59 (a rebase onto current main).

fixed (carried unchanged into f513d59): _manifest_path runs git rev-parse --git-path with env=hardened_git_env(), matching _resolve_oid, so the BENIGN_PACKAGE_SPAWNS row's stated basis holds for both probes.

src/kiro_crew/pod/provision.py::_manifest_path passes env=hardened_git_env(); inherited GIT_* variables are dropped before the spawn.
Regression test test/test_pod_provision.py::TestProvisionProvenance::test_inherited_git_dir_cannot_redirect_the_manifest sets GIT_DIR to another repo's git dir and asserts the manifest still resolves inside the checkout's own git dir.

Rebase: still one commit, same six files. The only conflict was a comment in dev_fleet/runtime.py; main's new _POD_AVAILABLE wording (systemd, launchd or Task Scheduler) is kept and the PR's has_dist wording re-applied. test/test_pod_provision.py (67) and the related dev-fleet / stale-bundle / spawn-audit suites (262) pass locally on the new head.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving f513d59 with auto-merge. I re-read the whole diff on this head, tests and docs included. iamwhatever approved this head at 05:35, so the points from the 10-30 review are settled.

  • has_venv / has_dist / _has_node_modules now mean "present and built from the current build input": pyproject.toml + setup.cfg for the venv, website/package-lock.json + package.json for node_modules, and HEAD:website against the dist's own build-id.json read through stale_bundle_guard.read_dist_tree.
  • The input oid is captured before the build and the old stamp is cleared before the first mutating step, so a mid-build rebase or an interrupted install cannot certify a stale tree. Both are tested.
  • The manifest lives in the git dir, git is resolved only through trusted_git_bin under hardened_git_env, and a non-git tree or missing trusted git falls back to presence-only. The main checkout's venv uses require_stamp=False, since Pull + Build never stamps it.
  • No regex in product code. All 98 checks on this head are green and the PR is clean to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dev Fleet: provisioning reports presence, not the revision an artifact was built from

4 participants