Skip to content

fix(agents): heal a default agent only after a scan that read every spec - #16512

Merged
bolichen97 merged 1 commit into
mainfrom
fix/heal-only-after-complete-scan
Oct 3, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/heal-only-after-complete-scan

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: the default-agent healer never resets the owner's default_agent because of a scan that could not read every spec.

_scan_materialized_index (config/loader.py) skips a spec it cannot read or parse: the except (ValueError, OSError): continue. If the owner's catalog fetch lands while the default's spec is mid-write, briefly unreadable (EACCES, EMFILE) or hand-edited into invalid JSON, that agent is missing from the snapshot even though its file is still on disk. The healer (#16410, #16455) compares the snapshot against what it has seen and treats the missing name as a removal. It then durably resets default_agent to default, and nothing points it back once the file reads again. Found by review of the release/0.8.0 backport (#16385).

Why it matters

A transient read failure becomes a lasting change to the owner's settings.

Not a goal

  • No change to the scan's (names, stems) return, or to dispatch during a partial scan.
  • No change to healing after a complete scan.

What changed (motivation → approach → change)

  • The scan records whether it read every candidate in a thread-local, _SCAN_STATE.complete. A failed directory listing or a skipped spec sets it to false. The (names, stems) return contract is unchanged, because the scan has other callers and tests that stub it with 2-tuples.
  • refresh_materialized_agents reads the flag on the thread that ran the scan. It publishes the flag as _MATERIALIZED_COMPLETE together with the snapshot it installs, under _MATERIALIZED_AGENTS_LOCK, and heals only when heal_default is set and its own scan was complete.
  • reset_dangling_default_agent takes the installed snapshot, its removals and its completeness in one lock hold (_removal_evidence). It declines when the installed snapshot is partial. It reads all three again inside the locked config write, so a partial snapshot that another refresh installed in between stops the reset. This covers the overlap where a complete refresh's healer would otherwise read a later partial refresh's snapshot.
  • The integration harness clears _MATERIALIZED_COMPLETE between boots and checks it in its residue witness.
  • A non-object JSON file still counts as read: it is not an agent config, so it cannot hide an agent's name.

Backwards compatibility

Compatible. A real removal resets the default on the next complete scan, exactly as before.

Tests

  • New: test/test_config_loader.py::TestDanglingDefaultAgentReset::test_a_scan_that_could_not_read_every_spec_is_not_evidence_of_a_removal. The default's spec is torn: the owner's refresh skips it and does not reset. The spec is repaired, and still nothing changes. The spec is deleted, and the default resets. On origin/main it fails ('default' == 'pkg').
  • New: test_a_partial_snapshot_installed_after_a_complete_scan_is_not_evidence and test_a_partial_snapshot_installed_inside_the_locked_write_stops_it (the overlap case, with the partial refresh landing inside the locked write).
  • Mutations:
    • Removing the refresh's scan_complete gate fails the torn-spec test, and so does removing the flag set in the scan's except.
    • Publishing _MATERIALIZED_COMPLETE = True unconditionally fails both overlap tests.
    • Dropping the in-lock completeness re-check fails the in-lock test.
    • Dropping only the pre-lock check is still caught by the in-lock re-check, as intended.
  • test_config_loader.py, test_agent_execution_catalog.py and test_config_api.py: 720 passed. The wider set, every file referencing the refresh, the healer, dispatch or the snapshot plus test/integration, gives 20775 passed. Its 2 failures are the test_chat_runner_composition_contract.py ones that fail identically on origin/main.
  • The baselined black and subprocess-encoding gates, isort, flake8, mypy, comment-history, agent-SDK boundary and docs-lint gates pass.

Manual verification

N/A: the test drives a real refresh over a real agents directory and config.json.

Related Issues

Part of #16409. Follow-up to #16410 and #16455. Found on the backport #16385.

Pattern harvest

Rule candidate: review-prompt
Pattern: a durable write that acts on "X is absent from this scan" must first establish that the scan was complete. A read that skips failures silently has to report that it skipped.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested a review from a team as a code owner October 3, 2026 12:19
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 3, 2026
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — 🟡 CONCERNS

Design-level review of b126725e9c51d6145a38991f095d769f923cb7ca — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

One spec that never parses, such as a stray backup or a half-written .md, makes every scan partial and silently turns the default-agent healer off for good.

Watch

  • A permanently unreadable spec keeps a dangling default in place. In the diff, except (ValueError, OSError): _SCAN_STATE.complete = False marks the scan partial, and if heal_default and scan_complete: then skips healing. The flag is global: it covers every spec in the directory, not the one whose agent went missing. So an invalid JSON or .md file left in kiro/agents stops the healer on every refresh. If the owner then really deletes the agent named in default_agent, nothing resets it, and nothing is logged to say why. That is the condition fix(agents): reset a default agent only when its template was removed #16410 was built to fix. It is reversible: deleting the bad file brings healing back, but nothing tells the owner to do that. The cost is bounded and the conservative choice is defensible, but it should be visible.
    Clears when: declining to heal because of a partial scan logs the skipped file's path at least once, and the description's "A real removal resets the default on the next complete scan" acknowledges that a persistently bad spec means there is no next complete scan.

[DESIGN-REVIEWED] b126725

Description read

[DESCRIPTION-READ] 0196fa83b37fd3f4460b43a49b620eeaa9025751f4684ae0ac07f201d7028e8d

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of b126725e9c51d6145a38991f095d769f923cb7ca — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The heal_default and scan_complete gate in refresh_materialized_agents does the same job as the installed-snapshot completeness check, so it can go.

Not justified as shipped

  • Item 3 — duplicate of _removal_evidence() completeness check (src/kiro_crew/config/loader.py): when a refresh reaches its heal, it has just installed its own snapshot with _MATERIALIZED_COMPLETE = scan_complete. reset_dangling_default_agent already refuses a partial installed snapshot (if not names or not removed or not complete), and it checks again inside the lock. So the refresh-level gate can only skip a heal that a later complete snapshot would allow. The description says "Removing the refresh's scan_complete gate fails the torn-spec test". By my reading it can't: refresh 2 installs COMPLETE=False, and the healer declines on that alone.
    Subtraction: change if heal_default and scan_complete: back to if heal_default: and drop the local scan_complete read. The value still goes into _MATERIALIZED_COMPLETE through the thread-local.
    Clears when: a test fails on this branch when only and scan_complete is removed, or the gate is deleted.

What this change ships

Inventory (5 items) — 4 justified

Intent: stop a passing read failure of the default's spec from permanently resetting the owner's default_agent. FIX.

  1. A refresh whose scan skipped an unreadable or unparseable spec no longer resets the default agent — justified
  2. The scan reports whether it was complete through a thread-local, so its (names, stems) return stays the same — justified
  3. Refresh-level gate stops healing when this refresh's own scan was partial — duplicate of _removal_evidence completeness check
  4. The healer declines when the installed snapshot is partial, and checks this again inside the locked config write — justified
  5. The integration harness resets _MATERIALIZED_COMPLETE between boots and checks it in its residue witness — justified

[FIRST-PRINCIPLES-REVIEWED] b126725

Description read

[DESCRIPTION-READ] 4ce444edde7b766875e583781c3a4ef417e8d44ba61c4ac8fedd096dc9f9b8ba

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed b126725e9c51d6145a38991f095d769f923cb7ca — this comment is updated in place on each push.

Review details

One advisory finding; nothing blocks the merge.

FINDING — src/kiro_crew/config/loader.py:5585 — a plain markdown file with no --- frontmatter in the agents directory (a README or a shared prompt fragment, which opens_frontmatter_fence documents as a normal non-spec) is still returned by iter_agent_spec_files. parse_markdown_spec then raises ValueError, and the except (ValueError, OSError): branch sets _SCAN_STATE.complete = False. That happens on every scan, so heal_default and scan_complete is always false and the not complete check always holds. default_agent therefore never gets reset after its template is really removed. → Fix: in that branch, treat a .md file whose text fails opens_frontmatter_fence as not-a-spec and skip it without marking the scan partial, the way the non-dict JSON branch is skipped (as connections/ownership.py:241 already does).

[OPUS-REVIEWED] b126725

Verdict parsed from the review's SHA-scoped output markers for commit b126725e9c51d6145a38991f095d769f923cb7ca.

False positive or not applicable? A repository writer can comment:
/ai-review override fable b126725e9c51d6145a38991f095d769f923cb7ca: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of b126725e9c51d6145a38991f095d769f923cb7ca and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] b126725

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b126725e9c51d6145a38991f095d769f923cb7ca: <one-sentence reason>

@bolichen97
bolichen97 force-pushed the fix/heal-only-after-complete-scan branch from 3194275 to 087cd29 Compare October 3, 2026 12:39
The agents-directory scan skips a spec it cannot read or parse, so a spec
caught mid-write or briefly unreadable during the owner's catalog fetch left
its agent out of that snapshot while the file was still on disk. The healer
took that absence as a removal and durably reset default_agent to "default".

The scan now records, in thread-local state beside its unchanged
(names, stems) return, whether it read every candidate, and the refresh
publishes that flag with the snapshot it installs, under the same lock. The
healer reads the installed snapshot, its removals and its completeness in one
lock hold, and re-reads them inside the locked config write, so a partial
snapshot another refresh installed in between is never taken as evidence that
an agent was removed. A real removal still resets on the next complete scan.

Part of #16409
@bolichen97
bolichen97 force-pushed the fix/heal-only-after-complete-scan branch from 087cd29 to b126725 Compare October 3, 2026 13:19
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 3, 2026
@bolichen97
bolichen97 merged commit 4576a29 into main Oct 3, 2026
135 of 139 checks passed
@bolichen97
bolichen97 deleted the fix/heal-only-after-complete-scan branch October 3, 2026 14:27
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Oct 3, 2026
bolichen97 added a commit that referenced this pull request Oct 3, 2026
…/0.8.0)

Backport of #16323, #16410, #16455, #16512 to release/0.8.0, cherry-picked in main's merge order.

* fix(agents): let an installed agent become the default agent (#16323)
  (cherry picked from commit 5c6268a)
  Backport adaptation: test/integration/conftest.py conflicted only on context; the
  source's added reset lines (config_loader import, docstring bullet, the three
  _MATERIALIZED_* resets and checks) were applied, nothing else. Every other file
  applied cleanly.

* fix(agents): reset a default agent only when its template was removed (#16410)
  (cherry picked from commit f16fa23)
  Backport adaptation: test/integration/conftest.py conflicted on context only (a
  main-only updates-loop entry next to it); only this change's _MATERIALIZED_SEEN
  lines were applied. Every other file applied cleanly.

* fix(agents): harden the default-agent healer and the file-name map (#16455)
  (cherry picked from commit 4a11ed2)
* fix(agents): heal a default agent only after a scan that read every spec (#16512)
  (cherry picked from commit 4576a29)
bolichen97 added a commit that referenced this pull request Oct 3, 2026
…/0.8.0) (#16385)

* fix(catalog): restore companion runtime_policy on member rows (#12686)

Co-authored-by: Premshay <28099628+Premshay@users.noreply.github.com>
(cherry picked from commit fa1b47d)

* fix(agents): let an installed agent become the default agent (release/0.8.0)

Backport of #16323, #16410, #16455, #16512 to release/0.8.0, cherry-picked in main's merge order.

* fix(agents): let an installed agent become the default agent (#16323)
  (cherry picked from commit 5c6268a)
  Backport adaptation: test/integration/conftest.py conflicted only on context; the
  source's added reset lines (config_loader import, docstring bullet, the three
  _MATERIALIZED_* resets and checks) were applied, nothing else. Every other file
  applied cleanly.

* fix(agents): reset a default agent only when its template was removed (#16410)
  (cherry picked from commit f16fa23)
  Backport adaptation: test/integration/conftest.py conflicted on context only (a
  main-only updates-loop entry next to it); only this change's _MATERIALIZED_SEEN
  lines were applied. Every other file applied cleanly.

* fix(agents): harden the default-agent healer and the file-name map (#16455)
  (cherry picked from commit 4a11ed2)
* fix(agents): heal a default agent only after a scan that read every spec (#16512)
  (cherry picked from commit 4576a29)

---------

Co-authored-by: Premshay <premshay@gmail.com>
Co-authored-by: Premshay <28099628+Premshay@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant