Repository navigation
fix: warn when a bot's own file sets a model no session reads - #39466
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS This only adds a log line on the one read every worker start already makes. Nothing changes in the spec, the session model or persisted data, and it reverts cleanly. Notes
[DESIGN-REVIEWED] 9c02018 Description read[DESCRIPTION-READ] 73dd327a7f472ee2e6bf050bcbc1814459a0135628ecd45bb07dbe9a5e8b6c6e That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The fix logs a warning at the one read that drops the key, and its defect is tracked in a linked issue (#38165) with a test that fails on base. Notes
What this change shipsInventory (3 items) — 2 justifiedIntent: a Bot owner learns that the
[FIRST-PRINCIPLES-REVIEWED] 9c02018 Description read[DESCRIPTION-READ] ffdd241f3c5d00c38d8aa31d5a671184d94626a1da087f996807b616a5bb3910 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
5262ad7 to
e026e26
Compare
|
Rebased: main moved in docs/system-specs/modules/ (this PR's spec file, plus 360 files in the directories this PR touches) since this head went green (7505181 -> dc531c1); CI re-running. The same push fixes the [operator: bolichen97] |
A Bot's own file under ~/.kiro/mates/agents/ may carry a model key (older builds copied the template into it, and the template and capability edits can still write it), but no session reads that key on any backend: a Bot's chat runs on its Model setting and its workers on its Worker model setting. A model named there silently never ran. resolve_worker_spec, the one reader of that file on every worker start, now logs one warning per process per Bot and value when the file's model is neither empty nor auto, naming the Bot, the ignored model and the settings that decide it. The worker spec is unchanged; nothing is refused, written or re-resolved. Closes #38165 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e026e26 to
9c02018
Compare
|
Rebased: main moved in this PR's spec file ( The previous head's only reds were main's Windows flakes, each fixed on main after that head's base: [operator: bolichen97] |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
I reviewed head 9c02018. When a Bot's own file names a model other than empty or auto, the change logs one warning per process per Bot and value. It does not change the resolved worker spec or the session model. I confirmed that neither the session model resolver nor the KAS projection reads the model key from a Bot's own file, so the warning is accurate. The new parametrized test fails on the base and passes here, and the rest of the mate worker suite passes. Nothing blocks.
Problem / Motivation
Goal: A Bot whose own file sets a model says so in the log, so a model that never runs is no longer silent.
A Bot's own file under
~/.kiro/mates/agents/<id>.jsoncan carry amodelkey: older builds copied the source template into it verbatim, and the template edit and the Bot capability edit can still write it. No session reads that key on any backend (rfc-mate-owned-agent-config§1/§5): a Bot's chat runs on its Model setting (agents.<name>.model) and its workers on its Worker model setting (agents.<name>.worker_model; empty follows Model), and with neither set, on the default model. Nothing said so. The reporter's three Bots named a model in their own files and every turn ran on a different, 2.2x more expensive model, with no log line and no notice.Why it matters
Anyone with a Bot file from an older build, or one written by the template or capability edit, believes their Bot runs on the model named in that file. It does not, and they find out only from the bill. A log line naming the Bot, the ignored value and the settings that decide the model lets them see and fix it.
Not a goal
modelthere: the template PATCH, the Bot capability edit).model, or re-resolving the session model from it.set_modelpath.What changed (motivation → approach → change)
A Bot whose own file names a model now gets one warning in the log saying that model is not used, and which settings are.
modelin a Bot's own file never runs, and nothing says so.mate_worker.resolve_worker_specreads the file on every worker start, on every backend, and dropped the key without a word.modelother than empty orautologs one WARNING per process per Bot and value. It names the Bot, the file and the ignored model, and points to the Bot's Model and Worker model settings. The helper isnote_unread_file_modelinsrc/kiro_crew/crewmate_harness.py, kept beside that module's other once-per-process notes and under the same lock, somate_workerstays stateless.docs/system-specs/modules/crew-mode.mdand themate_workerdocstring name the warning where they already saymodelis ignored.Backwards compatibility
Compatible: the change only adds a log record. The resolved worker spec, the session model and every refusal are the same as before.
Removes nothing: no path, key, record, validator or public function is hidden, deleted or narrowed.
Tests
test/test_mate_worker.py::TestMateFileForMember::test_a_model_in_the_bots_own_file_is_logged_once_as_unread, parametrized:model: "example-model-9"logs exactly one WARNING across two resolves, naming the Bot and that value."auto", or with nomodel, logs none.modeland both resolves are equal.tmp_pathhomes.assert 0 == 1) and passes with the fix.-n0and 10 at-n 4 --dist loadgroupbeside two sibling test files all passed, and every test intest/test_mate_worker.pypassed in a shuffled order (seed 16726).Proof: PROVEN test/test_mate_worker.py::TestMateFileForMember::test_a_model_in_the_bots_own_file_is_logged_once_as_unread[example-model-9-1] (head 9c02018)
Manual verification
Run on Linux (Ubuntu 24.04.5 LTS, x86_64) against the real code path, in an isolated home (
HOME=$(mktemp -d),KIRO_HOME=$HOME/.kiro,KIROCREW_HOME=$HOME/.kiro/crew). A realconfig.jsonbinds Botpixelto its own file$KIRO_HOME/mates/agents/pixel.json, which sets"model": "example-model-9". The script resolves the worker spec twice throughresolve_worker_specand readsresolve_effective_model(cfg, "pixel"). No live model is needed: the defect is the missing log line, not a model call.Before the fix:
After the fix (same script, plus a happy-path step that rewrites the file with
"model": "auto"):Real-machine verification
Machine: EC2 m7i.2xlarge, Ubuntu 24.04.5 LTS x86_64, 8 vCPU / 30.8 GiB
Before (main 983fd61, with only this PR's test file applied):
python3 -m pytest -n0 -p no:cacheprovider -q <the three test_a_model_in_the_bots_own_file_is_logged_once_as_unread cases>->1 failed, 2 passed;[example-model-9-1]fails onassert 0 == 1After (head 9c02018): the same command ->
3 passedCleanup: the throwaway base worktree and the smoke's isolated home were removed; the host is a reused disposable one with no inbound rules and no key pair, terminated by its idle reaper rather than per PR
Related Issues
Closes #38165
Sibling sweep
Swept: git grep -n "load_agent_spec(.*mate_agents_dir|load_agent_spec(mate_dir" -- src/
Sibling: src/kiro_crew/acp/harness/kas.py:297 -- not affected -- resolves a Bot's agent on KAS and never projects
model; that Bot's workers start through resolve_worker_spec, which logsSibling: src/kiro_crew/mate_worker.py:1203 -- not affected -- this read and the one at :1247 take only tool limits and spec policy fields for a refusal detail or notice, not a worker start; the one at :1421 loads the shared worker base and then resolves the Bot through resolve_worker_spec, which logs
Sibling: src/kiro_crew/mate_capabilities.py:243 -- not affected -- this read and the one at :486 load the shared worker base file, not a Bot's own file
Sibling: src/kiro_crew/mcp_gateway/mate_overlays.py:558 -- not affected -- loads the shared worker base file, not a Bot's own file
Pattern harvest
Not generalizable: the Bot file's
modelis the one reserved placeholder key that a writer can still set and no reader consumes. The class of a key that is written but never read has no syntactic signature that a lint rule could catch.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.