Skip to content

fix(config): let a user model pick outrank a template's pinned model - #36045

Merged
mbriones98 merged 1 commit into
mainfrom
fix/template-model-override-P551364879
Nov 17, 2026
Merged

mbriones98 merged 1 commit into
mainfrom
fix/template-model-override-P551364879

Conversation

@mbriones98

@mbriones98 mbriones98 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A user can choose the model an agent template's sessions start on, instead of always getting the model the template's own file pins.

Sessions started from an installed agent template (reported for a packaged template that pins claude-opus-4.8) always run that pin. The user set agents.<template-name>.model to another model in config.json, and nothing changed: a template launch deliberately reads no crew row, even one that shares the template's name. There is no other per-template model setting.

Why it matters

Anyone running a packaged template whose shipped pin is not the model they want (cost, entitlement, a team standard) has no setting that sticks. The only workarounds are picking the model in every new chat, or turning the template into a crew, which also changes its workspace and memory.

Not a goal

  • A dashboard control for the new setting. This PR adds the config key, its resolver tier and its docs; the Settings/Templates UI is a separate change that needs a placement decision.
  • Changing the template-launch isolation: a template session still takes no field, model included, from a same-named crew row or from the default row.
  • Changing how the Templates tab's model field writes the template's own file.

What changed (motivation → approach → change)

A template launch resolves its model from the template file's pin and nothing above it except a crew's own pin and the session's own pick. Reading the model from a same-named crew row would break the template/crew isolation that three existing tests pin, and the dashboard never creates such rows for templates (the composer's "set as default" row is withheld for template sessions). So the user's choice gets its own key: templates.<name>.model.

  • New config section templates (TemplateSettingsConfig, one model field), loaded totally (a malformed table or entry reads as absent), saved by to_dict, listed as a known record-map section, and in config-baseline.json.
  • The template tier of resolve_effective_model (_template_model) reads templates.<name>.model first, then the template pin. Order: session pick > crew pin > template pick > template pin > apps.<name>.model > global. Empty or auto inherits. A pick the active harness cannot claim defers, as every tier does.
  • The same tier is read by resolve_app_spawn_model, by the provider factory's fallback (acp_effective_model), and by kirocrew doctor's model tier list, so the chip, the session and doctor agree.
  • Live edits: the session manager subscribes to templates and treats it as a factory input (session_parts/config_apply.py), so an edit reaches new sessions and new warm processes without a restart.
  • Warm claims: the post-claim check in session_allocation_parts/get_or_create.py compares the requested model with the model the pooled process launched on (the pool template's pick when the harness claims it, else the spec pin, read from the config in force when the provider was dequeued), so a request for the spec pin switches the process and a request for the pick does not. A config apply empties the warm pool in the same step that installs the new config (session_lifecycle_parts/fleet.py), so no provider launched under the previous config is claimable afterwards.
  • kirocrew doctor scopes the template pick to the active harness the way the resolver does, so a pick the harness cannot claim is never shown as the deciding tier.
  • Spec: model-selection.md documents the key; the _session_model docstring in session_parts/policy.py names the new tier.

Backwards compatibility

Compatible: the key is new and absent by default; with it unset every session resolves exactly as on main.

Removes nothing: the diff only adds a config section and a tier above the template pin.

Tests

  • test/test_template_model_setting.py: the table loads, round-trips, and reads empty when absent or malformed; a template launch runs the user's pick over the spec pin and over the global; no pick, "" or auto keeps the spec pin; another template's pick does not apply; an out-of-scope pick falls back to the spec pin; the factory fallback reads the pick; a caller's own model still wins; a same-named crew row and the default row still do not reach a template launch; a crew's own pin outranks its template's pick; a crew that pins nothing runs it; an app spawn defers to the template's pick and still gets the app setting without one; a templates.<name>.model edit rebuilds the factory with no restart; a warm claim asking for the spec pin switches off the pick, one asking for the pick sends no switch, and a template edit applied mid-claim does not change which model the claim compares against.
  • test/test_cli_doctor_config_sections.py: doctor marks the template pick as the deciding tier and agrees with the resolver; a pick the harness cannot claim is not marked.
  • test/test_session_config_hot_reload.py: the subscription and factory-path pins include templates; a claim made while a config apply discards old providers finds none left, and a drain cancelled mid-discard hard-kills the providers it had taken.
  • Existing isolation tests are unchanged and pass (test/test_agent_default_model.py, test/test_app_model_setting.py, test/test_session_model_ladder.py).

Proof: not provable -- other: prove.py reports BASELINE_RED on this host because four TestProjectSectionAndAuthRow tests in the changed doctor test file fail on main here too. On a local commit with the same production code, before the doctor test was added, prove.py returned PROVEN for the five template-launch tests above; the doctor test was checked by hand against main's model_section.py and fails there.

Manual verification

N/A — unit coverage drives a real SessionManager and the real config loader; the change has no UI.

Related Issues

Closes #36044

Sibling sweep

Swept: git grep -nE "_resolve_named_agent_model(" -- src/
Sibling: src/kiro_crew/config/loader.py -- changed here
Sibling: src/kiro_crew/config/loader_config.py -- changed here
Sibling: src/kiro_crew/cli_doctor_parts/model_section.py -- changed here
Sibling: src/kiro_crew/agent_parts/spec_lookup.py -- not affected -- docstring mention only, no model read

Pattern harvest

Not generalizable: the gap was a missing user setting tier for one resolver, not a repeated coding pattern.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@mbriones98
mbriones98 requested review from a team and KiroCrewCaptain as code owners November 12, 2026 19:31
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 12, 2026
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision bfafd9a8f124a37b7019a2d5cd107fcae5d36322 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of bfafd9a8f124a37b7019a2d5cd107fcae5d36322 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design blockers: the setting is new and opt-in, it is a dedicated key rather than a same-named crew row, so template/crew isolation holds, and it can be reverted cleanly.

Notes

  • Once templates.<name>.model is set, the Templates tab's model field (which writes the template's own file) silently stops having any effect. The chip and doctor agree with the resolver, but the Templates editor does not. The follow-up UI PR should show or clear the overriding pick there.
  • The fleet.py drain-ordering fix (_take_all/_discard_taken) closes a warm-claim race that templates hot-reload has in common with every existing factory path. It is a separate concern from the new key and would review more cleanly as its own commit.

[DESIGN-REVIEWED] bfafd9a

Description read

[DESCRIPTION-READ] 9af1e5a71f02e59f857a55b37504adacbebe8e1dd9cc17e0afb8030c625523bb

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of bfafd9a8f124a37b7019a2d5cd107fcae5d36322 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks: the new key is absent by default, it has real consumers, and packaged templates are read-only in the Templates tab, so no existing setting does this job.

Notes

  • Item 1 — inherited framing: the title is fix(config), yet the description says "the diff only adds a config section and a tier above the template pin". This is an addition (a new user setting) presented as a fix.

What this change ships

Inventory (6 items) — 6 justified

Intent: let a user choose the model sessions on an installed (including packaged) agent template start on, overriding the model the template's file pins. Labelled a FIX, but it is an ADDITION.

  1. New config.json key templates.<name>.model overrides the template file's pinned model — justified
  2. A crew that pins nothing, and app spawns of the template, now run the user's template pick — justified
  3. kirocrew doctor lists the template pick as a deciding tier — justified
  4. Editing the key reaches new sessions and warm processes without a restart — justified
  5. A warm-pool claim compares against the model the pooled process actually launched on — justified
  6. A config apply empties the warm pool before discarding, and a cancelled drain hard-kills the providers it had taken — justified

[FIRST-PRINCIPLES-REVIEWED] bfafd9a

Description read

[DESCRIPTION-READ] e1afe5a0d5d08673fd0e801c76fcb71109e7bd3b9f4c826058909dc8d7e836a4

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed bfafd9a8f124a37b7019a2d5cd107fcae5d36322 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] bfafd9a

Verdict parsed from the review's SHA-scoped output markers for commit bfafd9a8f124a37b7019a2d5cd107fcae5d36322.

False positive or not applicable? A repository writer can comment:
/ai-review override fable bfafd9a8f124a37b7019a2d5cd107fcae5d36322 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of bfafd9a8f124a37b7019a2d5cd107fcae5d36322 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] bfafd9a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt bfafd9a8f124a37b7019a2d5cd107fcae5d36322 false-positive: <reason, 20 to 500 characters>

@KiroCrewCaptain

KiroCrewCaptain commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider thinks the change is probably fine but estimates a 15% chance that this change introduces a defect, above the 12% limit for approving on its own.

Adds a templates..model setting that ranks above a template file's pin and below a crew pin and the session's own pick. Every model-choosing path reads it the same way: the session resolver, the factory fallback (acp_effective_model, also covering the warm pool and the subagent report), app spawns, the dashboard chip, the inherited-default check and doctor. Template/crew isolation is kept. Live edits rebuild the factory. A config apply empties the warm pool in the same step it installs the new config, so a claim always compares against the config its process launched under. The setting is absent by default, the loader treats a malformed table as absent, and tests cover each changed path. No blocking defects found.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit ab71f6e3b6 · 688 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 12, 2026
@mbriones98
mbriones98 force-pushed the fix/template-model-override-P551364879 branch from 6c11c5f to 5ab6693 Compare November 12, 2026 23:23
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 12, 2026
@mbriones98
mbriones98 force-pushed the fix/template-model-override-P551364879 branch from 5ab6693 to fa4b45b Compare November 13, 2026 00:02
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 13, 2026
@mbriones98
mbriones98 force-pushed the fix/template-model-override-P551364879 branch from fa4b45b to 16e5845 Compare November 13, 2026 18:05
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 13, 2026
A session started from an agent template ran the model the template's
spec pins, and no user setting changed it: a template launch reads no
crew row. The new config key templates.<name>.model is the user's pick
for a template and ranks above its spec pin. Order: session pick > crew
pin > template pick > template spec pin > apps.<name>.model > global.
Empty or auto keeps the spec pin; a pick the active harness cannot
claim defers. Template launches still read no crew row.

The same tier feeds resolve_app_spawn_model, the provider factory's
fallback (acp_effective_model) and kirocrew doctor's tier list, which
scopes the pick to the active harness the way the resolver does.

The session manager subscribes to the templates section and treats it
as a factory input, so an edit reaches new sessions and new warm
processes without a restart. A warm claim compares the requested model
with the model the pooled process launched on, read from the config in
force when the provider was dequeued: the pool template's pick when the
harness claims it, else the spec pin. A config apply now empties the
warm pool in the same step that installs the new config, before the
first discard awaits, so no provider launched under the previous config
can be claimed once the new one is in force.

Fixes #36044
@mbriones98
mbriones98 force-pushed the fix/template-model-override-P551364879 branch from ab71f6e to bfafd9a Compare November 17, 2026 18:40
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Nov 17, 2026
@mbriones98
mbriones98 enabled auto-merge (squash) November 17, 2026 20:26

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. On this head, the review traced the new templates.<name>.model pick through every reader: the resolver, app spawns, the factory fallback, the warm-pool claim and doctor. All of them go through one _template_model tier, so the model chip, the running session and doctor agree. A pick the harness cannot run falls back to the template pin, and with the key unset, resolution is the same as on main. Reverting the production hunks makes 16 of the new tests fail, and the tests in the three changed test files passed on repeat runs.

Done well: the new section reuses the existing apps.<name>.model shape end to end. The fleet.py drain change also closes a real race, where a stale warm provider could be claimed between discards.

Optional:

  • docs/system-specs/modules/session.md still lists the subscribe sections and _FACTORY_CONFIG_PATHS without templates, and its get_or_create ladder does not mention the new tier. The atomic drain and the hard-kill on a cancelled drain are not described there either.
  • No test covers the drain rewrite in reload_provider_factory (fleet.py:197), so reverting it makes no test fail. You could parametrize the two new drain tests over both refresh_defaults and reload_provider_factory.
  • _take_all repeats owner.drain_warm_pool(), which already empties the queue without awaiting. Calling it would remove the copy.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Nov 17, 2026
@mbriones98
mbriones98 merged commit f6a144e into main Nov 17, 2026
107 checks passed
@mbriones98
mbriones98 deleted the fix/template-model-override-P551364879 branch November 17, 2026 21:26
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Template sessions always run the template's pinned model; no user setting overrides it

2 participants