Repository navigation
fix(acp): record a withheld kiro startup pin as refused - #19024
Conversation
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Check the dedicated client: confirm that a later live What this change shipsInventory (5 items) — 5 justifiedIntent: when a kiro session runs on the backend default because the account does not serve the pinned model, the cron "Model unavailable" note and the usage row should say so. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 1d972a7 Description read[DESCRIPTION-READ] 25115b889e87f9238aa1ba6981a24c79bed796fe19b4aa27ed2274720bf8d42b That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the kiro [DESIGN-REVIEWED] 1d972a7 Description read[DESCRIPTION-READ] 7751c103721fcdaa748d1cd64ed83c0fa31ac0056cbeecd6bc6e0bceea17f1ad That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
When a kiro session starts with a pinned model the account does not serve, both start paths withhold the pin and leave the session on the backend default, with a warning in the log. Neither recorded the pin in model_pin_refused, the field every caller that bills or labels a turn by the pin reads through provider_model_pin_refused. The cron acquire step therefore reported no downgrade: the result carried no "Model unavailable" note and the usage row billed the pinned model, which never ran. AcpClient._apply_startup_model (dedicated runtime) now sets model_pin_refused to the pin before it records the session as running the default, and the kiro start in providers/acp.py (shared runtime) sets it on the session handle in the same withhold branch. The field comments, the provider_model_pin_refused docstring and the model_pin_refused passage of the model-selection spec name the withhold.
39e5040 to
1d972a7
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
This PR records a kiro startup pin that the account does not serve in model_pin_refused, on both start paths (AcpClient._apply_startup_model and the shared-runtime kiro start in providers/acp.py), so the cron result gets its "Model unavailable" note and the usage row stops billing a model that never ran. That is the root cause: only the config-option branch wrote the field, and the entitlement withhold that reaches the same outcome did not. I read the full diff at head 1d972a7 and checked every writer and reader of the field: a later explicit set_model still resets it first on both the client and the handle, so a later pick reads only its own verdict; after a withhold _model is DEFAULT_MODEL, so a re-run of the startup path returns early and leaves no stale refusal for a pin that later ran; and the foreign-scope branch is left alone, as stated. The new tests fail on main and pass on the branch, with controls for a served pin. It changes no entitlement check, only records its outcome. — crew-pr-reviewer (deep re-check)
Problem / Motivation
Goal: when a kiro session withholds a startup model pin that the account does not serve, the pin is recorded in
model_pin_refused, so the cron "Model unavailable" note and the usage row reflect the model that actually ran.AcpClient._apply_startup_model(src/kiro_crew/acp/client.py). The shared runtime does it in the kiro start insrc/kiro_crew/providers/acp.py.model_pin_refusedthroughprovider_model_pin_refused(src/kiro_crew/llm_helpers.py), so they saw nothing.src/kiro_crew/slack/gateway.pyreturnsbool(job.model) and provider_model_pin_refused(client)as its "downgraded" flag. With the field empty, the result gets no "Model unavailable" note and the usage row bills the pinned model.Why it matters
A user reads a cron result as the pinned model's work, and the cost report bills a model that never ran. kiro is the default backend, so this is the common case of the problem that PR #14971 fixed for config-option backends. That PR listed this path under its Not a goal ("A pin dropped before the push because the advertised list lacks it").
Not a goal
src/kiro_crew/subagent_manager/run.pyandsrc/kiro_crew/workflows/agent_exec.pybill the explicit pin and read neithermodel_pin_refusednormodel_pin_partial. PR #14971 names them under its Not a goal, and they stay there.model_scopebranch on both start paths). It is scoped out on purpose and without a warning, because it answers "which harness was this picked in", not "does the account serve it". Whether a cron pin picked under another harness should read as a downgrade is a separate decision.chat_handlers._try_live_model_switch) carry the change.What changed (motivation → approach → change)
model_pin_refusedstays"".AcpClient._apply_startup_modelwrotemodel_pin_refused. The kiro entitlement withhold in the same function, and the matching withhold in the shared-runtime start, logged a warning and returned.src/kiro_crew/acp/client.py: the withhold branch of_apply_startup_modelsetsself.model_pin_refused = self._modelbefore it records the session as runningDEFAULT_MODEL. The field's comment names both writers.src/kiro_crew/providers/acp.py: the shared-runtime kiro start setshandle.model_pin_refused = configured_modelin its withhold branch. A laterAcpSessionHandle.set_modelcall still resets the field first, so an explicit pick reads only its own verdict.AcpSessionHandle.model_pin_refusedcomment (src/kiro_crew/acp/session_handle.py), theprovider_model_pin_refuseddocstring and themodel_pin_refusedpassage ofdocs/system-specs/common/model-selection.mdname the withhold.job.modelat the cron call site. The field is the record both start paths and both readers already share. A pin folded to its advertised spelling (openrouter::z-ai/glm-5.3-flashruns asz-ai/glm-5.3-flash) is served under a different id than the pin, so a string comparison would report a downgrade that did not happen.AcpModelUnavailableat startup. The withhold is deliberate: the pin was not chosen for this turn, and the cron path'sexceptretry would cold-start a second session.Backwards compatibility
Compatible: only the withhold branch on kiro changes, and only by setting an existing field that every reader already treats as "the pin never ran". A pin the account serves records nothing, as before.
Removes nothing: the change adds two assignments, comment text and one spec paragraph.
Security: no redaction, auth or sandbox rule is touched; the entitlement check that withholds the pin is unchanged, and the change only records its outcome.
Tests
Added:
test/test_acp_client.py::TestModelEntitlementPreflight::test_startup_withhold_records_the_pin_as_refused: the dedicated client records the withheld pin.test/test_acp_provider.py::TestStartKiroRuntimeModelEntitlement::test_a_withheld_configured_model_is_recorded_as_refused: the shared-runtime start records it on the handle and never callsset_model.test/test_acp_provider.py::TestStartKiroRuntimeModelEntitlement::test_an_applied_configured_model_records_no_refusal(control): a pin the account serves is applied and records nothing.test/test_acp_client.py::TestModelEntitlementPreflight::test_startup_still_applies_a_usable_model(control) gainsassert client.model_pin_refused == ""._runstarts withmodel_pin_refused = "", as a realAcpSessionHandledoes.Before the fix, on origin/main 7bb9d57, this branch's base (both test files copied into a clean checkout,
-k "TestModelEntitlementPreflight or TestStartKiroRuntimeModelEntitlement" -n 0, 3 of 3 serial runs). Each failure is a withheld pin left out ofmodel_pin_refused, on the dedicated client and on the shared-runtime handle:After the fix: the same selection gives 27 passed in 3 of 3 serial runs. Every other test file that names
model_pin_refused,_apply_startup_model,_start_kiro_runtimeorprovider_model_pin_refused(21 files, includingtest/test_cron_mirror_backend_model_pin.py, the cron model-pin tests that useprovider_model_pin_refused) gives 1637 passed, 5 skipped, 1 failed.test/test_acp_client.pyruns only through the selection above, because its other classes use the real home directory. The failure istest/test_model_served_default.py::TestClientInheritExits::test_withheld_pin_still_lands_on_a_served_model, and this change does not cause it. Run alone against upstream main at7bb9d57617, this branch's base, it fails the same way: it sends the pinnedclaude-opus-4.8where the test expects the served defaultgpt-5.6-sol. mypy, flake8, isort and black are clean on the changed files.scripts/check_black_formatting.py,scripts/docs-lint.shandscripts/check_comment_history.pywithCOMMENT_HISTORY_BASE_REF=origin/mainpass.Manual verification
N/A: the change sets a field on two start paths. The tests drive both paths with the advertised-model shapes the backends report.
Related Issues
None found for this path. Searched PRs and issues for
model_pin_refused,pin withhold,not available to this account,model downgrade,served model usage,withheld pin. Related: PR #14971 (config-option backends), PR #18248 (records an explicitset_modelrefusal on the shared runtime).Pattern harvest
Rule candidate: review-prompt
Pattern: an outcome record ("this pin never ran") is written on one branch that reaches the outcome and not on a sibling branch that reaches the same outcome.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)docs/system-specs/common/model-selection.md