Repository navigation
Conversation
732d739 to
68fb2dc
Compare
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that an OpenCode [DESIGN-REVIEWED] 4c55b7a Description read[DESCRIPTION-READ] 1a933d4cd9ae878f8843994efa92c294a0db479561996489b57983f37df83c36 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Before merge, check that no backend lists a model id starting with Not justified as shipped
What this change shipsInventory (6 items) — 5 justifiedIntent: let OpenCode users save the
[FIRST-PRINCIPLES-REVIEWED] 4c55b7a Description read[DESCRIPTION-READ] dd0c31b552cde3ffb6e5bbd0ab85c908b71c2269c2d73e3ab4b5e7ce3e981604 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings block this merge. One advisory: on the wake-judge pin, a FINDING — src/kiro_crew/dashboard/handlers/core.py:3024 — [OPUS-REVIEWED] 4c55b7a |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
|
On the two "before merge" checks from the current-head reviews (
Separately, CI: |
…offers kirodotdev#16417 let the Decisions tier pickers save the provider-qualified provider/model ids OpenCode advertises. The role-model, throttle-fallback, refusal-fallback and wake-judge pickers list the same advertised names (useAvailableModelsQuery) and validate through the same _validate_role_model, but their PATCH gate kept the slash-free pattern, so picking one of the offered models answered "Could not save this setting" and reverted. Load-time config already accepts those ids. Give every validated model pin one grammar, _MODEL_PIN_PATTERN (the tier pattern, byte-identical), so a picker cannot offer an id a sibling pin refuses. agent.model is unchanged: it has no entitlement check behind its grammar. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
68fb2dc to
4c55b7a
Compare
|
Rebased onto latest
|
|
Thank you @leonlaiyc for this fix. Because the fork workflow needs maintainer approval to run its checks, I have opened #18399 on the main repo with your change (you are credited as co-author) so it can go through CI directly. Closing this in favour of #18399 — all the credit for the fix is yours. 🙏 |
Problem / Motivation
Goal: Every Settings model picker can save any model it offers, including the provider-qualified
provider/modelids OpenCode advertises.#16417 fixed this for the three Decisions (Jev) tier pickers. Five sibling pickers still hit it on main @
63796306f: the two per-role model pins, the throttle-fallback model, the refusal-fallback model and the wake-judge model.agent.role_models.backgroundagent.role_models.subagentagent.fallback_modelagent.refusal_fallback_modeldecisions.nudge_wake.llm_modelEvery one of these pickers lists the same names as the tier pickers:
useAvailableModelsQuery()→m.name, inwebsite/src/pages/settings/ChatPanel.tsx(roleModelOptions,fallbackModelOptions) andDecisionsCard.tsx(judgeModelOptions). TheirPATCH /api/config/kirocrewgate still uses^[A-Za-z0-9._\-\[\]]*$, so on OpenCode picking an offered model answers "Could not save this setting" and the value reverts. The First Principles review of #16417 named these keys as the open siblings ("a follow-up issue tracks the same "/" rejection on agent.model, agent.role_models.* and decisions.nudge_wake.llm_model").Why it matters
An OpenCode user cannot choose a cheaper model for background or sub-agent work, cannot set the model that a throttled or refused turn falls back to, and cannot pick the model that judges loop wakes. Each picker offers exactly those models and then refuses to keep them. Hand-editing
config.jsonworks, because the load path accepts the same ids. So the refusal is only the dashboard gate disagreeing with its own picker.Not a goal
agent.model. It has novalidate_fn, so widening its grammar would accept an unadvertised qualified id with no entitlement check behind it. Its comment also forbids separators on purpose. That is a separate decision.[/]in a qualified id's last segment (provider/model[1m]). The First Principles review of fix(decisions): accept qualified models in Jev tier settings #16417 raised it; the tier pattern is reused byte-for-byte rather than widened again.decisions.model_route_judge.llm_modelfrom open feat(decisions): model.route tier lane on the session provider for hosts without a local model #15560. That key is not on main.What changed (motivation → approach → change)
Symptom: a picker offers an id its own save refuses. Root cause: the grammar was copied per key, and #16417 widened only the tier copy, so the copies drifted. Every one of these keys already shares
_validate_role_model, so only the pattern differed.src/kiro_crew/dashboard/handlers/core.py:_MODEL_PIN_PATTERN, byte-identical to the tier pattern fix(decisions): accept qualified models in Jev tier settings #16417 shipped (checked with==locally). It accepts a bare id, or slash-joined segments that each start with an alphanumeric, plus the empty value. It refusesa//b,/a,a/and shell metacharacters. The PATCH applies it withre.fullmatch, andmax_len: 64is unchanged.nudge_wakecomments already claimed parity with the role pins, and they are accurate again. That answers the other half of fix(decisions): accept qualified models in Jev tier settings #16417's clearing condition: its comments no longer describe a parity that does not exist.Load-time compatibility was checked rather than assumed.
coerce_role_models→normalize_agent_modelkeeps any non-autostring.coerce_fallback_modelandcoerce_refusal_fallback_modelpass an unregistered id throughmodel_registry.to_provider_idunchanged.Specs, same commit:
docs/system-specs/common/model-selection.mdgains a "save gate accepts every id its picker offers" rule beside the existing "Pickers MUST list from/api/models" rule.docs/system-specs/modules/decisions.md's tier paragraph now says the wake judge and the role/fallback pins share the grammar.Backwards compatibility
Breaking: for these five keys, a dashboard save of a value whose FIRST character is
.,_,-,[or]is now refused (400). The old pattern admitted that shape and the tier pattern being reused does not. Every other value the old pattern accepted still matches the new pattern's first alternative, and_validate_role_modelstill judges entitlement for every non-empty, non-autovalue. Writer sweep on origin/main @63796306f:_EDITABLE_CONFIG'spatternis read in exactly one place, the PATCH handler (dashboard/handlers/core.py: lookup at:3089,re.fullmatchat:3142). Nothing re-validates a stored value with it.GET /api/modelsnames plusautoand"". An advertised model id starting with punctuation would be refused here as it already is on the tier pickers.config.jsonis read by the load-path coercers above, which apply no pattern, so an existing stored value of any shape keeps loading and working.Removes nothing: no key, route or function is removed or renamed.
Tests
New
test/test_qualified_model_pins.pyruns the realapi_kirocrew_config_patchhandler against a tempconfig.json(the harness #16417'stest_decisions_config_api.pyuses), parametrized over the five pins:test_a_provider_qualified_picker_model_can_be_saved[*]:example-provider/example-model→ 200 and stored verbatim. Red on main for all five (400), green with the fix.test_a_pin_refuses_a_value_outside_the_model_id_grammar[*]:a; rm -rf /,provider//model,/modelandmodel/→ 400 and nothing stored. Green on main and with the fix (20 cases), so the widening opens nothing else.test_every_model_pin_shares_the_tier_grammar[*]: each pin'spatternequals the tier's, and itsvalidate_fnis the tier's. Red on main; it keeps the copies from drifting apart again.Adjacent, local, with the fix: every test file that names these keys or the PATCH handler (53 files,
-n 6): 5547 passed, 85 skipped, 2 xfailed, 4 failed. The 4 failures aretest_chat_threads.pysymlink tests that fail the same way on unmodified main (OSError: [WinError 1314], because creating a symlink needs elevation on the local Windows box); they never reach the config route. The tests that read the two edited specs (test_decisions_config_api.py,test_decisions_model_route.py,test_model_scope_wire_paths.py) pass: 133.black --check --target-version py310,flake8andisort --check-onlyare clean on the changed files.Manual verification
N/A — unit coverage sufficient: the defect is the PATCH gate's pattern, and the tests drive the real handler with the exact id shape OpenCode advertises. Not run end to end against a live OpenCode session. The picker-side claim (these pickers list
useAvailableModelsQuery()names) is read off the source cited above.scripts/docs-lint.shwas not run (it is slow on this box). The two spec edits are prose inside existing sections, with no new link or file.Related Issues
Follow-up to #16417: the sibling keys its First Principles review named. No separate issue was filed.
Pattern harvest
Rule candidate: review-prompt
Pattern: a validation grammar copied per key, one copy widened for a new id shape while sibling keys fed by the same picker keep the old copy; hoist the grammar to one named constant and pin every consumer to it.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
🤖 Generated with Claude Code