Skip to content

fix(acp): accept qualified model ids - #9054

Closed
Premshay wants to merge 1 commit into
kirodotdev:mainfrom
Premshay:upstream/qualified-model-ids
Closed

Premshay wants to merge 1 commit into
kirodotdev:mainfrom
Premshay:upstream/qualified-model-ids

Conversation

@Premshay

@Premshay Premshay commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Cron and subagent model selections can use the provider-qualified ids (such as gpt-6-astra[high]) that the model picker itself offers.

Provider-qualified effort variants such as gpt-6-astra[high] are rejected when a dashboard cron or a subagent spawn records a model selection, despite being safe model identifiers advertised by kiro-cli's own model list.

Why it matters

Cron and spawn model selections whose ids carry a provider qualifier cannot be saved or applied.

Not a goal

  • Widening MODEL_ID_RE or the AcpRuntime model validation.
  • Accepting arbitrary bracket content: exactly one non-empty trailing qualifier.
  • The hidden-models config gate, which still rejects bracketed ids (follow-up).

What changed (motivation → approach → change)

The shared spawn and cron _MODEL_NAME_RE accepts one optional, nonempty trailing bracket qualifier. Unbalanced, nested, repeated, empty and non-trailing qualifiers remain rejected, as do shell metacharacters. MODEL_ID_RE and ACP runtime validation are unchanged.

The unrelated frontend restyle-baseline edit is removed. The touched model-selection test is Black-clean, so the required formatting ratchet removes its baseline exemption.

Backwards compatibility

Compatible: existing unqualified ids match the new pattern unchanged; the pattern only adds the bracket-qualifier class.

Pattern harvest

Rule candidate: when a provider protocol qualifies a value with a bounded suffix, extend a constrained allowlist rather than bypassing validation.

Tests

Verified locally on the repaired branch:

  • 43 focused model-selection and cron tests passed.
  • Frontend scoped gate: 10,562 tests passed across 348 test files.
  • Black ratchet, isort, flake8 and subprocess-encoding checks passed.
  • Changed validator passes mypy.

Broader backend run previously reported 49,499 passes, 135 failures and 112 skips. All 135 failed identifiers were rerun against clean upstream base 6819b80069d1bf56c4263be5c453d54cad82de34: 134 failed there too; the remaining session-control case passed on both that base and this repair branch. The full backend suite is not claimed green.

Full mypy checked 1,995 source files and reported one Python 3.14 compatibility error at src/kiro_crew/cli.py:448 (asyncio.PidfdChildWatcher). A focused mypy run against the same CLI file on the clean base reproduces that error.

Current-head Windows evidence: shard 1 failed test_a_third_turn_routes_from_the_base_model_again: the third decision-log row was missing after an exclusive file-lock timeout (0.5 seconds). The exact test passes on Linux at this head and clean base d8fa6a527e927504b399246332cd243d7b450e2f; its test and decision-log implementation are byte-identical there. This does not establish a Windows baseline failure or a flaky result. Native Windows reproduction is still needed to identify the lock holder.

The failed PR Hygiene job read an older description missing **Goal:** and ## Not a goal. The current body contains both and passes the repository's description checker.

Manual verification

N/A — focused automated coverage is sufficient.

Related Issues

N/A.

Checklist

  • Existing tests pass and regression coverage is included (Windows failure remains under investigation).
  • Self-review completed; code follows project style guidelines.
  • Documentation updated where applicable.
  • No secrets, credentials, or internal references in the diff.

Contribution License Agreement

N/A — template placeholder; no CLA wording is supplied.

@Premshay
Premshay requested review from a team and patrigao September 6, 2026 16:07
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 6, 2026
@github-actions

github-actions Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed 66014748e933326b2a4dba73bf537c5b52427a8e via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 6601474

@github-actions

github-actions Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — ✅ PASS

Design-level review of 66014748e933326b2a4dba73bf537c5b52427a8e via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify kiro-cli's --list-models really advertises bracket-qualified ids (e.g. [high], [1m]) — the premise the whole widening rests on.

Suggestions

  • Follow-up (separate PR): the hidden-models config gate (src/kiro_crew/dashboard/handlers/files.py:6943, MODEL_ID_RE) now rejects hiding an id the picker and cron accept — same user-visible dead end this PR fixes, one gate over.

[DESIGN-REVIEWED] 6601474

Description read

[DESCRIPTION-READ] 1c21174994eebe4d1962b0e00cb3aa433e725059da48209bdb6dea68d4ec5fd7

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed 66014748e933326b2a4dba73bf537c5b52427a8e via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 6601474

@github-actions

github-actions Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — ✅ PASS

Premise-level review of 66014748e933326b2a4dba73bf537c5b52427a8e via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify a bracketed id saved today actually applies at cron fire time — MODEL_ID_RE (validation.py:1976, acp/runtime.py:1071) still rejects brackets, so the stored spelling must always take the catalog_key/resolve_pin_spelling fold first.

What this change ships

Inventory (3 items) — 2 justified

Intent: let users save cron/spawn model selections whose ids carry a provider qualifier like [1m]/[high], which kiro-cli and the registry already advertise — a FIX.

  1. A provider-qualified model id (one trailing bracket) is now accepted when saving a cron, spawn, CLI or session model selection — justified
  2. The model-selection spec gains a section describing the accepted name format — justified
  3. test_model_selection.py leaves the Black baseline and its two with blocks are reformatted — rides along (declared; the ratchet requires removing a file that became clean)

Provenance checked: model_registry.json carries global.anthropic.claude-opus-4-8[1m] as a provider id and the added accept-tests fail on base's _MODEL_NAME_RE (no bracket class). Sibling gates (MODEL_ID_RE ×2, decisions/types.py) guard a different wire whose advertised spellings are fold-resolved; the description declares them unchanged.

[FIRST-PRINCIPLES-REVIEWED] 6601474

Description read

[DESCRIPTION-READ] cdd06c538af127bf398bf4e3e8dfa4c6d42963ed3718147fed140c12c8855e62

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 6, 2026
@bolichen97

Copy link
Copy Markdown
Contributor

@Premshay Thanks for the patch. I checked it against main: none of this has landed there. MODEL_ID_RE in src/kiro_crew/validation.py is still ^[a-zA-Z0-9][a-zA-Z0-9._-]{0,127}$, and no merged commit has touched it, so the widened charset is still yours alone.

What main already does is different from what the body describes. Bracketed suffixes such as [1m] are stripped in src/kiro_crew/acp/client.py, and the model-name fields that actually carry a configured selection are gated by _MODEL_NAME_RE (src/kiro_crew/validation.py:129), which this PR leaves untouched. MODEL_ID_RE has a single consumer, AcpRuntime.__init__ in src/kiro_crew/acp/runtime.py, and no production call site passes model= to it (src/kiro_crew/providers/acp.py, src/kiro_crew/session_background.py, src/kiro_crew/review_pool.py). So the diff cannot deliver "configured provider model selections cannot reach the ACP adapter", which is the same premise the First Principles lane blocked on.

The remaining scope worth keeping is the narrower one: show one real entry point that rejects gpt-6-astra[high], then widen the charset that guards that path. If that turns out to be the shared validator, _MODEL_NAME_RE is the target rather than MODEL_ID_RE. Please also update the pattern literal duplicated in the runtime.py rejection message, which this change makes stale; the Opus and Design lanes both flagged it.

Two blockers before it can land: the branch is 130 commits behind main and needs a rebase, and #5274 adds two new MODEL_ID_RE call sites that would silently inherit the wider charset, so the two should be coordinated.

Posted from the 2026-09-08 open-PR relationship audit (read-only, one auditor per PR); reply here if any of this is wrong.

@Premshay
Premshay force-pushed the upstream/qualified-model-ids branch 2 times, most recently from 15f350a to 17f72a5 Compare September 14, 2026 15:20
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 14, 2026
@Premshay
Premshay force-pushed the upstream/qualified-model-ids branch from 17f72a5 to 13a8a66 Compare September 17, 2026 15:31
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 17, 2026
@Premshay
Premshay force-pushed the upstream/qualified-model-ids branch from 13a8a66 to 57817f6 Compare September 18, 2026 15:26
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 18, 2026
@Premshay

Copy link
Copy Markdown
Contributor Author

Checked this against your note. The configured selections that carry a model id are gated by _MODEL_NAME_RE (src/kiro_crew/validation.py:129), not MODEL_ID_RE, and the earlier head widened the wrong gate.

The branch now widens _MODEL_NAME_RE to accept a bounded bracket qualifier while keeping the leading-alphanumeric and shell-metacharacter protections. MODEL_ID_RE and AcpRuntime validation are untouched. Regression coverage: an accepted gpt-6-astra[high] end to end through api_crons_create, and a still-rejected gpt-6-astra[high]!.

The description's "kept in sync" line referred to that earlier, wider patch; the corrected description goes with this head. Rebased onto current main as one commit — please re-review at the new head.

@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Sep 26, 2026
@Premshay
Premshay force-pushed the upstream/qualified-model-ids branch from 86d45af to 32615dd Compare September 30, 2026 08:36
@Premshay
Premshay requested a review from a team as a code owner September 30, 2026 08:36
@Premshay

Copy link
Copy Markdown
Contributor Author

Fixed in 32615dd: the unrelated website/scripts/restyle-baseline.json hunk is gone. The repaired branch has no website diff against its upstream base.

@Premshay

Copy link
Copy Markdown
Contributor Author

Fixed in 32615dd: _MODEL_NAME_RE now accepts exactly one optional, nonempty trailing bracket qualifier. Regression tests reject empty, unbalanced, nested, repeated and non-trailing qualifiers through both spawn validation and the cron endpoint. All 43 focused tests pass.

@Premshay

Copy link
Copy Markdown
Contributor Author

The Black baseline change is retained because test/test_model_selection.py is touched by the regression coverage and is now Black-clean. scripts/check_black_formatting.py requires removing newly clean files from the exemption list; restoring that entry would fail the ratchet. The formatting gate passes at 32615dd. This is limited to the touched test, with no frontend baseline changes.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 30, 2026
Configured model selections — cron job pins and per-spawn overrides — are
validated by _MODEL_NAME_RE, which rejects provider-qualified ids such as
gpt-6-astra[high] that the live kiro-cli model list advertises, so a user
selecting one cannot save the job. Admit [ and ] in the charset; the
leading-alphanumeric anchor still blocks CLI-flag injection.

MODEL_ID_RE and its AcpRuntime rejection message stay on the narrower
charset: no production call site passes model= to AcpRuntime.
@Premshay
Premshay force-pushed the upstream/qualified-model-ids branch from 32615dd to 6601474 Compare October 1, 2026 08:55
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 1, 2026
@Premshay

Premshay commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Verified unchanged head 6601474. The current PR description passes the repository hygiene check, so the earlier missing-section report is stale. Qualified-model validation and cron approval tests plus the exact reported third-turn routing node: 44 passed on Linux. The third-turn node also passes on clean upstream 5ca51b0. This does not reproduce the Windows failure or establish a baseline Windows failure; that remains unresolved. No code or branch rewrite was needed for the stale hygiene finding.

@Premshay

Premshay commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Further triage of the exact Windows failure at head 6601474: job https://github.com/kirodotdev/KiroCrew/actions/runs/36839359625/job/110297329572 reports only two receipt rows in test_a_third_turn_routes_from_the_base_model_again, and its captured log includes “decisions: could not append log row: could not acquire exclusive file lock: still held after waiting 0.50s (limit 0.5s); refusing to proceed unserialized”. It also reports the fixture AsyncMock's non-iterable OAuth drain result.

I traced the decision receipt through the bounded log append/commit-event path and inspected the test's _settle helper: it awaits or cancels the slot task; it does not synchronize the decision log's background file operation. This narrows the investigation to native-Windows receipt/file-lock behavior, but does not prove the cause of that lock timeout. The PR does not change these decision-log/test-helper files.

The previously passing exact Linux test is not Windows verification. No portable deterministic defect has been established here, and no timing threshold, lock guard, or assertion was weakened. Next verification needs the Windows 3.12 shard environment with instrumentation identifying the lock holder and append-task completion. The PR-description hygiene finding remains already addressed.

@Premshay

Premshay commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Current hygiene is already verified on unchanged head 6601474: PR Hygiene job 110317817746, created after the failed 09:02 job, passed at 10:02 UTC. Its workflow run reports this exact head SHA. The older failure remains visible in the rollup but is not evidence that the current description is malformed. The current body also passes the repository description checker locally.

Updated only the Tests evidence and unchecked the blanket existing-tests-pass checkbox to record the unresolved Windows result; Goal, Why it matters, and Not a goal remain unchanged. No source or branch change.

The remaining Windows shard 1 failure is test/test_decisions_model_route_ratchet.py::test_a_third_turn_routes_from_the_base_model_again: the third decision row is absent, alongside could not acquire exclusive file lock: still held after waiting 0.50s (limit 0.5s). The test and decisions/log.py are byte-identical at this head and clean base d8fa6a5. The exact node passes on Linux at both refs. This does not prove a clean-base Windows failure or a flake. The logs do not identify the holder; next action is a native Windows targeted run with lock-holder/timing evidence. No timeout increase, weakened serialization, or platform emulation was used.

@iamwhatever iamwhatever mentioned this pull request Oct 7, 2026
3 of 4 tasks
iamwhatever pushed a commit that referenced this pull request Oct 8, 2026
Widen the shared cron/subagent _MODEL_NAME_RE to accept exactly one
optional non-empty trailing bracket qualifier (e.g. gpt-6-astra[high]).
Empty, unbalanced, nested, repeated or non-trailing qualifiers and shell
metacharacters remain rejected. MODEL_ID_RE and ACP runtime validation
are unchanged. The touched test file is Black-clean, so its baseline
exemption is removed.

Fixes #17891
Replaces #9054 by @Premshay

Co-authored-by: Premshay <28099628+Premshay@users.noreply.github.com>
@iamwhatever

Copy link
Copy Markdown
Contributor

Thank you for this fix, @Premshay! Your _MODEL_NAME_RE bracket-qualifier change is valuable and still needed on main. To get it through the full CI and review lanes, I've opened an in-repo replacement that carries your change verbatim with you credited as co-author: #17892 (Fixes #17891). All automated checks are green there. Closing this one in favor of the replacement.

@iamwhatever iamwhatever closed this Oct 8, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Oct 8, 2026
chenmingwei23 pushed a commit that referenced this pull request Oct 8, 2026
Widen the shared cron/subagent _MODEL_NAME_RE to accept exactly one
optional non-empty trailing bracket qualifier (e.g. gpt-6-astra[high]).
Empty, unbalanced, nested, repeated or non-trailing qualifiers and shell
metacharacters remain rejected. MODEL_ID_RE and ACP runtime validation
are unchanged. The touched test file is Black-clean, so its baseline
exemption is removed.

Fixes #17891
Replaces #9054 by @Premshay

Co-authored-by: cixuuz <26782336+cixuuz@users.noreply.github.com>
Co-authored-by: Premshay <28099628+Premshay@users.noreply.github.com>
@Premshay
Premshay deleted the upstream/qualified-model-ids branch November 13, 2026 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants