Skip to content

fix(acp): apply crew reasoning effort under the advertised model id - #16783

Closed
ash663 wants to merge 1 commit into
kirodotdev:mainfrom
ash663:fix/claude-effort-follows-wire-model
Closed

ash663 wants to merge 1 commit into
kirodotdev:mainfrom
ash663:fix/claude-effort-follows-wire-model

Conversation

@ash663

@ash663 ash663 commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A Claude crew that pins both a model and a reasoning effort runs at that effort.

A crew configured with a model pin (claude-opus-5.5) and reasoning_effort: max on the Claude backend runs at Claude Code's own default effort (medium in the session transcript). Nothing is logged: no "will not be applied" warning from the factory, and no "initial effort apply failed" warning from the provider.

Why it matters

Every Claude crew that pins a model whose configured spelling differs from the spelling claude-agent-acp advertises silently loses its configured effort on every session start. The dashboard shows the configured level while the session runs another.

Not a goal

  • No change to how the factory resolves or validates the level, or to the effort step-down ladder in _set_effort_config_option.
  • No change to the kiro-family overlay path or the shared-runtime start path.
  • No effort is applied when the pin itself was refused (see below).

What changed (motivation → approach → change)

  • Symptom: the configured effort never reaches claude-agent-acp when the crew also pins a model.
  • Cause: the factory keys effort_per_model by the configured model id. During startup AcpClient._apply_startup_model rewrites the session model to the advertised spelling (claude-opus-5-5[1m]) before pushing it. _apply_initial_effort then resolves effort for the advertised spelling, misses the entry keyed under the configured one, and returns without pushing anything.
  • Fix: AcpProvider keeps the model it was constructed with. On the client start path, after ensure_ready(), start moves that entry to the model the client records, so _apply_initial_effort finds it and pushes ('effort', <level>). An entry already present under the recorded spelling wins.
  • Refused pin: when every spelling of the pin is refused, the client records the backend default (auto). The entry stays under the configured spelling and no effort is pushed. The level was chosen for the pinned model, and auto is not effort-capable in the registry, so the session keeps the adapter's own default. That is the same outcome as before this change.
  • docs/system-specs/modules/providers.md states the re-keying next to the factory's threading of the level.

Backwards compatibility

Compatible: when the configured and recorded spellings are equal, the entry is popped and put back under the same key. Every other path is untouched.

Removes nothing: the change only moves an in-memory map entry to the key the existing lookup reads.

Tests

test/test_acp_provider.py::TestEffortControl::test_claude_start_pushes_effort_keyed_by_configured_model_spelling: a Claude provider built with effort_per_model={'claude-opus-5.5': 'max'}, whose client's ensure_ready() rewrites the model to claude-opus-5-5[1m], must push ('effort', 'max') from start(). On the unfixed tree it fails with Expected set_config_option to have been awaited once. Awaited 0 times. With the fix it passes.

Also run: test/test_acp_provider.py, test/test_acp_codex_harness.py, test/test_codex_effort_pair_model_switch.py and test/test_session_work_dir.py (316 passed); black, isort and flake8 on the touched files; mypy src/kiro_crew/; the black, subprocess-encoding, harness-parity and docs-lint gates.

Manual verification

N/A — the unit test drives the real start() → _apply_initial_effort() → _set_effort_config_option() path with the client's model rewrite reproduced. The observed failure was the missing set_config_option('effort', …) call, and that call is what the test asserts.

Related Issues

Found while testing per-crew ACP backend pins (#16200) with a Claude crew pinned to a model and an effort.

Pattern harvest

Not generalizable: this is the one map keyed by the model id before startup and read by it after. Every other effort site reads and writes under the recorded spelling.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 4, 2026
@dwu96

dwu96 commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review.

When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically.

2 similar comments
@dwu96

dwu96 commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review.

When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically.

@dwu96

dwu96 commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review.

When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically.

@ash663
ash663 marked this pull request as ready for review October 4, 2026 18:56
@ash663
ash663 requested a review from a team as a code owner October 4, 2026 18:56
@ash663
ash663 requested a review from CrysisDeu October 4, 2026 18:56
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 4, 2026
@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 111f2f23792e897d97c231ec574312a47e5d6380 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check whether the shared-runtime start path (_start_kiro_runtime, which codex now uses) also records a different spelling of the pin. If it does, the effort is still silently dropped there.

[DESIGN-REVIEWED] 111f2f2

Description read

[DESCRIPTION-READ] be894e33cee06b28f0a3dc1fa95aed2ebfc0bac3786910b0398766813d142168

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 111f2f23792e897d97c231ec574312a47e5d6380 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Check that no backend on the shared-runtime start path also rewrites the pin spelling. The re-key runs only on the AcpClient path.

What this change ships

Inventory (4 items) — 4 justified

Intent: a Claude crew that pins both a model and a reasoning effort actually runs at that effort. This is a FIX.

  1. A Claude crew whose pin is spelled differently from the advertised id now starts at its configured effort, not the adapter's default — justified
  2. When every spelling of the pin is refused, no effort is pushed and the session keeps the adapter default, as before — justified
  3. The providers spec now says the factory's effort entry is re-keyed at start — justified
  4. A regression test that fails on base ("Awaited 0 times") — justified

[FIRST-PRINCIPLES-REVIEWED] 111f2f2

Description read

[DESCRIPTION-READ] 553fde23d4f56ead9ded74d8828a4b785d7d96104310145d9eaa4177f8123251

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 111f2f23792e897d97c231ec574312a47e5d6380 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 111f2f2

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 111f2f23792e897d97c231ec574312a47e5d6380 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 111f2f2

@ash663
ash663 force-pushed the fix/claude-effort-follows-wire-model branch from d13132d to 10a608d Compare October 4, 2026 19:31
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 4, 2026
The factory keys effort_per_model by the configured model id, but
claude startup rewrites the session model to the advertised spelling
(claude-opus-5.5 -> claude-opus-5-5[1m]). The initial effort lookup
then misses and the configured level is never pushed, so Claude Code
runs its own default effort. Re-key the level to the model the session
runs after startup; a refused pin leaves it unapplied.
@ash663
ash663 force-pushed the fix/claude-effort-follows-wire-model branch from 10a608d to 111f2f2 Compare October 7, 2026 13:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 7, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Thank you for this fix, @ash663! 🙏

This PR conflicted with current main, so I re-ported your change onto a fresh branch in the main repo as #18742 — your authorship is preserved on the commit, with a Co-authored-by trailer. The replacement is green and links to a tracking issue (#18741). Closing this one in favor of #18742; your branch is left intact.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants