Skip to content

fix(usage): pick the /usage scrape engine from what kiro-cli offers - #25309

Merged
buluoray merged 1 commit into
mainfrom
fix/25287-usage-scrape-engine
Oct 28, 2026
Merged

buluoray merged 1 commit into
mainfrom
fix/25287-usage-scrape-engine

Conversation

@buluoray

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The dashboard's credit reading keeps working after kiro-cli retires its v2 engine.

The usage scrape appends --agent-engine v2 to its kiro-cli chat argv on every kiro-cli 2.28+ (src/kiro_crew/dashboard/handlers/sessions_parts/usage_scrape.py). kiro-cli moves its default engine to v3 on 11/2 and then retires v2. Once v2 is gone, that argv fails every scrape.

Why it matters

The scrape is the fallback whenever the GetUsageLimits API read returns no plan. With v2 gone, every such refresh spawns a kiro-cli that rejects its own argv, three of them park the scrape for six hours, and the credit pill loses its fallback for every user on the kiro-cli path.

Not a goal

  • Making /usage work on v3. On kiro-cli 2.35.1, kiro-cli chat --no-interactive --agent-engine v3 /usage sends /usage to the model, which answers that it "came through to me as a chat message". There is no local v3 equivalent to pin.
  • The Crew sign-in path from fix(usage): read credit usage from Crew sign-in without kiro-cli #21956. It never runs the scrape and is unchanged.
  • The unpinned PATH-only binary case. It already kept the plain argv and still does.

What changed (motivation → approach → change)

Symptom: a hardcoded v2 pin fails once v2 is retired. Root cause: the engine was chosen from a version floor (USAGE_SCRAPE_ENGINE_PIN_MIN_VERSION), never from what the binary actually offers. Pinning v3 instead would spend a model turn per refresh and print no table.

Change: the scrape now asks the pinned kiro-cli which engines it accepts and picks from that.

chat --help lists for --agent-engine Scrape argv
includes v2 --agent-engine v2, same as today
engines, but no v2 no scrape at all; the API reading stands alone
no selector, or the probe could not run plain argv, same as today below the old floor
  • src/kiro_crew/kiro_cli.py: chat_agent_engines(help_text) reads the selector's own [possible values: ...] line, and chat_agent_engines_at(binary) runs one bounded chat --help, cached by path and mtime. Like the --version probe it replaces, it runs only on the pin_kiro_cli binary, never on a PATH hit, because it runs outside the sandbox.
  • usage_scrape.py: _usage_scrape_argv returns None when no offered engine answers /usage. _wrap_argv_usage_scrape then wraps nothing.
  • sessions.py: _run_scrape returns None without spawning. That is not counted as an attempt and feeds no back-off. The refresh ends the way a parked one does (same-identity prior reading dimmed stale, else available: false with the API's partial fields and reason) but is not reported as scrape_parked, since nothing will lift it. The ExternalIdP corroboration scrape leaves account_unproven, because no_credential alone never decides.
  • docs/system-specs/modules/learn-cron-dashboard.md describes the new engine rule in place of the version floor.

Backwards compatibility

Compatible: on every kiro-cli in use today (2.35.1 lists v2, v1, v3) the argv is byte-for-byte what main builds.

Removes USAGE_SCRAPE_ENGINE_PIN_MIN_VERSION and the sessions facade's re-export of kiro_cli_version_at (the scrape no longer imports it).
Reader: src/kiro_crew/dashboard/handlers/sessions_parts/usage_scrape.py:_usage_scrape_argv -- release -- the constant's only reader, rewritten here
Reader: test/test_session_usage_scrape.py:TestUsageScrapeEngineFollowsTheInstalledKiroCli -- release -- replaces the old version-floor tests
Reader: test/campaign_facade_inventories/kiro_crew.dashboard.handlers.sessions.json -- release -- both names leave the inventory deliberately; git grep on origin/main finds no other reader of either through the facade

Tests

  • TestUsageScrapeEngineFollowsTheInstalledKiroCli::test_v2_retired_runs_no_scrape: a kiro-cli offering v3, v1 gets no scrape argv. Goes red if v2 is pinned again.
  • TestRefreshWithV2Retired: with v2 retired, an API plan is published with no spawn; a plan-less API spawns nothing, parks nothing and leaves available: false across four refreshes; a same-account prior reading is kept stale.
  • TestChatAgentEnginesParse: the selector's values are read from its own block, not from a later option's [possible values] that happens to list v2.
  • v2-offered, no-selector/unknown, and the pin-only probe rules are kept from the old tests.

Proof: not provable -- other: prove.py's all-hunk revert removes the new kiro_cli helper, so the tests fail at import, and its per-hunk mode wrote no report on one hunk. Manual mutation instead: putting back an unconditional argv += ["--agent-engine", "v2"] reddens 6 tests, including test_v2_retired_runs_no_scrape and TestRefreshWithV2Retired::test_no_plan_spawns_nothing_and_parks_nothing (4 spawns instead of 0).

Local runs: 1882 passed across the usage, kiro_cli, spawn-audit, facade-inventory and doctor suites; 995 passed / 44 skipped across the docs and spec suites.

Manual verification

On this Linux host with kiro-cli 2.35.1: chat_agent_engines_at(<pinned kiro-cli>) returns ('v2', 'v1', 'v3') and _usage_scrape_argv builds ... chat --no-interactive --agent-engine v2 --agent kirocrew-lite /usage, same as main. The v2-retired case is covered by unit tests only; no kiro-cli build without v2 exists yet. Windows and macOS are unverified here; the probe uses the same CREATE_NO_WINDOW and UTF-8 pinning as the existing --version probe.

Related Issues

Closes #25287

Sibling sweep

Swept: git grep -nE "agent-engine.*v2|"v2"" -- src/
Sibling: src/kiro_crew/dashboard/handlers/sessions_parts/usage_scrape.py -- changed here
Sibling: src/kiro_crew/acp/kas_transport.py -- not affected -- pins v3 for the KAS relay, never v2

Pattern harvest

Not generalizable: the only kiro-cli spawn that pins a retiring engine is this scrape; every other engine pin is v3.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners October 28, 2026 16:38
@buluoray
buluoray requested a review from cruisercohen October 28, 2026 16:38
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 28, 2026
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision e1dd3dc99cac04af5a324faeaa00800e74da8d7f touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of e1dd3dc99cac04af5a324faeaa00800e74da8d7f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: choosing the engine from what the binary offers fixes the actual cause, and neither removed name has a reader left outside the patch.

Notes

  • chat_agent_engines_at caches a failed probe (a timeout or nonzero exit) as None until the binary's mtime changes. That None keeps the plain argv. Once v3 becomes the default, each such scrape spends a model turn until back-off parks it. Consider caching only probes that succeeded.

[DESIGN-REVIEWED] e1dd3dc

Description read

[DESCRIPTION-READ] b3a775f0cf57354c1919aa45e556e0436804aac39ea0c8f5cd073d702c772318

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed e1dd3dc99cac04af5a324faeaa00800e74da8d7f — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e1dd3dc

Verdict parsed from the review's SHA-scoped output markers for commit e1dd3dc99cac04af5a324faeaa00800e74da8d7f.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e1dd3dc99cac04af5a324faeaa00800e74da8d7f false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of e1dd3dc99cac04af5a324faeaa00800e74da8d7f — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The change replaces an assumed version floor with a reading of the engines kiro-cli actually offers, which is the cause #25287 names. On today's binaries the scrape argv is unchanged.

What this change ships

Inventory (7 items) — 7 justified

Intent: keep the dashboard's credit reading working after kiro-cli retires its v2 engine. This is a FIX.

  1. On a kiro-cli whose engine list has no v2, the /usage scrape never spawns and the API reading stands alone — justified
  2. Whether v2 is pinned now depends on what chat --help lists, not on a version number — justified
  3. A new bounded chat --help probe runs once per pinned binary (cached by path and mtime), never on a binary found on PATH — justified
  4. A refresh where the scrape cannot exist does not count as an attempt, adds no back-off and is not reported as scrape_parked — justified
  5. The ExternalIdP corroboration answers account_unproven instead of sign-in-required when no scrape can run — justified
  6. The USAGE_SCRAPE_ENGINE_PIN_MIN_VERSION constant and the sessions facade's kiro_cli_version_at re-export are removed (declared, with Reader: lines) — justified
  7. A spawn-audit allowlist entry for the new probe — justified

[FIRST-PRINCIPLES-REVIEWED] e1dd3dc

Description read

[DESCRIPTION-READ] b68391947eadc3463b876b87dfdf9d771e1a110bda2b1b119208ae681a3f5855

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of e1dd3dc99cac04af5a324faeaa00800e74da8d7f and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e1dd3dc

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e1dd3dc99cac04af5a324faeaa00800e74da8d7f false-positive: <reason, 20 to 500 characters>

The credit scrape always appended `--agent-engine v2` on kiro-cli 2.28+.
kiro-cli moves its default to v3 and then retires v2, after which that
argv fails every scrape. Pinning v3 instead does not work: v3 sends
`/usage` to the model as a prompt (checked on kiro-cli 2.35.1).

The scrape now reads the `--agent-engine` values from the pinned
binary's own `chat --help` (cached by path and mtime). v2 listed: pin
it, as before. A selector without v2: no engine answers `/usage`
locally, so the scrape is not spawned, counts as no attempt, feeds no
back-off, and the GetUsageLimits API reading stands alone. No selector
or an unreadable probe keeps the plain argv, as before.

Closes #25287
@buluoray
buluoray force-pushed the fix/25287-usage-scrape-engine branch from ae1fc6b to e1dd3dc Compare October 28, 2026 16:55
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 28, 2026
@buluoray
buluoray enabled auto-merge (squash) October 28, 2026 18:53
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 28, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on behalf of the watch holder (bolichen). Design and First Principles pass on this head, UX skipped or pass, all checks green, no conflict, no open change request.

@buluoray
buluoray merged commit 38ac0f0 into main Oct 28, 2026
142 of 148 checks passed
@buluoray
buluoray deleted the fix/25287-usage-scrape-engine branch October 28, 2026 21:39
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Usage scrape pins kiro-cli's v2 engine and breaks when v2 is retired

2 participants