Skip to content

feat: report subagent credit usage alongside elapsed time - #14913

Merged
iamwhatever merged 1 commit into
mainfrom
fix/subagent-credit-usage-inrepo
Sep 29, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/subagent-credit-usage-inrepo

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

In-repo replacement for #8003 (fork welikoiwanenko:codex/subagent-credit-usage), rebased onto fresh main so a clean CI wave fires. The author's commit is cherry-picked with authorship preserved. Supersedes #8003.

Problem / Motivation

Goal: Show the credits consumed by each managed subagent alongside its terminal elapsed time.

Managed subagent completion reports elapsed time but not the credits consumed by that run. Parents and users cannot attribute the cost of a successful, failed, or stopped subagent without comparing noisy account-level totals.

For maintainer approval: cumulative positive reported credits appear in expanded Subagents activity cards, with elapsed time retained in compact headers and usage included in completion messages and spawn_status.

Why it matters

A failed or stopped attempt can consume credits even when it produces no final answer. Per-run attribution makes those costs visible, including billed transient retries, without relying on unrelated activity in the same account.

Not a goal

  • Estimate dollar costs or invent usage for providers that do not report credit billing.
  • Attribute native-harness subagents whose billing remains part of the parent turn.
  • Add a new billing ledger, dependency, configuration option, or schema migration.

What changed (motivation → approach → change)

  • Accumulate the shared TurnUsage representation per attempted provider turn. Completion events supply usage on success; the shared billing resolver supplies fresh usage on interrupted or failed attempts. Each attempt settles once, including cancellation during event handling and force-reaping.
  • Settle active usage before force-reap reporting so stopped runs cannot publish partial totals.
  • Preserve terminal credits and elapsed time in existing delivery/failure tombstones, then expose them through completion notifications and full or paged spawn_status responses. Parent delivery does not wait for a new post-completion state write.
  • Carry credits in live and replayed subagent_done frames so dashboard reconnects retain terminal usage. Legacy retained records remain readable without invented fields.
  • Display positive credits in expanded completed, failed, and stopped cards as the run-scoped copy Used … credits. Compact headers continue to show elapsed time. Backend summaries switch to minutes and seconds at 60 seconds and retain the existing credit precision.
  • Keep the credit formatter in the shared i18n formatting module so the activity chunk does not pull in the assistant-message component tree.

No dependency, configuration, schema migration, model-default, or changelog changes. Native harness subagents remain unattributed because they share the parent's billed turn. Providers that do not report credit billing return zero; displays omit that credit label rather than claim the run was free. This feature does not estimate charges or add dollar billing. Run totals are completion summaries, not additional billing-ledger entries; failed-attempt credits can make them larger than successful-turn records alone.

Backwards compatibility

Compatible: existing providers and retained records without credit data continue to render elapsed time without inventing usage.

Tests

  • Backend tests cover subagent execution, persistence, reaping, delivery, messaging, scale, and session behavior; a new format_subagent_usage test locks in credit precision, minute formatting, and omission of unreported/zero credits.
  • Frontend tests cover the activity viewer expanded/compact rendering and the Used … credits copy.
  • CI is the authoritative validation for this in-repo replacement.

Manual verification

N/A — the author's manual verification (fixture-rendered ActivityViewer expanded successful/failed/stopped cards in dark and light themes, credit-only expanded copy, elapsed-only compact headers, minute formatting) carries over unchanged; this replacement makes no code change to the feature. Live provider billing verification remains for maintainers.

Screenshots / video

Fixture data rendered by the current component; each card shows its own terminal usage.

kirocrew-credit-usage-dark-456x760 kirocrew-credit-usage-light-456x760

Related Issues

Implements the credit-attribution request in #4721. Supersedes #8003.

Fixes #4721

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested a review from a team September 29, 2026 03:20
@iamwhatever
iamwhatever requested a review from a team as a code owner September 29, 2026 03:20
@iamwhatever iamwhatever added the drive-to-green PR claimed by drive-to-green pipeline label Sep 29, 2026
@iamwhatever iamwhatever self-assigned this Sep 29, 2026
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 4fc1b5799eb2fa2b5cce8d9542113ac8d2e210e7 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify one live-provider run's displayed credit total — every backend test mocks the billing surface, and the description itself defers live billing verification to maintainers.

[DESIGN-REVIEWED] 4fc1b57

Description read

[DESCRIPTION-READ] 27294a02bdce60fed33ac125df424a3f3ec0bfb53855fe10b10577e6340be669

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 4fc1b5799eb2fa2b5cce8d9542113ac8d2e210e7 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

Two mechanisms this PR needs already exist — format_elapsed and its own _finite_nonnegative_number — and it re-spells both inline instead of calling them.

Not justified as shipped

  • Item 2 — duplicate of src/kiro_crew/messaging/status_reactions.py:138: format_subagent_usage re-implements format_elapsed's exact 12s / 3m 7s shape inline, differing only by round-vs-truncate (grep % 60}s: 2 backend spellings after this PR).
    Subtraction: delete the inline minute/second branch; call format_elapsed(math.floor(elapsed_value + 0.5)) to keep the dashboard-matching rounding.
    Clears when: the inline formatter is deleted, or the rounding delta is shown to need a separate function.
  • Item 4 — duplicate of src/kiro_crew/subagent.py:955: the tombstone-field loop in api_spawn_status re-spells _finite_nonnegative_number (grep isfinite: 2 spellings, both added by this PR), and they already diverge — the inline math.isfinite(value) raises OverflowError on a huge int, the trap the helper's try/except closes.
    Subtraction: replace the inline check with _finite_nonnegative_number.
    Clears when: the inline check is deleted in favor of the helper.
  • Item 8 — undeclared: abnormal-exit elapsed is now finalized once so tombstone and terminal event agree; the description never mentions this behavior change (spec documents it in the same commit).

What this change ships

Inventory (9 items) — 6 justified

Intent: Show the credits each managed subagent consumed alongside its terminal elapsed time (implements linked issue #4721) — ADDITION.

  1. Expanded terminal subagent cards show "Used N credits" or "Credit usage not reported" — justified
  2. Parent completion, delivery-failure and wave-digest messages gain a Usage: … credits · … line — duplicate of src/kiro_crew/messaging/status_reactions.py:138 (inline elapsed formatting)
  3. spawn_status MCP responses carry a [usage: …] prefix/header on terminal runs — justified
  4. GET /api/spawn/{id} returns elapsed/credits from memory or tombstone — duplicate of src/kiro_crew/subagent.py:955 (inline validator)
  5. Live and reconnect subagent_done frames carry credits — justified
  6. Delivered/failure tombstones persist terminal usage; delivery-debt ledger carries snapshots, not bare ids — justified
  7. Credits accumulate per attempted turn, including billed retries, cancels and force-reaps — justified
  8. Abnormal-exit elapsed finalized once so tombstone and terminal event agree — undeclared: not in the description
  9. fmtCredits moved from AssistantMessage to shared i18n format module — justified

[FIRST-PRINCIPLES-REVIEWED] 4fc1b57

Description read

[DESCRIPTION-READ] 073b8e7a5477815c66cb1f67e589e4a6146b11023bdd32a3d68b1ecabb961815

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 4fc1b5799eb2fa2b5cce8d9542113ac8d2e210e7 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 4fc1b57

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 4fc1b5799eb2fa2b5cce8d9542113ac8d2e210e7: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 29, 2026
@iamwhatever
iamwhatever force-pushed the fix/subagent-credit-usage-inrepo branch from d54bb58 to 5cd0764 Compare September 29, 2026 03:45
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Fable 5) — 🟡 CONCERNS

UX-level review of 4fc1b5799eb2fa2b5cce8d9542113ac8d2e210e7 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: CONCERNS

"Credit usage not reported" is ambiguous — the blind reader could not tell whether the run was free, unknown, or the counting broke.

Watch

  • The blind reader filed the new string under "Could not tell": "I can't tell whether 'not reported' means it was free, unknown, or something went wrong with the counting," and "the two cost lines don't match" against the sibling "Used 12.5 credits" (en.manual.json credits_not_reported). Every expanded terminal card on a provider that never reports credit billing carries this line permanently, so the ambiguity recurs on every card — frequent, low-impact informational friction. Smallest fix: name the cause in the string, e.g. "Credits not reported by this provider," which removes the "something went wrong" reading.

Suggestions

  • credits_not_reported in all locale files: attribute the absence to the provider ("Credits not reported by this provider") rather than the bare passive "not reported."

[UX-REVIEWED] 4fc1b57

Description read

[DESCRIPTION-READ] 38eb2ae65e23664f6d65ba34d8e9e259bd05efc0355b0956d74123b3d1fea76b

That sha256 names everything this verdict read: the PR description, plus the 5 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 4fc1b5799eb2fa2b5cce8d9542113ac8d2e210e7 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 4fc1b57

Verdict parsed from the review's SHA-scoped output markers for commit 4fc1b5799eb2fa2b5cce8d9542113ac8d2e210e7.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 4fc1b5799eb2fa2b5cce8d9542113ac8d2e210e7: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 29, 2026
Attribute reported credits to managed subagent runs, including failed and
cancelled attempts, so users and parent agents can compare their costs.

Reuse shared turn billing, preserve terminal usage in delivery/failure
tombstones, and show it across completion events, status queries, and
dashboard cards without delaying parent delivery.

Finalize the run's elapsed time once, before any abnormal-exit tombstone
write, so the persisted record and the terminal subagent_done event carry
the same value. Every abnormal arm (timeout, cancel, reap, error) wrote the
tombstone before elapsed was finalized in _run's finally / the reaper, so
sampling the wall clock inline at write time made a replayed spawn_status
disagree with the terminal event. _write_tombstone now finalizes and
persists info.elapsed; the run finally and the reaper set it before/instead
of re-sampling. In the reap-race test, delegate unrecognized
asyncio.to_thread calls to the real implementation so the process-wide
patch stays scoped to the state-writer seam it drives.

In the delivery-ledger test, the ledger's move from bare id strings to
SubagentDelivery objects forced the whole-dict assertions to change; assert
against a whole-ledger {key: [agent_id]} map (_pending_map) rather than the
ids under a single key, so the "exactly one key" pin the successor-dequeue
race depends on is preserved (debt parked under a second key still fails).

Co-authored-by: Vadym Velykoivanenko <5074317+welikoiwanenko@users.noreply.github.com>
@iamwhatever
iamwhatever force-pushed the fix/subagent-credit-usage-inrepo branch from 5cd0764 to 4fc1b57 Compare September 29, 2026 04:25
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 29, 2026
@iamwhatever
iamwhatever merged commit a7e26a1 into main Sep 29, 2026
97 of 99 checks passed
@iamwhatever
iamwhatever deleted the fix/subagent-credit-usage-inrepo branch September 29, 2026 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Surface per-session credit usage in subagent completion events

2 participants