Skip to content

test(dashboard): point the credit capture harnesses at the kiro backend - #25348

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/credit-harness-kiro-backend
Oct 28, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/credit-harness-kiro-backend

Conversation

@RohanK6

@RohanK6 RohanK6 commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The two Kiro credit screenshot harnesses produce their frames on main again.

On current main, website/scripts/capture-usage-refresh.mjs exits 1 at frame 02 (the failed read): the Kiro Account modal fades out before it can be checked. website/scripts/capture-credit-pill-states.mjs exits 1 waiting for the same modal in its failed scenario.

Why it matters

These harnesses are how a PR that touches the credit pill or the Kiro Account modal shows reviewers what it looks like. While they fail, that evidence has to be built by hand, and a broken frame can pass for a product bug, which is how this was found while working on #23655.

Not a goal

  • No change to the dashboard itself: hiding the credit surface on a backend that is not kiro is correct.
  • No change to KIROCREW_CONFIG_FIXTURE, which every other harness uses.
  • No change to any other capture harness.

What changed (motivation → approach → change)

Frames time out → the harnesses serve a config with agent.acp_backend unset, which since #20359 resolves to kas, not kiro, and on a backend that is not kiro a failed or empty read hides the credit pill and closes the modal → give both harnesses a config that names kiro.

Part Change
website/scripts/lib/stub-dashboard-api.mjs adds KIRO_BACKEND_CONFIG_FIXTURE: the base fixture with acp_backend set to kiro
capture-usage-refresh.mjs serves it for /api/config/kirocrew; its comments no longer say unset means kiro
capture-credit-pill-states.mjs serves it for /api/config/kirocrew

Backwards compatibility

Compatible: only screenshot harnesses and a test change, and the existing fixture export is untouched.

Removes nothing: the change adds one export and two route answers.

Tests

website/src/test/stubDashboardApiConfigFixture.test.tsx adds offers a config that resolves to the Kiro backend for the credit harnesses. It checks that the new fixture resolves to the kiro backend through isKiroBackend, and that it differs from the base fixture only in agent. On main it fails with a TypeError reading agent of undefined, since the export does not exist. With this change the file passes, 7 of 7.

Manual verification

Built the SPA on main at fe317a6 and ran both harnesses, before and after:

Harness main this PR
capture-usage-refresh.mjs exit 1 at frame 02 exit 0, 28 asserted frames
capture-credit-pill-states.mjs exit 1, dialog timeout exit 0, 12 shots

Related Issues

Closes #25345

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

capture-usage-refresh.mjs and capture-credit-pill-states.mjs served a
config with agent.acp_backend unset. Since KAS became the default
backend that no longer resolves to kiro, so a failed or empty usage
read hid the credit pill and closed the Kiro Account modal, and both
harnesses timed out before writing their frames.

The shared stub now offers KIRO_BACKEND_CONFIG_FIXTURE, and both
harnesses serve it. A vitest pins that it resolves to the kiro
backend.

Closes #25345
@RohanK6
RohanK6 requested review from a team and KiroCrewCaptain as code owners October 28, 2026 17:38
@RohanK6
RohanK6 requested a review from cruisercohen October 28, 2026 17:38
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of fdcf9efb54003ddaec5d797c2e980aafc91d1059 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check. The diff only touches screenshot harness scripts and a test, so no user-visible control changes and no screenshot is needed.

[UX-REVIEWED] fdcf9ef

Description read

[DESCRIPTION-READ] 9fdd34d6c31c372cb7fc8372bbd5feecddf69a081e927e55bd992517473c6311

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of fdcf9efb54003ddaec5d797c2e980aafc91d1059 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the change only touches test harnesses and fixes the cause directly. It names the kiro backend explicitly, so the harnesses no longer depend on the unset default resolving to kiro.

[DESIGN-REVIEWED] fdcf9ef

Description read

[DESCRIPTION-READ] d7c46f2a52c2a5928b303c885f57742568fba1e37af38918cc15b49aa9c1cec8

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision fdcf9efb54003ddaec5d797c2e980aafc91d1059 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of fdcf9efb54003ddaec5d797c2e980aafc91d1059 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] fdcf9ef

False positive or not applicable? A repository writer can comment:
/ai-review override gpt fdcf9efb54003ddaec5d797c2e980aafc91d1059 false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed fdcf9efb54003ddaec5d797c2e980aafc91d1059 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] fdcf9ef

Verdict parsed from the review's SHA-scoped output markers for commit fdcf9efb54003ddaec5d797c2e980aafc91d1059.

False positive or not applicable? A repository writer can comment:
/ai-review override fable fdcf9efb54003ddaec5d797c2e980aafc91d1059 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 28, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 28, 2026 21:05

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (4 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test harness only; the credit capture scripts and their fixture test get a config that resolves to the kiro backend, no shipped code touched.

@KiroCrewCaptain
KiroCrewCaptain merged commit 7b92d8f into main Oct 28, 2026
118 of 121 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/credit-harness-kiro-backend branch October 28, 2026 21:06
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 28, 2026
tim-finnigan added a commit that referenced this pull request Nov 1, 2026
Four utility lanes that sit where a human is waiting to merge move from
the shared hosted pool to the `kirocrew-runners` larger-runner group,
reached by its `kirocrew-linux-4-core` label inside a
`github.repository` conditional so a personal fork keeps a hosted
fallback:

- `issue-gate.yml`: `issue-gate`
- `pr-size-override.yml`: `record`
- `sensitive-change-review-comment.yml`: `rerun`
- `disposition-deferral-check.yml`: `validate-deferral`

Each is seconds of work whose wall clock is almost all queue wait, so
the move buys queue position, not capacity. Both branches of every
expression are GitHub-hosted Ubuntu VMs carrying no fleet credentials,
so each lane's isolation contract is unchanged; none of these events can
reach the AWS webhook's actor allowlist, which is why they stay off the
CodeBuild fleet.

`merged-readiness-audit.yml`'s `audit` job is deliberately NOT included.
`readiness_at_merge` in `scripts/audit_bypass_merge.py` returns the
newest `PR Readiness` state created at or before `merged_at`, so a run
taken hours late records the same value; the lane has no latency harm to
remove and would only occupy a capped group.

The `docs/ci/ci-and-reviews.md` inventory edit for this wave is deferred
to #26425 rather than carried per PR.

Part of #25348
iamwhatever pushed a commit that referenced this pull request Nov 2, 2026
…25349)

Four utility lanes that sit where a human is waiting to merge move from
the shared hosted pool to the `kirocrew-runners` larger-runner group,
reached by its `kirocrew-linux-4-core` label inside a
`github.repository` conditional so a personal fork keeps a hosted
fallback:

- `issue-gate.yml`: `issue-gate`
- `pr-size-override.yml`: `record`
- `sensitive-change-review-comment.yml`: `rerun`
- `disposition-deferral-check.yml`: `validate-deferral`

Each is seconds of work whose wall clock is almost all queue wait, so
the move buys queue position, not capacity. Both branches of every
expression are GitHub-hosted Ubuntu VMs carrying no fleet credentials,
so each lane's isolation contract is unchanged; none of these events can
reach the AWS webhook's actor allowlist, which is why they stay off the
CodeBuild fleet.

`merged-readiness-audit.yml`'s `audit` job is deliberately NOT included.
`readiness_at_merge` in `scripts/audit_bypass_merge.py` returns the
newest `PR Readiness` state created at or before `merged_at`, so a run
taken hours late records the same value; the lane has no latency harm to
remove and would only occupy a capped group.

The `docs/ci/ci-and-reviews.md` inventory edit for this wave is deferred
to #26425 rather than carried per PR.

Part of #25348
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(dashboard): Kiro credit capture harnesses fail on main since KAS became the default backend

2 participants