Skip to content

test(issue-radar): cover failed reads in the crew wake sweep - #35514

Merged
KiroCrewCaptain merged 2 commits into
mainfrom
kc-wt-32663
Nov 19, 2026
Merged

KiroCrewCaptain merged 2 commits into
mainfrom
kc-wt-32663

Conversation

@dodgeblaster

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Pin, with tests, that an Issue Radar wake sweep whose comment read or open-issues read fails wakes no crew and leaves its saved wake state untouched.

The crew wake sweep (new work and @mentions) has two failure paths that no test covers. One is the GitHub repository-comments read raising during the mention scan, for example on a rate limit. The other is the open-issues read failing during new-work detection. The code already keeps the mention cursor, the creation floor and the per-crew marks where they were on both paths. Nothing pins that, so a later change could move the cursor past unread comments, or reseed a crew's mark, and no test would go red.

Why it matters

If either path regresses, a crew silently misses a question someone asked it, or it is woken for backlog issues it has already been told about. Both failures are quiet: no error reaches the operator, and the only symptom is a crew that ignores a mention or replays old work.

Not a goal

  • Changing any production behaviour. The diff adds tests only.
  • Covering the GitLab notes-read failure paths, which have their own tests in test_issue_radar_crew_runtime_wake.py.

What changed (motivation → approach → change)

Two tests drive each failure through sweep_repo, with the same fake forge client, fake gateway state and real crew store that the existing wake tests use. Each checks that no crew is woken and that crew-wake-checks.json is byte-identical after the failed sweep. Then the read is allowed to succeed, and the test checks that the next sweep finds the mention or the new issue.

  • Failed comment read: the saved state starts with a floor held behind the cursor, as an unfinished earlier scan leaves it. A change that moves either one on failure turns the test red.
  • Failed open-issues read: a second crew is first seen during the failure. A sweep that treated the failure as an empty page would seed that crew below the backlog and later wake it for old issues.

The tests live in a new sibling module, test/test_issue_radar_crew_runtime_wake_failed_reads.py, because test/test_issue_radar_crew_runtime_wake.py is already past the 1,600-line split target. The new module imports the existing _WakeClient and the cutoff fixture from that file.

Backwards compatibility

Compatible: the diff adds a test module and changes no production code.

Removes nothing: only a new test file is added.

Tests

  • test_a_failed_comment_read_moves_no_mention_state_and_wakes_nobody: when list_repo_comments_since raises, nobody is woken, the cursor, floor, seen ids and at_cursor are kept, and the next good read wakes the crew for the mention.
  • test_a_failed_open_issues_read_moves_no_mark_and_wakes_nobody: when list_open_issues_first_page raises, nobody is woken, scope marks and new-issue marks are kept (a crew first seen during the failure gets none), and the next good read wakes the scoped crew for the new issue only.

Mutants, each applied to crew_runtime.py by hand and reverted:

  • The mention-read except advances the cursor (next_cursor = "9999-..."): the comment-read test fails.
  • The mention-read except marks the scan complete (complete = True), so the floor moves: the comment-read test fails.
  • _read_updated_open_issues returns [] instead of None on failure: the open-issues test fails.

Determinism: 20/20 runs at -n0. 10/10 runs at -n 4 --dist loadgroup --max-worker-restart=2, with test_issue_radar_crew_runtime_wake.py and test_issue_radar_crew_runtime_sweep.py as co-tenants. One shuffled order of this file plus test_issue_radar_crew_runtime_wake.py (71 tests, seed 29977) passed. The tests use fixed forge timestamps and no sleeps, and test/test_flake_pattern_ratchet.py passes.

Manual verification

N/A: unit coverage is sufficient for a test-only change.

Related Issues

Fixes #32663

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 11, 2026
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 024f90681835ce961e2de8447426557f5366950e ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 024f90681835ce961e2de8447426557f5366950e touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 024f90681835ce961e2de8447426557f5366950e — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

This PR only adds tests and changes no production code, so there is no design-level concern.

Notes

[DESIGN-REVIEWED] 024f906

Description read

[DESCRIPTION-READ] e85639e571ea14e0740c31466017a874cf60d9ee3ed4dd88c655b7b2828b52e1

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 024f90681835ce961e2de8447426557f5366950e and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 024f906

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 024f90681835ce961e2de8447426557f5366950e false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 024f90681835ce961e2de8447426557f5366950e — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 024f906

Verdict parsed from the review's SHA-scoped output markers for commit 024f90681835ce961e2de8447426557f5366950e.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 024f90681835ce961e2de8447426557f5366950e false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 11, 2026
@dodgeblaster

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in test/test_mcp_remote_transport.py since this head ran (d161816 -> d057009). The two TestRewriter reds on Backend Tests shard 8 and the Coverage Gate failure came from main (#35534, fixed by #35541), not from this diff. CI re-running.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 11, 2026
@dodgeblaster

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in test/test_update_boot_apply.py and the updates handlers since this head ran (d057009 -> 0229fd9). The wait-discipline ratchet, updates handlers facade and Coverage Gate reds came from main (broken by #35088, fixed by #35625 and #35725), not from this diff. CI re-running.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 12, 2026
@dodgeblaster
dodgeblaster marked this pull request as ready for review November 12, 2026 18:53
@dodgeblaster
dodgeblaster requested review from a team and KiroCrewCaptain as code owners November 12, 2026 18:53
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 12, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 18, 2026 10:53

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved automatically by the code change approver, not by a person. The decider estimates a 6% chance that this change introduces a defect, within the 12% limit for approving on its own, and the change has at most 400 changed lines.

Test-only PR adding two sweep_repo tests for failed comment/open-issues reads. Traced both against crew_runtime._detect_mentions/detect_new_work and crew_mention_scan: on a failed GitHub comment read the cursor, floor (held behind the cursor), seen ids and at_cursor stay as they were, so the saved state is unchanged and crew-wake-checks.json is not rewritten. On a failed open-issues read issues is None, so no crew is seeded, and the GitLab-path mention scan with no threads returns its state unchanged. Recovery-sweep expectations (#5 asked c1; #6 scope-label only, Bode seeded at top) match the code. The fixtures and imports follow patterns the repo already uses, and the stated mutants would be caught. No production change, no state/lifecycle/trust concerns.

Tier: 1, every tier gate passed.
Criteria: no merge conflict, no owner block, no change request, the default branch, no security-sensitive path, no design document to agree on, the readiness status, the review lanes, the P0 history, the linked issues' tier, the review lessons, no concern label, the file count, a safe change category, not its own change, the decider, the security checklist, the change matches its title, no competing change.

Decider v3c (217fe607). To overrule it, request changes or dismiss this review.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 18, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these tests. They check the whole wake-checks file byte for byte, then run a recovery sweep that confirms the work is still found, so a failure read as an empty page goes red. Both tests pass at this head and on current main. The two mutants from the PR body also turn them red as you describe. One change is needed before merge:

  1. test/test_issue_radar_crew_runtime_wake_failed_reads.py:48: setUp holds self._tmp = tempfile.TemporaryDirectory() and uses its .name as the radar store root. After this PR's base, main added the semgrep rule kirocrew.test-shared-tmp-dir (severity ERROR, diff-scoped). The no-test-side-effects rule in AUTOSDE.yaml now refuses the same shape, because the Issue Radar store creates its root on read and can re-create a removed directory after cleanup. Today's green SAST ran before that rule existed. After a rebase or any new CI run against main, SAST will flag this new line and go red. Smallest fix: drop import tempfile and the three TemporaryDirectory lines, and add the fixture that test/test_issue_radar_crew_runtime_wake.py already uses on main: @pytest.fixture(autouse=True) on def _per_test_root(self, tmp_path): self.root = tmp_path, with import pytest. Keep the write_settings and checks_file lines in setUp. The fixture runs before it. Then rebase onto main.

Optional:

  • issue-radar is now pinned in src/kiro_crew/apps/adoption-pins.json, so adopted hosts run the app repository's copy. Mirroring these two cases there would keep the coverage when the app leaves core. This is not required for a test-only PR.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 18, 2026
The new-work and mention wake sweep has two failure paths with no
test: the GitHub repository-comments read raising (a rate limit) during
the mention scan, and the open-issues read failing during new-work
detection. Both must wake nobody and leave crew-wake-checks.json as it
was, so the mention cursor, the creation floor and the per-crew marks
never move past work nobody read.

Add one test per path in a sibling module (the wake test file is
already past the 1,600-line target). Each drives the failure through
sweep_repo, asserts no wake and a byte-identical state file, then lets
the read succeed and asserts the next sweep finds the mention or the
new issue. The open-issues test adds a crew first seen during the
failure, so a sweep that took the failure for an empty page would seed
it below the backlog and wake it for old issues.

Fixes #32663
The radar store creates its root on read, so a self-removed
TemporaryDirectory can be re-created after cleanup. Use the per-test
tmp_path fixture the sibling wake tests use, as review asked and the
test-shared-tmp-dir rule requires.
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 19, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 19, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider estimates a 5% chance that this change introduces a defect, within the 12% limit for approving on its own, but it is held at tier 2: changes requested.

Test-only PR adding two sweep_repo tests for failed comment and open-issues reads. Checked against crew_runtime.py: on a failed mention read (lines 1304-1306) the cursor, floor, seen, at_cursor and pending state are written back unchanged, so crew-wake-checks.json comes out byte-identical. A None open-issues read skips all marking (1161-1179). Each of the three listed mutants would turn its test red; a [] return, for example, seeds the new unscoped crew's mark at 0. The fixture imports resolve through conftest's _floor_monkeypatch, and no test classes are re-collected. CI is fully green, including new-test stability and the flake ratchet. No production state is touched.

Tier: 2, held for a person: changes requested.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 024f906818 · 134 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 19, 2026 22:14

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. On head 024f906 the review traced both tests against the sweep code. A failed comment read keeps the mention cursor, the floor (held behind the cursor), the seen ids and at_cursor, so crew-wake-checks.json stays byte-identical, and the recovery sweep finds #5 asked c1. A failed open-issues read seeds no crew, and the recovery sweep wakes only the scoped crew for #6. The three mutants listed in the body each turn a test red. PR Readiness is green on this head.

Earlier change request, item 1 (shared tempfile.TemporaryDirectory store root): fixed. The head uses an autouse tmp_path fixture, and import tempfile is gone. The rebase onto main is also done.

Done well: adding the Bode crew during the failed read catches the case where a failed read treated as empty would seed a new-issue mark below the backlog.

Optional:

  • Byte equality cannot tell the local catch in _detect_mentions from the outer catch in sweep_repo. A scope-labelled issue in the failing sweep in test 1, asserted to still wake, would show the failure is handled where it happens.
  • issue-radar is pinned to its app repository. Carrying this module there in the next port keeps the coverage after the app moves out of core.

@KiroCrewCaptain
KiroCrewCaptain merged commit 9ceb508 into main Nov 19, 2026
101 of 103 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the kc-wt-32663 branch November 19, 2026 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(issue-radar): cover failed comment and open-issue reads in the wake sweep

2 participants