Skip to content

fix(monitoring): wake review-only pull request watches - #21072

Closed
LuckVd wants to merge 3 commits into
kirodotdev:mainfrom
LuckVd:fix/21070-monitor-watch-review-wakes
Closed

LuckVd wants to merge 3 commits into
kirodotdev:mainfrom
LuckVd:fix/21070-monitor-watch-review-wakes

Conversation

@LuckVd

@LuckVd LuckVd commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Make monitor_watch wake its session when a pull request it watches finishes its checks green, or receives a comment or review from a person, instead of staying silent while the PR only waits for review.

The structured GitHub pull-request monitor never woke its session during a 3-hour observe-only watch of PR #20852: review lanes posted CONCERNS while checks ran, then 112 checks went green, then a maintainer commented — the monitor stayed silent through all three events, while a plain timer loop woke 32 times. Three causes were confirmed on main (0600e2804): more than 100 passing checks keep the check board incomplete forever (_bounded_checks in src/kiro_crew/monitoring/github_pull_request.py, canonical_pull_request_facts in src/kiro_crew/monitoring/pull_request.py); a PENDING reading never wakes (_decide_effect in src/kiro_crew/monitoring/decision.py); and the comment read carried no author, so a person cannot be told apart from a bot and conditions were attached only to ACTIONABLE readings.

Why it matters

The monitor exists so a session can stop polling and still hear about its pull request. Anyone watching a heavily-checked PR (the very PRs that benefit most from watching) hears nothing during the two moments the author must act: when the PR becomes ready for review and when a maintainer asks for something. The author falls back to a timer loop, which wakes far more often and costs more — exactly the waste the monitor was built to remove.

Not a goal

What changed (motivation → approach → change)

Observed symptom → root causes → change:

  1. Verdictless overflow poisoned the board. _bounded_checks marked the board incomplete when any state bucket exceeded the 100-row cap, so a PR with 112 passing checks was PENDING/checks_incomplete forever. Change: only truncated verdict-bearing buckets (failed/pending/unknown) mark incompleteness; extra passing rows cannot change the verdict, so bounding them silently is safe (src/kiro_crew/monitoring/github_pull_request.py:_bounded_checks, src/kiro_crew/monitoring/pull_request.py:canonical_pull_request_facts).
  2. A PENDING reading with news had no path to a wake. Change: _decide_effect routes a PENDING observation that carries named conditions through the same coalescing window as ACTIONABLE; a terminal SUCCESS carrying a fresh activity condition gets one wake before the terminal stop, then the condition is masked and normal success handling stops the watch. The stall count is untouched: pending ticks still never advance it (src/kiro_crew/monitoring/decision.py).
  3. The monitor could not see who spoke. The comment read selected nodes{body} only. Change: the core selection reads author{login}, the comments read gains an independent reviews connection (per-subject bound cursor, same GraphQL-variable discipline), and two new digests keep only bounded fingerprints of activity by User accounts that are neither the PR author nor [bot] accounts. Missing author data fails closed for the new digests; the legacy all-bodies digest (bot in-place verdict edits still wake) is unchanged (src/kiro_crew/monitoring/github_pull_request.py).
  4. Condition plumbing. pull_request_conditions is now status-aware: ACTIONABLE keeps every existing blocker; PENDING adds checks_complete (only when checks are complete with none failed/pending/unknown, review still required, non-draft open PR) plus the human comment/review activity conditions; terminal success carries only activity that arrived with the transition. MAX_MONITOR_FIXED_CONDITIONS 5 → 7 because six fixed keys can now co-occur (src/kiro_crew/monitoring/pull_request.py, src/kiro_crew/monitoring/models.py).

Backwards compatibility

Compatible: no field, kind, key or validator is rejected, renamed or removed. The two new digest fields are absent from the canonical shape when empty, preserving the established projection; the new conditions only add wakes that did not exist before, and every existing wake path is byte-for-byte the same for ACTIONABLE subjects.

Removes nothing: the change only adds read fields to two GraphQL selections, adds two digest inputs, and widens wake routing; no reader of any existing surface changes behavior.

Tests

New test/test_issue_21070_monitor_wakes.py:

  • test_green_checks_waiting_for_review_wake_once_and_do_not_stall — checks finishing green on a review-required PR wake exactly once; 14 further ticks are NO_CHANGE with stall_streak == 0 (no repeated wake, no stall trip).
  • test_human_comment_on_a_pending_pr_wakes_once_but_own_and_bot_comments_do_not — a human comment wakes once, an in-place edit wakes again, the PR author's own and a [bot] comment produce none.
  • test_human_review_wakes_a_pending_or_newly_ready_pr — a human review wakes a pending PR once and also surfaces on the success transition before the terminal stop.
  • passing-check overflow stays complete while failed/pending overflow keeps the incomplete sentinel.

test/test_github_pull_request_monitor.py: test_more_than_one_hundred_passing_checks_still_yields_a_complete_board (112 green checks → complete board, single checks_complete condition), independent comment/review cursor paging, human-vs-bot-vs-author digest separation, bot in-place edits still changing the legacy digest.

Verification run: 34 monitoring test files — 1077 passed, 300 skipped (marker-isolated integration/real_network), exit 0; focused run 6 passed; wheel build from the fix tree exit 0; git diff --check origin/main...HEAD clean.

Manual verification

N/A — unit coverage sufficient: every wake path is exercised through decide_monitor with real MonitorState transitions, and the GraphQL reads are exercised against recorded envelope fixtures.

Related Issues

Fixes #21070

Pattern harvest

Rule candidate: review-prompt

Pattern: a completion signal (check board) computed with an overflow sentinel treats verdictless rows as verdict-bearing, so scale alone (many green checks) permanently suppresses the "ready" state it exists to report.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@LuckVd
LuckVd requested review from a team and KiroCrewCaptain as code owners October 16, 2026 04:25
@LuckVd
LuckVd requested a review from CrysisDeu October 16, 2026 04:25
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 16, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor

Thanks for picking up #21070 so quickly.

I had started on #21070 too. I will not open a second PR, so the fix stays here. I ran a set of scenario tests against head 86e664b and found these gaps. Each test drives the real GitHub provider with fake GraphQL answers, then build_pull_request_probe_result and decide_monitor, one tick at a time (120 s cadence). The 11 existing related test files all pass on this head.

  1. A watch armed on a PR that is already green wakes on its first reading.
    Input: open, non-draft PR, every check passed, review required. Arm the watch and tick 4 times.
    Expected: no wake. Nothing happened during the watch.
    Observed: a wake on tick 1. The same happens when the PR already has an old comment by a person, and when that old comment first becomes readable after a failed comment read.
    Where: pull_request_conditions in pull_request.py names checks_complete and human_comment_activity:<digest> on every PENDING reading. They are levels, not edges, so the first reading always finds them unmasked.

  2. A PR waiting for review wakes again every 6 hours.
    Input: checks settle, a person has commented, then the PR waits for review for 25 hours (runtime budget 24 h).
    Expected: one wake, then quiet until something new happens.
    Observed: wakes at ticks 1, 3, 181, 183, 361, 363, 541, 543. When DEFAULT_MONITOR_REALERT_SECS runs out, both level conditions re-assert.
    Where: same conditions, through _coalesce_actionable in decision.py.

  3. A new head with an empty board counts as settled, and the real settle is then missed.
    Input: green PR, force-push. The first reading of the new head has no checks yet (GitHub often reports none for a moment after a push). Next reading: checks pending. Then all pass.
    Expected: one wake, when the checks really finish (tick 4).
    Observed: wakes on tick 1 (the first green reading) and tick 2 (the empty board). No wake when the checks really finish, because checks_complete on that head is already masked.
    Where: the checks_complete branch of pull_request_conditions treats an empty board as complete.

  4. Checks finishing on a PR that waits on something other than review never wake.
    Input: a repo with no required reviewers (reviewDecision null), mergeStateStatus BLOCKED. Checks go from pending to all passed.
    Expected: one wake. The issue asks for a wake when checks finish, and mergeability_pending is one of the PENDING reasons it names.
    Observed: no wake. Reason stays mergeability_pending.
    Where: the checks_complete branch requires review_decision == "review_required".

  5. One comment from a deleted account turns off person wakes for the whole PR.
    Input: the PR has one comment whose author is null (a deleted user). Later a person posts a comment.
    Expected: one wake on the new comment.
    Observed: no wake.
    Where: _human_activity_fingerprint in github_pull_request.py returns "incomplete" for a null author. _pr_comments then drops the whole human digest to "" on every read.

  6. An existing wake is now held one coalescing floor.
    Input: checks settle (wake), and a check fails on the very next reading.
    Expected: the failure wakes at once, as it does on main when no window is open.
    Observed: the failure wakes one tick later. The settle wake left its window open, so the new failure waits out DEFAULT_MONITOR_COALESCE_SECS. A settle that lands while a person-comment window is open is held the same way.
    Where: the PENDING branch in _decide_effect routes the new conditions through _coalesce_actionable.

  7. The contract text still says pending probes never wake.
    The monitor_watch description in src/kiro_crew/mcp_tools/control.py says the session "is woken only when a new revision needs action; unchanged, pending, retry, and terminal probes use no agent turn". src/kiro_crew/docs/monitoring.md (line 5) and src/kiro_crew/docs/monitor-loops.md (line 100) say the same.

If they help, the scenario tests are here: https://github.com/kirodotdev/KiroCrew/blob/3deb0aa57b9080891c5ac0a5f16f82fcb775e8ba/test/test_monitor_watch_settle_wake.py . The fake GitHub host in that file answers each GraphQL document by what it selects, so it should run against this branch after a small change to the reviews answer: add pageInfo and state.

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision fe043671a68b0f2753ecf3af186ed5999dd3924d touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — 🔴 BLOCK (blocking)

Design-level review of fe043671a68b0f2753ecf3af186ed5999dd3924d via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: BLOCK

Untested against live GitHub: the new reviews/author{__typename} query, joined into the legacy comment read, has only hand-written fixtures and "Manual verification: N/A".

Blockers

"Manual verification: N/A" on a path only live GitHub exercises

  • The description says "N/A — … the GraphQL reads are exercised against recorded envelope fixtures", and it also says the 300 skipped tests are the real_network ones.
  • _PR_COMMENTS_SELECTION now adds reviews(first:PAGE_SIZE,after:$REVIEW_CURSOR){…author{login __typename}} to the same query that feeds the legacy digest.
  • If GitHub rejects that query, or prices it differently, payload is None sets comments_complete=False. The existing review_comment_bodies: wake then goes silent along with the new wakes, and it does so without any error.
  • That would break an existing wake, which the PR's "Not a goal" promises to keep. The bug was found on a live 3-hour watch; the fix is checked only against fixtures written by hand.
  • Fix: run monitor_watch once against a real PR and record the result.

Clears when: the description records a live probe against a real PR with >100 green checks plus a human comment/review, showing the checks_complete and human_*_activity wakes and a non-empty pr_comment_body_digest

Watch

Old human activity wakes like new activity

  • The activity keys are built from aggregate digests over all human comments and reviews ever posted, and coalesce_alerted starts empty.
  • So the first tick of a new watch wakes on any existing human comment. A PR approved by a person always has a human review, so a review_ready watch wakes once before STOP_SUCCESS.
  • Masked keys also come back after DEFAULT_MONITOR_REALERT_SECS, so nothing new still triggers a wake.
  • The description's "activity that arrived with the transition" has no code behind it.

Clears when: activity digests seen on the first observation (or carried over from previous_observation) don't wake, with a test showing a watch started on a PR that already has human comments records it and does not wake

[DESIGN-REVIEWED] fe04367

Description read

[DESCRIPTION-READ] 39bd33e149624e7a753df6356c76ee881f697c5e0d9c1de6ba7bfe01687e5cac

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — 🔴 BLOCK

Premise-level review of fe043671a68b0f2753ecf3af186ed5999dd3924d via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: BLOCK

product-shape change without accepted RFC: a successful PR now costs a model turn, which the implemented monitor RFC forbids.

Not justified as shipped

  • Item 4 — oversized: the SUCCESS and observation.conditions branch returns WAKE_ACTIONABLE. The description says it fires on "only activity that arrived with the transition", but pull_request_conditions attaches human digests for every status. So arming a watch on an already-approved PR with old human comments wakes the session.
  • Item 5 — duplicate of review_comment_bodies (pull_request.py): on ACTIONABLE, a human comment now moves two keys from the same connection. That duplication is part of why MAX_MONITOR_FIXED_CONDITIONS grew from 5 to 7.
    Subtraction: emit human_comment_activity only when the status is not ACTIONABLE.
    Clears when: the ACTIONABLE emission is removed.
  • Item 7 — zero consumers: after this change, nothing in src/ calls _pr_comment_page. The old caller at github_pull_request.py:904 is replaced; only tests import it.
    Subtraction: delete it and point TestRetentionIsBounded at _pr_comment_activity_page.
    Clears when: the function is deleted.

What this change ships

Inventory (7 items) — 4 justified

Intent: let a session hear when its watched PR goes green or a person comments or reviews. This is a FIX.

  1. More than 100 green checks no longer leave the board stuck incomplete — justified
  2. A pending PR wakes once when all checks finish green — justified
  3. A person's comment or review wakes a pending PR — justified
  4. A review that turns the PR successful wakes the session before the watch stops — oversized
  5. Human comment activity also wakes ACTIONABLE PRs, on top of the existing body digest — duplicate of review_comment_bodies
  6. The PR author's login is read on GraphQL and REST — justified
  7. _pr_comment_page is kept in the new pages module — zero consumers

Blockers

  • product-shape change without accepted RFC: success now wakes the session. rfc-token-efficient-monitors.md has status implemented on the base, and §5 says "success or deterministic terminal state: zero model turns". Today STOP_SUCCESS just stops the watch (monitor_records.py:522-535).
    Subtraction: delete the SUCCESS branch in _decide_effect and send activity conditions only with PENDING. Item 3 already handles reviews that leave the PR pending.
    Clears when: an RFC covering a success-time wake is merged non-draft on the base (then push or rebase), or a maintainer posts /ai-review override first-principles fe043671a68b0f2753ecf3af186ed5999dd3924d: <decision>.

[FIRST-PRINCIPLES-REVIEWED] fe04367

Description read

[DESCRIPTION-READ] cd417708ca6492e41ffe9a11509a5c39dcc9cceb6f92bb78a266caf65838d0c1

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed fe043671a68b0f2753ecf3af186ed5999dd3924d via the fork AI-review pipeline; updated in place on each push.

Review details

No findings that block the merge.

FINDING — docs/system-specs/modules/monitor-architecture.md:1262 — the PR adds human comment/review activity digests, wakes from PENDING and SUCCESS conditions, and stops counting passing-check overflow as an incomplete board, but changes no spec. The owning spec still says a structured observation carries "a digest over the PR-level comment bodies, and nothing else", which AGENTS.md requires be updated in the same commit → Fix: in this commit, document in monitor-architecture.md the human_comment_activity/human_review_activity/checks_complete conditions, how they are deduplicated and reset, and the passing-overflow rule.

[OPUS-REVIEWED] fe04367

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — 🔴 changes requested (blocking)

Reviewed fe043671a68b0f2753ecf3af186ed5999dd3924d via the fork AI-review pipeline; updated in place on each push.

Adjudication upheld 1 of 1 blocking finding(s); those must be resolved before merging.

BLOCKING -- src/kiro_crew/monitoring/github_pull_request.py:1362 -- Passing-check truncation loses its overflow accounting

if state != "passed" and len(matching) > MAX_MONITOR_CHECK_IDENTITIES_PER_BUCKET:
112 passing checks -> _checks -> _bounded_checks -> _build_result -> persisted canonical snapshot lists only 100 checks with checks_complete=True, without a count or marker identifying the 12 discarded rows.
Anchor: a-bound-bounds-every-field-it-retains
Fix: Preserve and report the passing-bucket overflow count through the canonical snapshot while keeping verdict completeness true; do not restore the pending-producing incompleteness sentinel.

[BLOCK-MERGE] fe04367
[GPT-REVIEWED] fe04367

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F1 is an UPHOLD under the AUTOSDE blocking rule. The finding cites a-bound-bounds-every-field-it-retains. In the base-branch snapshot .review-base-rules/AUTOSDE.yaml:1275-1276, that rule has blocking: true and its file pattern src/kiro_crew/**/*.py covers this file. Line 1302 of the rule says overflow must be "COUNTED and said out loud once per snapshot". The changed guard in _bounded_checks (patch line 356) still cuts the passing bucket to the first 100 entries ([:MAX_MONITOR_CHECK_IDENTITIES_PER_BUCKET]), but it no longer flags or counts what was dropped. The same is true of canonical_pull_request_facts (patch lines 874-878). Because the rule's flag is authoritative, the remedy cost is not weighed here.

[ADJUDICATION] fe043671a68b0f2753ecf3af186ed5999dd3924d total=1 uphold=1 downgrade=0
UPHOLD F1 src/kiro_crew/monitoring/github_pull_request.py:1362 reason=autosde-blocking-rule
[GPT-ADJUDICATED] fe043671a68b0f2753ecf3af186ed5999dd3924d

@LuckVd
LuckVd force-pushed the fix/21070-monitor-watch-review-wakes branch from 3b4649d to fe04367 Compare October 18, 2026 17:28
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 18, 2026
A review-only watch could only hear a new revision, so a pull request
parked at "review required" stayed silent while its checks went green
or a person commented or reviewed. The GitHub adapter now digests
comments and reviews by people other than the PR author beside the
existing all-comment body digest, a pending or terminal-success PR
carries those signals as named conditions, and the decision layer
routes exactly those signals through the same coalescing window, one
wake per re-alert interval. A watch's first readable observation
records whatever signals it sees as a baseline, so activity that
predates the watch never reads as new.

Passing-check overflow is cut and carried as its own marker beside the
verdict instead of being folded into board completeness, and the paging
and fingerprint readers move whole into
monitoring/github_pull_request_pages.py to hold the adapter under the
2,000-line cap.
Address the changes-requested review and the scenario gaps:

- keep passing-row overflow evidence: the adapter reports the cut as
  its own flag, the canonical carries `checks_passed_overflow` beside
  the untouched verdict, and monitor_inspect names `passed_incomplete`
  beside the count, so a saturated passed list is never read or counted
  as the whole list (blocking finding at github_pull_request.py:1355);
- a refused reviews-only continuation invalidates only connections the
  round never finished, so a completed comment read keeps its bot-edit
  and human-activity wakes through a reviews-only rate limit
  (github_pull_request.py:922);
- the contract text says the named signals wake once per re-alert
  interval, which is what the engine does (control.py, monitoring.md,
  monitor-loops.md, decision.py);
- a watch armed on a PR that is already green or already approved
  baselines its first readable observation instead of waking on the
  pre-existing level, in decision.py with tests;
- an empty check board is no longer "all checks completed
  successfully", and checks finishing wake the session even when the PR
  waits on something other than review (no review_required gate);
- a deleted account's comment is fingerprinted unattributed instead of
  blanking the human digest, so one deleted account cannot silence
  person wakes for the PR's lifetime;
- delete `_pr_comment_page`, whose only remaining callers were tests,
  and point TestRetentionIsBounded at `_pr_comment_activity_page`;
- record in docs/system-specs/modules/monitor-architecture.md which
  statuses carry conditions and why.
@LuckVd
LuckVd force-pushed the fix/21070-monitor-watch-review-wakes branch from fe04367 to e33d365 Compare October 23, 2026 19:21
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 23, 2026
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 6, 2026
@LuckVd

LuckVd commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Closing this to keep the queue tidy: it has conflicts with the current main, CI is no longer green, and there has been no movement for several days. Happy to reopen or resend a fresh rebased fix if this is still useful — thanks!

@LuckVd LuckVd closed this Nov 8, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

monitor_watch never wakes its session on a pull request that only waits for review or for a person's comment

2 participants