Skip to content

fix(autonudge): wake when the judge sheds a fresh row instead of judging it quiet - #16192

Merged
bolichen97 merged 2 commits into
mainfrom
fix/16151-judge-cursor-lag
Oct 3, 2026
Merged

bolichen97 merged 2 commits into
mainfrom
fix/16151-judge-cursor-lag

Conversation

@chenmingwei23

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a judged auto-nudge loop wakes on the tick a worker writes DONE:, BLOCKED: or a question, however busy that worker is.

The wake judge reads each chat-* target 12 rows per tick, starting at its cursor. A worker that writes more than 12 rows between ticks (tool rows count) leaves the cursor further behind every tick, so its final DONE: line is never offered to the judge. The judge answers quiet, and the loop wakes only when the quiet-streak floor (_MAX_QUIET_STREAK = 10) fires: five hours at a 30-minute interval. Two later caps lose rows the same way: screen_evidence keeps at most 40 items and build_state at most 8000 characters, and both shed fresh rows the cursor has already passed, so those rows are never offered again either. Reported in #16151.

Why it matters

A conductor patrolling workers with a judge sleeps through finished, blocked and questioning workers for hours, and nothing tells it a row was lost: the tick reads as an honest quiet.

Not a goal

  • Moving judge evidence off transcript reads onto the crew-log bus (see Other suggestions).
  • Changing the work-ledger, PR or plain-timer paths; they do not read transcripts.
  • Changing the 12-row page, the 40-item cap or the 8000-character budget.
  • Changing the quiet-streak floor.

What changed (motivation -> approach -> change)

Observed symptom -> root cause -> change.

The cursor promises every row is screened once, while the three caps drop rows past the cursor without saying so. Only the scrub path (scrub_fresh) and an unreadable target already wake on a loss. This PR applies that one rule to the other three: a fresh row that does not reach the judge wakes the tick.

  • Read to the end. collect_evidence reads a session target again from next_since while it is short of the transcript's total, up to MAX_PAGES_PER_TARGET pages (3, derived as MAX_EVIDENCE_ITEMS // MAX_ROWS_PER_TARGET, so the walk never exceeds what the item cap can carry). A target still behind after the last page is counted as unread, which fires the tick. A page that refuses keeps the pages already read. The gateway's _read_session returns total beside the rows and cursor.
  • Shed seen rows first. screen_evidence (40 items) and _shed_index (8000 characters) drop items marked seen on an earlier tick before any fresh item. A seen item is context the judge already answered about; a fresh session row is never offered again.
  • Count and wake. Both caps count the fresh items they still drop as shed_fresh. judge_tick wakes when shed_fresh > 0, through the same exit scrub_fresh uses.
  • rfc-wake-judge (on main, status: partial) gains the paging rule in the session evidence entry and the seen-first order plus the shed_fresh wake in the shed-order paragraph.

Backwards compatibility

Compatible: the only read_session caller is the gateway's _read_session, changed in this PR to return the extra total. A loop whose evidence fits its caps judges exactly as before. A loop whose fresh evidence does not fit wakes where it judged quiet.

Removes nothing: no field, route, record or capability is removed; the caps still shed, only the order and the wake on a fresh loss change.

Tests

New TestNothingNewIsLostAndJudgedCalm in test/test_autonudge_wake_judge.py:

  • test_a_busy_target_is_paged_to_its_end_so_the_last_line_is_read: 29 new rows, the last a DONE: line, are all collected in one tick.
  • test_a_target_still_behind_after_the_page_budget_is_dropped: a target past the 3-page budget counts as unread and the tick falls back to a wake.
  • test_a_page_that_refuses_keeps_the_pages_already_read: a refusal on a later page keeps the earlier pages.
  • test_the_page_budget_stays_inside_the_points_item_walk: pages x rows never exceeds the 40-item cap.
  • test_the_item_cap_sheds_seen_items_before_fresh_ones and test_the_item_cap_counts_every_fresh_item_it_sheds.
  • test_the_char_budget_sheds_seen_items_first and test_the_char_budget_counts_a_fresh_item_it_sheds.
  • test_a_busy_worker_whose_rows_do_not_fit_fires_the_tick: a fresh shed wakes judge_tick.
  • test_shedding_only_seen_context_does_not_take_the_shed_exit: dropping only seen context stays quiet, so a busy loop does not wake every tick on old rows.

test/test_autonudge_wake_judge.py: 305 passed. With the six other suites that touch the judge and nudge-wake point: 946 passed. mypy, flake8, black, isort on the changed files: clean. scripts/docs_lint.py: passes.

Manual verification

N/A -- the judge has no UI; the tests drive collect_evidence, screen_evidence, build_state and judge_tick with real row shapes.

Related Issues

Fixes #16151

Other suggestions

The judge polls transcripts for session evidence. #16149 (merged) gives the crew-log bus keyed subscriptions with a late-join baseline. Session evidence could later come from a keyed session subscription instead of a transcript poll; that is a design change for rfc-wake-judge, not this fix.

Pattern harvest

Rule candidate: review-prompt
Pattern: a bounded read or cap that drops rows a cursor has already passed must report the loss, never let the caller treat the result as complete.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

…ing it quiet

The wake judge read each session target 12 rows per tick from its cursor, so a
worker writing faster than that left the cursor behind and its DONE line was
never screened until the quiet-streak floor fired. The 40-item and 8000-char
caps also shed fresh rows silently, oldest first.

A session target is now paged to its end, up to 3 pages a tick; both caps shed
already-seen items before fresh ones; and any fresh item a cap still drops, or a
target left behind after the last page, wakes the tick.

Fixes #16151
@chenmingwei23
chenmingwei23 requested a review from a team as a code owner October 2, 2026 16:33
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — 🟡 CONCERNS

Design-level review of a13369ef1a1bd35e45cf480a6e708af3ebf3bc6f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The second commit wakes every new judged loop on its first tick: any target with more than 12 rows of history fires, and the description never mentions it.

Watch

  • First-tick wake is not in the description. The cause is the new skipped branch (if page == 0 and cursor == 0 and next_cursor - len(rows) > 0: skipped = True). With no stored cursor, the gateway reads since or None, which serves only the tail. Any older rows then count as a drop, and judge_tick returns FALLBACK. As a result, arming a loop on a busy worker, adding a new target, or clearing a cursor through cursor_unavailable wakes the owner once. The rows that trigger it were written before the loop was watching, so they are not a delta the cursor ever promised to screen. The description only describes paging from next_since and the shed_fresh cap, and its Backwards-compatibility line ("A loop whose evidence fits its caps judges exactly as before") does not hold for a new loop.
    Clears when: the description names the first-read wake and says why history from before arming counts as a loss, or the first cursorless read just sets the cursor without counting a drop.

[DESIGN-REVIEWED] a13369e

Description read

[DESCRIPTION-READ] f65f67b7d051068caa0d33f2f5efdeb69a98e981934b8c51430c7eafde1fbb79

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of a13369ef1a1bd35e45cf480a6e708af3ebf3bc6f — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The second commit's "skipped" rule makes every first judged tick wake once a target has more than 12 rows, so a reset now forces a turn, not just a re-read.

Not justified as shipped

  • Item 8 — undeclared, inherited, oversized: the description never mentions it. The Goal is rows a worker writes between ticks, but this hunk counts rows written before the loop was armed as lost: if page == 0 and cursor == 0 and next_cursor - len(rows) > 0: skipped = True. Four sites reset judge_cursors = {} and say "Resetting costs at most one re-read": autonudge_service/judge_tick.py:230,270 and mutations.py:608,661. With this rule, each reset now forces a FALLBACK fire whenever a target has more than 12 rows of history. The gateway's since=since or None tail start was the "start from now" choice, and the RFC text this PR edits does not cover the change.
    Subtraction: delete skipped, its branch in collect_evidence, and the two test_a_tail_read_* tests (all from commit a13369e).
    Clears when: the rule is removed, or a linked report shows a defect from pre-arming rows going unread.

What this change ships

Inventory (8 items) — 6 justified

Intent: a judged loop wakes on the tick a busy worker writes DONE:/BLOCKED: or a question. This is a FIX (#16151).

  1. A busy target is read up to 3 pages per tick until it catches up — justified
  2. A target still behind after 3 pages wakes the tick — justified
  3. The 40-item cap drops already-seen items before fresh ones — justified
  4. The 8000-character budget drops already-seen items before fresh ones — justified
  5. A fresh item dropped by either cap wakes the tick (shed_fresh) — justified
  6. The gateway reader also returns the transcript total — justified
  7. The judge trace gains a shed_fresh count — undeclared
  8. The first tick after arming, a brief edit or a retarget wakes whenever a target has more than 12 rows of history — undeclared, inherited, oversized

[FIRST-PRINCIPLES-REVIEWED] a13369e

Description read

[DESCRIPTION-READ] 07cc8e0bde8405fe616500284b1ca09c90ec20a761b43003634292e62072c473

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of a13369ef1a1bd35e45cf480a6e708af3ebf3bc6f and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] a13369e

False positive or not applicable? A repository writer can comment:
/ai-review override gpt a13369ef1a1bd35e45cf480a6e708af3ebf3bc6f: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed a13369ef1a1bd35e45cf480a6e708af3ebf3bc6f — this comment is updated in place on each push.

Review details

Advisory finding only; nothing blocks.

FINDING — src/kiro_crew/autonudge_judge.py:856 — The first tick always fires on a target with more history than one page (more than 12 rows), or with a trimmed prefix, whenever no cursor is stored. That covers every fresh arm, every brief update (mutations.py:661 sets loop.judge_cursors = {} with the promise that this "lets the next tick judge those rows"), and every retarget. The gateway does a tail read via since or None, so start = base + offset > 0 and next_cursor - len(rows) > 0 sets skipped = True. That forces a FALLBACK turn saying "could not read every target", even though the read succeeded and the skipped rows came before the watch began, so the judge is never asked on that tick. → Fix: count a tail read as skipped only when a cursor had been stored for this target and was then cleared, for example by the cursor_unavailable path. Keep a marker for that case instead of keying on cursor == 0.

[OPUS-REVIEWED] a13369e

Verdict parsed from the review's SHA-scoped output markers for commit a13369ef1a1bd35e45cf480a6e708af3ebf3bc6f.

False positive or not applicable? A repository writer can comment:
/ai-review override fable a13369ef1a1bd35e45cf480a6e708af3ebf3bc6f: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 2, 2026
A first read with no cursor starts at the tail and never sees the rows
before it, so a target with more history than one page is now reported
as behind and wakes the loop instead of being judged quiet. Drop issue
numbers from the comments the comment-history gate flags.
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 3, 2026
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of a13369ef1a1bd35e45cf480a6e708af3ebf3bc6f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: PASS

Nothing to check.

[UX-REVIEWED] a13369e

Description read

[DESCRIPTION-READ] 805163fd5a21b892c49b40659da4ec6ccb5d8d42d93965e4e16b37d2a3928a65

That sha256 names everything this verdict read: the PR description, plus the 43 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 3, 2026
@bolichen97
bolichen97 enabled auto-merge (squash) October 3, 2026 05:20
@bolichen97
bolichen97 merged commit e4ea90b into main Oct 3, 2026
97 checks passed
@bolichen97
bolichen97 deleted the fix/16151-judge-cursor-lag branch October 3, 2026 05:20
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

wake judge: session-target cursor advances at most 12 rows per tick, so a busy worker's DONE line is never read until the quiet-streak floor

2 participants