Skip to content

feat(monitor): the gh-pr reader fetches, the wake judge decides - #13759

Merged
bolichen97 merged 2 commits into
mainfrom
feat/gh-pr-probe-pure-fetch
Sep 26, 2026
Merged

bolichen97 merged 2 commits into
mainfrom
feat/gh-pr-probe-pure-fetch

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

What is the problem?

The gh-pr probe both read a pull request and decided whether the tick deserved a
model turn. It classified the board against its own notion of actionable -- a
failing check absent from known_reds, every lane green when wake_on_green
allowed it, a conflict -- and emitted a wake from that. One module holding the
reading and the judgment has three consequences.

  • The notion is not the owner's. Two loops watching one board want different
    things: one waits for a reviewer, one waits for a rebase to settle. The probe
    answers the same question for both.
  • The signal that matters most on a review round is unreachable by design. The
    probe reports that a comment arrived and deliberately never reads it, so a
    reviewer's ask -- which routinely sits in a comment body while the lane carrying
    it reports success -- cannot be a wake criterion at all.
  • The reading has no channel to the judge. Pull-request evidence comes from
    MonitorState.last_observation, written only by the structured controller's
    provider. A gated loop observes through the raise-based kernel, whose verdict
    carries prose and no facts, so the judge screens every such loop against an
    empty observation.

The transport is also thin: one gh pr view, one retry-free attempt, no
rate-limit handling, and a check rollup read as a bare array -- which carries no
count, so a truncated read cannot be detected.

Why this issue matters to the user

A wake criterion the owner writes in their own words is the feature
monitor_start advertises. With the probe deciding, wake_when: "a reviewer asks for a change" cannot be honoured: the judge is handed check tallies and a comment
fingerprint, so it answers about lanes while the reviewer waits.

The reverse failure is quieter. A rollup read short looks exactly like a small
clean board, and a rate-limited tick looks like a failed fetch. Both resolve
toward a reading the owner cannot tell apart from calm.

How our fix solves it

The reader fetches, the judge decides, and the core owns the irreversible steps.
Each component holds exactly one of those jobs.

  • The reader (probes/gh_pr.py) returns facts and nothing else. One
    _Transport per tick gives bounded retry with jittered backoff, honours a
    rate-limit window the response names and caps that wait, paginates check runs
    and commit statuses, and reconciles rows against total_count. Every call is
    bounded by what the tick has left. A row's identity is its workflow rather than
    an app slug, so two workflows sharing a job name stay two rows. A reading that
    fell short is partial, with incomplete naming what was missed.
  • The judge decides from the owner's criterion. nudge.wake carries the
    evidence kinds pr_state, pr_checks, pr_comment and pr_review; the
    default brief names a failing check, a reading that is not whole, and a new
    remark. The shed rule pins the two typed summaries and drops the oldest prose
    first, so a tight char budget gives up a comment body rather than the board.
  • The core (autonudge.py) owns what cannot be undone.
    observation.is_terminal maps to the existing deactivation with merged and
    closed kept distinct; the reading is persisted before it is published, so a
    refused write publishes nothing and fires; the judge baseline is staged and
    committed under the lock, and only while the brief it was earned under still
    stands. The remark delta is computed here because the fetcher holds no memory
    between ticks.
  • The script cron is retired. babysit/scripts/pr_watch.py is deleted, and a
    cron registered against a deployed copy of it is refused with its job KEPT
    rather than silently removed.

What tests we did

Local, scoped, -n0: 837 passed across the eight test files this diff owns --
test_autonudge_wake_judge.py 259, test_autonudge.py 216,
test_wake_judge_feedback.py 175, test_gh_pr_fetch.py 80 (new),
test_autonudge_stop_auth.py 59, test_review_repair_routing_skill.py 20,
test_irq_port_baseline.py 15, test_monitor_start_ack.py 13.
test_babysit_pr_watch.py is deleted with the driver it tested; its surviving
invariants live in the files above. New coverage is one case per behaviour across
the transport, the terminal mapping, the shed rule, the persist-before-publish
order and the baseline lifecycle and the retained board's bounds, each pinned in both
directions.

Measured rather than assumed: a 90-check board carrying five comment bodies
renders to 4,371 of MAX_STATE_CHARS 8,000 as 7 evidence items, with nothing
dropped and all five bodies intact. Summarising the board into one pr_checks
item is what makes that fit, so no ceiling is raised.

Prose reaches the judge through an in-memory per-loop stash, capped at 64. A
stash dropped to hold that cap is counted and logged, and makes that tick read
as not whole rather than as a remark with nothing in it, so the tick fires.

Gates run locally on this tree: flake8, isort, the baselined black gate,
comment-history, and mypy over 1,726 files -- whose only four complaints are
boto3-stub argument types in two files this diff does not touch. The full backend suite is
left to CI.

Any other suggestions on the work

  • The kernel's dedupe and coalescing have nothing left to act on. A fetcher
    raises no observations, so those layers idle. Collapsing them is 1,160 lines of
    kernel and 1,449 of tests with its own review surface, so it is out of scope.
  • The remark delta is reported, not acted on. first_seen_this_tick is
    rendered for the judge and read by nothing in the core. That is the line this
    design draws, and the first consumer that branches on it has rebuilt the
    comparator this change removes.
  • A window unreadable across many ticks is still a gap. An isolated 5xx is
    absorbed inside the tick, but the consecutive-failure backstop counts readings
    that never reached the subject, not ones that came back short, so a sustained
    run of partial raises no alarm. Coalescing that is a follow-up.
  • monitoring/github_pull_request.py is untouched. monitor_watch stays
    provider-decided and its reader still produces a comment fingerprint rather than
    bodies, so the collector keeps rendering that digest -- the only discussion
    signal that path has.

no linked issue: this is feature work, and the repository convention reserves
issues for reported problems -- the context lives in this description.

@chenmingwei23
chenmingwei23 requested review from a team and dwu96 September 25, 2026 08:47
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 3277d9545c48e5de47982b15623200a618c4b1be and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 3277d95

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 3277d9545c48e5de47982b15623200a618c4b1be: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 3277d9545c48e5de47982b15623200a618c4b1be — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

A third-order loss ledger (_PR_BODIES_FORGOTTEN globals) and two exports nothing calls ride inside an otherwise RFC-covered reader/judge split.

Not justified as shipped

  • Item 9 — oversized, undeclared: the description declares the stash cap and its dropped-notice ("counted and logged"), but not the second store's own overflow machinery — _PR_BODIES_FORGOTTEN / _PR_BODIES_FORGOTTEN_TAKES plus the inner eviction loop, reachable only after 64+ loops are each abandoned between fetch and judge twice over, and whose firing makes every gated loop on the process read short until a takes counter drains.
    Subtraction: delete both globals and the nested while len(_PR_BODIES_DROPPED) > MAX_BODY_STASHES loop; _PR_BODIES_DROPPED holds loop-id keys only, popped on take, so it needs no second cap.
    Clears when: the deletion is made, or a counted path to >64 concurrently-abandoned stashes is named.
  • Item 10 — zero consumers: probes/gh_pr.py exports sanitize_body (line 254) and BUCKETS (line 177); grepping sanitize_body|BUCKETS across src/kiro_crew finds 0 callers of either — the module itself uses _sanitized_body_with_clip directly, and BUCKETS appears only in a docstring.
    Subtraction: delete sanitize_body and BUCKETS.
    Clears when: deleted, or a real caller lands with them.

What this change ships

Inventory (10 items) — 8 justified

Intent: let a watch owner's own criterion — including reviewer prose — decide when a PR watch wakes its session, by splitting fetching from judging. ADDITION, licensed by rfc-wake-judge.md (partial on base; this PR is its §8 PR G, listed in its implementation-prs).

  1. A gated PR watch's wake is decided by the judge against the loop's own criteria; the probe's fixed wake set is gone — justified
  2. Comment and review bodies now reach the judge (kinds pr_state/pr_comment/pr_review, per-tick in-memory stash) — justified
  3. The script cron driver is deleted; a registered copy is refused each run and auto-paused, kept listed — justified
  4. Watch keys known_reds, wake_on_green, coalesce_secs, note stop being read (ignored, not rejected) — justified
  5. Transport hardening: bounded jittered retry, rate-limit windows, counted pagination; a short read is partial, which fires — justified
  6. Merged-vs-closed watch ending moved to the core, from typed facts, the two kept distinct — justified
  7. Default brief rewritten; the untrusted-prose clause rides on every judge prompt — justified
  8. With no judge lane, any changed reading now spends the turn; only an unchanged whole reading is free — justified
  9. A process-global "forgotten loss notice" state making every loop's tick read short until a takes counter drains — oversized: undeclared third-order guard, blast radius is every gated loop
  10. sanitize_body and BUCKETS exported from the new fetcher — zero consumers (grep count: 0)

Capped at 10: also riding along are the ack/tool-description relabels (the old wake-set promise no longer holds, so derived), secondary rate/abuse detection added to the shared provider-error classifier, and unknown check conclusions bucketing as unknown instead of failing.

[FIRST-PRINCIPLES-REVIEWED] 3277d95

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 3277d9545c48e5de47982b15623200a618c4b1be — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

A permanently-partial reading disables gating forever — every tick fires, no alarm distinguishes it from real activity, and the backstop never counts it.

Watch

  • pr_target_is_unread treats partial as unread and "the tick fires on it," while the doc concedes a shared check name can be persistently unresolvable and the description concedes "the consecutive-failure backstop counts readings that never reached the subject, not ones that came back short." One such board converts a gated loop into a turn-per-interval loop indefinitely, a cost the owner never chose, visible only in debug logs. Fail-open is the right direction for one tick; a permanent condition needs the alarm.
    Clears when: the failure backstop (or an equivalent counter) coalesces a sustained run of partial readings into one report, or the follow-up the description promises is filed and linked.
  • Already-armed watches carrying the old typed prefs lose them silently: _parse_config "ignores" unknown keys, so wake_on_green (old default true) and known_reds vanish — the ready-on-green wake stops arriving and an inherited red now fires once. "Keeps working" claims compat while the semantics shift with no signal, unlike the script path's loud paused-job trace.
    Clears when: the upgrade note (or the arm-time ack for a loop whose stored config carries those keys) states the green wake now requires a wake_when, or legacy keys are folded into the brief.

Suggestions

  • Render the incomplete note into the wake the not-whole path fires, so a partial-driven wake names its cause to the owner instead of reading as PR activity.

[DESIGN-REVIEWED] 3277d95

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 3277d9545c48e5de47982b15623200a618c4b1be — this comment is updated in place on each push.

Review details

Advisory only — nothing blocks: one in-memory/on-disk baseline divergence on the failed-persist path.

FINDING — src/kiro_crew/autonudge.py:6709 — loop.judge_pr_seen = {} clears the in-memory baseline the next tick actually reads, while the comment ("leaves the baseline exactly as the stored record has it") and the operator warning ("leaving the stored baseline in force") both describe the disk copy; the pre-stage value was already overwritten at line 6693, so a failed _persist_judge_state makes the next tick stamp first_seen_this_tick=True on every remark in the horizon — the judge is told remarks it already answered are "(new since the last tick)" and spends a turn on them — instead of only the ones from the lost commit → Fix: capture prior = loop.judge_pr_seen immediately before loop.judge_pr_seen = staged_pr_seen and restore prior in the failure branch rather than assigning {}.

[OPUS-REVIEWED] 3277d95

Verdict parsed from the review's SHA-scoped output markers for commit 3277d9545c48e5de47982b15623200a618c4b1be.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 3277d9545c48e5de47982b15623200a618c4b1be: <one-sentence reason>

@chenmingwei23
chenmingwei23 force-pushed the feat/gh-pr-probe-pure-fetch branch 3 times, most recently from bf6767b to a732c5f Compare September 25, 2026 09:38
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 25, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/gh-pr-probe-pure-fetch branch from a732c5f to 3b93af2 Compare September 25, 2026 10:04
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 25, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/gh-pr-probe-pure-fetch branch from 3b93af2 to b72bb05 Compare September 25, 2026 10:21
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 25, 2026
@chenmingwei23

chenmingwei23 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=e588e44e9056 — duplicate of the structured provider's rollup read

Accurate about the fields, and the answer is that the two paths do not share a budget.

_ROLLUP_SELECTION does carry everything named: totalCount + pageInfo, StatusContext in the same rollup, and workflowRun{workflow{databaseId name}} inline with no per-run call. So GraphQL would answer the workflow identity this reader resolves.

The named reason it is not the one path: GitHub meters GraphQL in points and REST in requests, and the budgets are separate. The structured provider's own module says so beside its REST paths and ships a REST fallback for exactly that case, so the redundancy is deliberate there already. Measured twice during this pull request's own babysit: the GraphQL quota for this user was refused outright while REST core read 5000/5000, and the REST board read kept answering through the whole window. A gated loop moved onto points alone would have gone blind there.

The resolution cost that motivates the objection is now bounded to where identity is genuinely ambiguous: a workflow is resolved only when one check name appears under two runs, so an ordinary board spends zero extra calls, measured at 40 rows across 19 runs with no shared name.

Routing the structured provider and this reader through one fetcher is the right end state and is raised again as the follow-up item c7bd5d487e13. It is a cross-module change this change's scope excludes, and it is recorded as a follow-up rather than done here.

@chenmingwei23

chenmingwei23 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=4b89fb429b35 — four keys with zero consumers

Correct, and taken: the four keys are gone from as_facts.

Checked before deleting rather than on the finding's word. rate_limit_remaining and remark_horizon_secs have no reader anywhere in the tree. calls and retries appear elsewhere only as unrelated dict keys in profile normalisation, crew-log projection and the CI runner watchdog, none of which read this observation. The judge's own collector renders state, the check tally, comments and reviews, and touches none of the four.

They also do not belong there on their own terms: they describe the fetch rather than the subject, and this record is written every tick into the same budget the judge's evidence has to fit inside. They stay on the transport, where a log line can read them at the moment they are true.

@chenmingwei23

chenmingwei23 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=ac88157f146b — a wrong quiet leaves no trace of what was screened

Taken. Each remark on the durable reading now carries a short digest of its body.

The gap is real as described: bodies are held for one tick and written nowhere, so a confident wrong quiet left a record saying a remark existed and nothing about what it said, and nobody could check the verdict against the words it was reached on.

Each remark row now carries body_digest, twelve hex characters over the body, beside the body_chars, clipped, verdict and remarks_total already recorded. That identifies what was screened on every tick, quiet or not, without putting prose in the record, which is the same trade the structured reader's own comment fingerprint makes. A remark with no body gets no digest, since an empty body is not a thing to identify.

@chenmingwei23

chenmingwei23 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=c7bd5d487e13 — route the structured provider through the same fetcher

Agreed as the end state, recorded as a follow-up rather than done here.

The asymmetry is real: wake_when: "a reviewer asks for a change" is honoured on a gated prompt loop, because that path now carries comment and review bodies to the judge, and the structured watch has only a comment fingerprint to offer, so the same arming text means different things on the two drivers.

Closing it means routing the structured provider and this reader through one fetcher, which is a cross-module change this change's scope excludes: that provider module is deliberately untouched here. It is the same point as item e588e44e9056 seen from the other side, and the reason it is not a simple deletion of this reader is recorded there: the two paths are metered on separate budgets, and the GraphQL one was refused for this user during this pull request's own babysit while REST kept answering.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • Item 5 (span=d56449c0213b) — accepted as correct, with the boundary objection checked and unavailable, and ticketed.

The duplication is real and I verified both halves rather than taking them. connections/vendors/github/rate_limit.py exposes read_rate_limit returning a RateLimitSnapshot that carries limit, remaining, reset epoch, used, resource and retry-after — and a secondary flag set when either a retry-after header is present or the body names the limit. My transport re-reads three of those headers through its own _H_REMAINING, _H_RESET and _H_RETRY_AFTER, and re-derives the retry-after-wins precedence in _rate_limit_wait.

The item offers a second clearing path — a named boundary reason why probes/ may not import that module — and I checked whether one exists rather than reaching for it. It does not. The module imports dataclasses and typing and nothing else, its own docstring states it is pure with no I/O, and the package __init__ chain above it is light. There is no existing probes/ import from connections/, so there is no precedent either way, but absence of precedent is not a boundary: nothing about that module makes it unsafe or expensive to read from a probe.

Worth naming what this makes the third reader of, because that is the part that matters more than the line count. The same secondary-limit vocabulary appears in read_rate_limit's body check, in the shared provider-error classifier this change widened, and in this transport's own header handling. Three readers of one provider signal, and only one of them owns it.

Ticketed onto #14074, which already carries the shared classifier's half of exactly this problem, rather than opening a second issue: the remedy is one decision about who owns GitHub's rate-limit signals, and splitting it across two tickets invites two different answers.

Not fixed on this head. Reading a RateLimitSnapshot changes the transport's wait computation, which is the part of this change that governs how long a tick may block — so it needs its own pins on the precedence and on the bounded wait, not a rename. The verdict on this head is CONCERNS and advisory, and the board's one remaining blocker is unrelated, so a push for it would re-roll clean verdicts to move a boundary that is not wrong today.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • Item 4 (span=0a64e7dd065c) — accepted as undeclared, already ticketed, and the description half is the part I cannot take here.

The fact holds. _parse_config reads repo, pr and host and nothing else, so a watch armed with known_reds, wake_on_green, coalesce_secs or note keeps parsing and loses what those keys meant. The only place that is said is a docstring inside the diff — "keys this fetcher does not read are ignored" — which tells a reader of the code and not a reader of the description.

The item is right that this rides along rather than being declared, and right that it matters most for watches armed by an earlier build: their stored message carries knobs that configured the wake, and nothing re-aims them at the judge criteria that replaced them.

That consequence is already tracked. #14072 carries it as one of its clearing conditions — the behaviour change for already-armed loops stated where an owner sees it, or the docs saying existing watches must be re-armed — alongside the settled-green default the same retirement removed. Both are the same question about what an existing watch inherits, so they belong in one place.

What I cannot do here is the other clearing path, and it is worth being precise about why rather than calling it impossible. Declaring it means editing the pull-request description, and a body edit re-rolls every review lane exactly as a push does. This head has Design at PASS and First Principles at CONCERNS with one remaining fenced blocker; trading those verdicts to add a sentence would cost more than the sentence is worth, and the next authorized round carries it for free. This comment is the declaration on the record until then.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • F1 (span=57012a9f6703) — the adjudicator downgraded it, and the rule it anchors to is answered structurally rather than per field.

Two things dispose of this, and the first is the adjudicator's own: DOWNGRADE F1 src/kiro_crew/probes/gh_pr.py:441 reason=disproportionate-remedy. The blocking claim is not upheld.

On the substance, the anchor is a-bound-bounds-every-field-it-retains, and that rule is answered on this diff by a pin rather than by another field. Four earlier rounds each named one more retained string, and a per-field assertion passes for the fifth — so the answer is test_every_string_the_facts_retain_is_bounded, which walks the whole retained dict at any depth and refuses any string past the ceiling. It found three fields no reviewer had named. Any field added later is caught by a test nobody has to remember to update, which is the property a per-field fix cannot have.

The specific mechanism this finding describes is a different claim from the rule, and worth answering on its own terms. It says two hidden lanes can exchange states inside an over-full bucket while the retained facts and the digest stay identical, suppressing the tick. That is true of a bucket cut at a cap, and it is why the cut is said out loud: an over-full board carries checks:incomplete in the retained dict, the reading is not whole, and pr_target_is_unread answers true for it — so the tick fires rather than screening. The suppression the finding describes needs the reading to read as whole, and a clipped board does not.

The remedy it proposes — per-bucket overflow counts in the digest and the rendered summary — would change the digest for a board whose live lanes were all measured, which is the outcome monitoring/models.py names as forbidden for the displaced population and which an earlier round on this same file corrected in the other direction. Adding counts back would re-introduce it.

So: no code change. The rule is closed on this diff by the whole-dict pin, the adjudicator declined to uphold the finding, and the mechanism it names is already covered by the truncation being visible rather than silent.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • F2 (span=ffdbb95d5aa2) — the adjudicator flagged it as an edge case, the fence keeps it blocking, and the remedy it asks for is the design this line replaced.

The adjudicator reached this line and declined to uphold it, in its own words:

Reaching the premature clear needs ~129 loops abandoned between publish and take in one process plus a specific take ordering, and the worst outcome is one loop's prose missing from a single judge call, with the same bodies republished on the next tick and an unconditional floor delivery within ten quiet ticks, so the loss is delayed notification rather than unrecoverable data loss.

The stamp is [GPT-ADJUDICATED-FENCED] 45e90663d4c31d16c3e41216546f1be92bee0e6c, fenced=1 flagged=1. Because the finding is security-class the fence keeps it blocking regardless of that reading, and the only clearance path is a human override. I am not a repository writer and do not post one; this comment records the reasoning, and the override decision belongs to a human who verifies it independently.

Why I am not taking the proposed fix, which is the part that matters most here. It asks that each forgotten stash's identity be preserved until its owning loop consumes the loss notice. That is exactly the design this line replaced, and it has a failure the current one does not.

A loop can be removed while it still owns a notice. Neither removal site clears the judge's stash, and the judge exposes no cleanup entry point, so nothing ever takes that notice and the store never empties. When the forgotten state was spent only on the store emptying, one such notice pinned it permanently — and while it was pinned every other loop's take reported a loss, so every gated pull-request watch fired every interval for the life of the process. That is not a delayed notification; it is the cost promise of the whole feature inverted, permanently, for every watch on the gateway.

I found that by asking what bounded the state rather than from any review, proved it with a test that failed on the old line, and fixed it by bounding the state to a rotation of takes. test_one_stuck_notice_does_not_pin_every_loop_forever pins it, and both mutations — removing the backstop, and never advancing the counter — are caught.

So the two behaviours trade against each other. The finding's remedy prevents a case needing ~129 abandoned loops plus a specific ordering, self-healing on the next tick and floor-delivered within ten. The current line prevents a permanent, gateway-wide one. Where they conflict the bounded and recoverable loss is the correct side.

The residue the finding is right about is real and now tracked in #14113: a removed loop leaves its stash entry and its notice behind, which costs a bounded slot, up to one rotation of forced fires, and a slightly overstated loss counter. Clearing at the source — a cleanup entry point both removal sites call — is deferred there deliberately, because the backstop already removes the unbounded consequence and the coupling that fix introduces is a decision worth making outside a review round.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • FINDING (span=9d3686d83273) — verified correct, and the likeliest cost defect anyone has named on this diff.

I checked the mechanism rather than taking it, and it holds exactly as described. At nudge_wake.py:710 the guard is if int(bounds.get("scrubbed") or 0): return irq.Verdict(irq.Outcome.FALLBACK, ...). It asks whether a scrub refusal happened at all. It does not ask whether the refused item is new this tick.

The branch above it cannot cover for that. if not state.get("since_last_tick") only reaches the calm answer when the whole delta is empty, and the pinned pr_state and pr_checks items keep it non-empty on every tick. So a refusal on prose the judge has already seen takes the same exit as a refusal on prose that just arrived.

The consequence is the one Opus states: one third-party comment whose body the per-item scrub refuses makes the tick fire every cadence interval for as long as that comment stays inside _MAX_REMARKS — up to DEFAULT_REMARK_HORIZON_SECS, five hours — spending max_cycles and max_runtime_secs on evidence that has not changed. The judge is never even asked.

Worth saying plainly: this is a more likely failure than the fenced finding on this same diff. That one needs roughly 129 loops abandoned between publish and take plus a specific ordering. This one needs one commenter to paste a long URL, which is ordinary behaviour on a pull request. It is also a defect I introduced — the scrub-refusal exit was added to stop a partial shed reaching the judge, and it over-fires.

The remedy is available rather than hypothetical, which is worth recording for whoever takes it. first_seen_this_tick already exists: autonudge.py:6156 stamps it per item, autonudge_judge.py:675 already reads it, and _PR_FACTS_TICK_KEYS already classifies it as tick-local so it does not perturb the digest. Gating this exit on a refusal whose item carries it true is the shape Opus proposes, and nothing needs inventing to do it.

Not on this head, and the reason is procedural rather than technical. This lane concluded with no blocking findings, the head's only remaining blocker is a security-fenced item awaiting a human override, and the push rule in force on this pull request does not authorize a push for an advisory finding. Changing the exit condition of the quiet decision also needs its own pins in both directions — a repeated refusal stays calm, a fresh one still fires — which is a round of work rather than a line.

So this is recorded as accepted and owed, not as answered away. Flagging it to the human who holds the override, because taking the override ships this with it, and that ought to be a decision made knowingly rather than by omission.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • Item 9 (span=270bedab28bc) — taking the second clearing branch: the counted path to an overflowing notice store is named, and it is filed.

The item offers two ways to clear: delete the machinery, or name a counted path that reaches more than 64 concurrently-abandoned stashes. The second one is available, so deleting would be wrong.

The path is loop removal. SlackGateway drops loops at autonudge.py:2932 and autonudge.py:3488, and neither site clears the judge's stash or its loss notice — the judge module exposes no cleanup entry point for them to call. Every removed loop that still owns a notice leaves one behind permanently, because nothing ever takes it. Abandoned stashes therefore accumulate with loop churn rather than being bounded by how many loops are live at once, which is what makes the cap reachable instead of theoretical. That gap is filed as #14113.

The number is not mine either. The GPT lane's own adjudication on the previous head put the condition at roughly 129 loops abandoned between publish and take in one process, which is the same population this cap counts, arrived at independently.

Reachability is also pinned rather than argued. test_one_stuck_notice_does_not_pin_every_loop_forever drives the store past its cap, leaves one notice owned by a loop that never returns, and asserts an unrelated loop still reads whole. It fails without the backstop.

What the item is right about is the description, and that is a fair hit. The body declares the stash cap and its counted drop notice but stops there, so a reader learns the first store is bounded and not that a second one is, nor what spends its state. That is under-declared for machinery this size.

I am not editing the description on this head. A body edit re-rolls every review lane exactly as a push does, and this head has GPT clear for the first time in the round with one lane still in flight. Trading that to add a paragraph costs more than the paragraph is worth; this comment carries the declaration until a round that needs a push for another reason.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • Item 10 (span=14b64192063b) — verified correct in full, and stronger than the item claims.

I checked both symbols rather than taking the grep. BUCKETS at probes/gh_pr.py:177 has exactly one other mention in the tree, and it is a docstring cross-reference at line 295, not a call. sanitize_body at line 254 has no caller in src/ at all — the module reaches for _sanitized_body_with_clip directly, which is the one that returns the clip flag the reading needs. Every other BUCKETS hit in the repository belongs to an unrelated module's own constant: _CHECK_BUCKETS in the Issue Radar normalizers, _FAST_BUCKETS_MS in the metrics provider, NAMED_CHECK_BUCKETS in the judge module. None of them touch these.

One fact the item does not mention makes the case tighter. probes/gh_pr.py declares an explicit __all__ of nine names, and neither sanitize_body nor BUCKETS is in it. So they are not a deliberately published surface that happens to have no consumer yet — they are undeclared module-level names with no consumer, which is the plainest form of the problem.

They are referenced from test/test_gh_pr_fetch.py, so the tree has tests pinning a surface nothing calls. That is worse than untested dead code, not better: the tests make the surface look load-bearing to the next reader and would have to be deleted with it.

Ticketed onto #14073, which already carries the dead-surface question for this module from the earlier round's finding on with_remark_bodies. One decision about what probes/gh_pr.py publishes covers all three names, and splitting it invites three different answers.

Not deleted on this head. Deleting a public name plus the tests that pin it is a behaviour-neutral change that still has to prove it is behaviour-neutral, and the push rule in force here does not authorize a round for an advisory finding while the board is otherwise settling. The names are inert — nothing calls them, so nothing is at risk from their staying one more round.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

The item's own clearing condition offers two routes: coalesce a sustained run of partial readings into one report, or file and link the promised follow-up. #14071 is that follow-up, and it was filed before this round.

The tension the item names is real and I am not arguing it away. pr_target_is_unread treats partial as unread, so the tick fires on it, and a shared check name that stays unresolvable can keep a reading partial indefinitely. The consecutive-failure backstop does not help, because it counts readings that never reached the subject rather than readings that reached it and came back incomplete. Those are different populations and only one is covered.

#14071 carries the coalescer for that, and it now has a second cause behind it rather than one: a body-stash drop also produces a not-whole reading, by a different route and with a different clearing condition, which is recorded on the issue as a comment. A coalescer that handled only the check-name case would leave the other open, so the ticket covers both.

Not built on this head. A coalescer is a state machine with its own persistence question — how many consecutive partials, counted where, reset by what — and the design decision it needs is exactly what a follow-up issue is for rather than a round appended to a pull request that is otherwise settling.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

The mechanism holds. _parse_config reads repo, pr and host, so an already-armed watch carrying wake_on_green, known_reds, coalesce_secs or note keeps parsing and silently loses what those keys did. The two consequences the item names are the right two, and they point opposite ways: the ready-on-green wake stops arriving, and an inherited red that the old known_reds suppressed now fires once. One is a wake lost, one is a wake gained, and an owner gets neither explanation.

Calling that compatible is too strong. The config still parses, which is all "keeps working" is really saying, and the item is correct that parsing is not the same as behaving as armed.

#14072 carries it, with both clearing routes the item offers recorded there: the upgrade note stating that a green wake now needs an explicit wake_when, or the legacy keys folded into the brief at arm time for a loop whose stored config still has them. The same issue holds the settled-green default that the same retirement removed, because both are the question of what an existing watch inherits.

Not on this head. Folding the legacy keys into the brief means deciding what wake_on_green: true translates into as a criterion sentence, which is a design answer rather than a mapping — and the alternative route is a description edit, which re-rolls every lane exactly as a push does while GPT has just come clear.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

The suggestion is to render the incomplete note into the wake that the not-whole path fires, so a partial-driven wake names its cause instead of reading to the owner as pull-request activity.

It is right, and the reason is the one the suggestion implies rather than states. The not-whole path fires precisely when the reading could not be trusted, and the wake it produces is indistinguishable from a wake caused by something a reviewer actually did. An owner who cannot tell those apart learns the wrong thing from the interruption: they go looking for the comment or the check that changed, find nothing, and the next such wake is the one they start ignoring. The reading already carries the reason — checks:incomplete, a scrub shed, a dropped body stash — so the information exists and is being discarded at the last step.

Folded into #14071 rather than filed separately, because that issue already owns the partial reading's cost and this is the same reading's presentation. A coalescer that reports one partial per run and still does not say why it fired would leave the owner with the same puzzle, so the two belong to one change.

Not on this head. Threading the note into the wake body touches the notice-rendering seam and needs a pin that the cause survives into the delivered text, which is a round of work. The suggestion is advisory, this lane is a WATCH rather than a block, and no push is authorized for it.

@chenmingwei23

chenmingwei23 commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor Author
  • FINDING (span=fd0d21da5fd4) — correct, and it is a defect I introduced: the code does not do what its own comment says.

Verified in full rather than taken. At autonudge.py:6694 the commit overwrites the prior value with loop.judge_pr_seen = staged_pr_seen. Fourteen lines later, when _persist_judge_state refuses, the failure branch assigns loop.judge_pr_seen = {}.

The comment on that branch says it "leaves the baseline exactly as the stored record has it", and the warning says "leaving the stored baseline in force". Neither is what {} does. The stored record holds the OLD baseline -- my own comment three lines above says so explicitly, because the awaited judge write inside the call has already landed one holding it. An empty dict is not the old baseline; it is no baseline.

The consequence is exactly as the finding states, and the chain is short. The next tick builds its seen set from judged.get("remarks"), which on an empty dict is None, so seen is empty and ident not in seen answers true for every remark in the horizon. The judge is then handed remarks it already answered, each rendered "(new since the last tick)", and spends a turn on them.

One consequence the finding does not name makes it worse, and it is worth recording. An emptied baseline also defeats the scrub-refusal freshness gate added on this head: that gate asks whether a refused item is new, and after this branch every item claims to be. So a persist failure re-opens the fire-every-interval path that gate exists to close, for every remark at once.

The proposed fix is right and is the minimal one: capture prior = loop.judge_pr_seen immediately before the commit and restore prior in the failure branch. That makes the code do what both the comment and the warning already claim, so nothing else has to change.

I am not pushing it. The board is fully green on this head -- 81 success, 11 skipped, zero failure or cancelled, every review lane concluded and fresh, PR Readiness success -- and the push rule in force here authorizes a push only for a new blocking rule or a red I caused. This finding is neither: the lane's own verdict is "no blocking findings". A push would re-roll six green lanes to fix a branch that is only reached when a durable write fails.

Tracked as #14137 instead, which carries the branch, the full consequence chain including the effect on the freshness gate, and the prior-restore remedy verbatim. The trade was weighed explicitly and went this way because the defect sits only on the persist-refused error path and self-heals on the next successful write, so a three-line change did not justify re-rolling six clean review lanes. Recorded as real and owed rather than answered away.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still required? Seems the new wake judge has already been merged? I guess we will need to use Jev like mechanism later when it is available.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Yes, still needed. #12787 merged the judge itself; this PR is the follow-up that makes the judge actually see the evidence.

Today the gh-pr probe still makes its own wake/quiet call: any board change wakes the owner directly, and comment bodies never reach the judge at all. This PR turns the probe into a pure fetcher -- retry, backoff, rate-limit handling and pagination live there -- and hands everything it fetches (checks, reviews, comment bodies, mergeability) to the judge, which makes the only decision. The core keeps one hard rule: PR merged/closed ends the loop.

Jev is already available and in use (nudge_evidence scope + TYPESAFE_API_KEY); the haiku lane from #12776 covers installs without a key. RFC #13790 (merged) documents this split.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

@iamwhatever Still required, and the merged RFC names this PR as the piece that is still open.

docs/request-for-change/rfc-wake-judge.md on main carries this in its own frontmatter:

status: partial
implementation-prs: [12776, 12787, 13759]

12776 and 12787 merged; 13759 is the remaining one. Its PR ladder item 4 describes this PR item for item: "PR G -- the reader fetches and the judge decides: probes/gh_pr.py keeps the transport and drops every wake decision, the whole reading reaches the judge as evidence (pr_state, pr_checks, pr_comment, pr_review), the core keeps the one mapping it can act on ... and the script-cron driver goes."

What merged was the judge's screening machinery and the design doc, not this reading:

The merged code says in its own words that this PR's reading does not exist yet. src/kiro_crew/decisions/points/nudge_wake.py:197-202, written by 2d5a800ee7: "There is deliberately no kind for a pull-request comment BODY: the reader that sees PR-level comments reduces each to a fixed-width fingerprint and retains no body, so no producer could fill such a kind and a declared one would promise a reading the collectors cannot make." And src/kiro_crew/autonudge_judge.py:421: "Comment body TEXT is not part of this evidence." docs/feature-map/README.md:222 states it flatly: "an owner's pull-request criterion is not evaluated on any build today".

Concretely, against main at 7b2da39c53:

Capability main this head 3277d9545c
probe emits its own wake yes, probes/gh_pr.py:618 absent (0 hits for Severity.WAKE / wake_on_green)
evidence kinds transcript_tail, pr_checks adds pr_state, pr_comment, pr_review
comment body reaches the judge no, fingerprint only yes, sanitized and clipped
transport retry / rate-limit / pagination 0 / 0 / 0 1 / 22 / 7
total_count reconcile, partial/incomplete 0 / 0 3 / 18
pr_watch.py script cron present deleted

Nothing else implements it: git log origin/main -- probes/gh_pr.py stops at #13310, and no other open PR touches that file.

On the Jev point: that lane already exists and is already selectable, decisions.nudge_wake.provider in {auto, jev, llm}, landed by #12776. What is missing is a key, tracked in #11522. So it is orthogonal to this PR rather than a replacement for it.

CI on this head is clean: 92 checks, 92 names, 0 pending, 0 non-pass, readiness: passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants