Skip to content

fix(autonudge): let the judge brief's own target list decide the watched PR - #14480

Merged
iamwhatever merged 1 commit into
mainfrom
fix/judge-targets-monitor-inference
Sep 28, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/judge-targets-monitor-inference

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A gated loop's pull-request monitor must come from the subject its judge
brief names, not from its message alone.

A gated auto-nudge loop decided which pull request it watches from its message
only
. The wake judge's own judge.targets list -- the field where an owner names
the subject for the judge -- was never read when the monitor was built.

So a loop armed like this got no monitor at all:

message:       "Babysit PR 13936 until every check is green."
judge.targets: ["https://github.com/kirodotdev/KiroCrew/pull/13936"]

targets.infer refuses a bare PR 13936 (a shorthand carries no host, and #N is
equally an issue reference -- both deliberate), the brief was not consulted, and
infer_monitor returned None.

Five sites resolved the subject independently from loop.message, and update never
re-resolved it when the brief changed, so replacing judge.targets left the watch
on the old pull request.

Why it matters to the user

The pull-request probe is what fetches review and comment bodies so the judge can
read them. No monitor means no probe, so such a loop recorded outcome=fallback with
zero evidence items and fired on the plain timer every interval -- a turn on a
schedule instead of a turn when a reviewer asked for something.

The two halves also disagreed. parse_targets reads judge.targets FIRST and asks
about nothing else once it is present, so the collector asked about the brief's strings
while a monitor built from the message published a reading of a different pull request
-- dropped every tick. A gh fetch that could never become evidence.

What changed (how the fix solves it)

Symptom -> cause -> fix: the turn fires on the timer because the tick has no evidence;
no evidence because there is no monitor; no monitor because the subject came from the
message while the owner named it in the brief. So it comes from both, in one place.

  • autonudge.infer_subject(message, judge) is now the single answer to WHICH pull
    request a loop is about, and all five sites call it.
  • The brief narrows a watch; it never overrides one. It supplies the subject only
    when the instruction names no pull request in ANY grammar. Otherwise the INSTRUCTION
    decides, which is what base answered before the brief was read at all. Falling back
    rather than refusing matters: for the ordinary "blocked on fix(sandbox): bypass toolbox shim to fix nested sandbox failure on macOS 26 #7" shape base bound a
    monitor on the loop's OWN pull request, polled it and retired the loop when it
    merged, so refusing would take merge-retirement away from loops that have it, stored
    ones included.
  • targets.names_pull_request(text) answers "does this name a pull request at all",
    across URL, owner/name#123 and bare PR #42. It has to match infer's own wider
    notion: infer selects only from a URL but treats a shorthand as naming one, so a
    URL-only predicate read "Babysit PR fix(sync): MeshClaw beta sync 2026-07-18/19 batch-41 addendum (1 frontend ported; cumulative 61: 37 backend + 25 frontend) #42" as naming nothing and handed the watch to
    the brief's blocker. Wider than infer(text) is not None too, so a two-subject
    instruction stays unresolved instead of being resolved by the brief.
  • A brief naming more than one pull request leaves the instruction deciding (INFO).
  • monitor_start's ack derives its gated disclosure after the brief is validated.
    monitor_update re-resolves the subject when the brief changes, and a rebind that
    changes the subject now advances config_generation, so a structural-terminal
    verdict for the old subject cannot deactivate the new watch.
  • docs/system-specs/modules/babysit-pr-watch.md now states this resolution order; it
    previously predicted message-only inference.

gate=false loops have no judge and are unchanged; monitor_watch monitors are
untouched. The grammar stays in one place: every candidate goes through
targets.infer, which shares one URL collector with the new predicate.

Tests we did

20 tests in one class in test/test_autonudge_wake_judge.py:

  • End to end: arms a gated loop with the URL only in judge.targets, asserts the
    monitor exists, and asserts a fresh comment arrives as a pr_comment evidence item,
    dropped == 0 (mocked gh reader).
  • Precedence: a brief naming a blocker keeps the instruction's own watch; the
    blocker is never watched in any of the three grammars; a stored loop in that shape
    keeps its monitor; a shorthand instruction refuses the brief's entry; an ambiguous
    instruction stays unresolved.
  • Over-refusal guards: the motivating PR 13936 spelling still gates, and a cited
    source location (autonudge.py#1751) is not read as a second subject.
  • Retarget: replacing judge.targets moves monitor.target and advances
    config_generation; an unchanged subject spends none.

Three mutations, one per fix, each caught: predicate back to URL-only (2 failed),
refusing instead of falling back (2 failed), no generation increment (1 failed).

Scope Result
test_autonudge_wake_judge.py + test_probe_targets.py 297 passed
test_autonudge.py + test_monitor_start_ack.py + 3 siblings 384 passed
mypy src/kiro_crew/ clean, 1735 files
22 static and lint gates all exit 0

The full suite is CI's job on the merge ref.

Not a goal

Resolving a bare PR <number> against a default repository is not implemented.
#123 is equally an issue reference, so a default-repo rule silently turns an issue
mention into a watched pull request, and it would be a second grammar beside
targets.infer. The observed failure is covered without it. Tracked separately.

Pattern harvest

Rule candidate: review-prompt

Pattern: a predicate extracted to guard a decision accepts LESS than the decision
itself. Here names_pull_request scanned only the URL grammar while infer also
treats owner/name#123 and PR #42 as naming a pull request, so the guard passed
text infer calls ambiguous. Ask of any extracted predicate: does it accept exactly
what the decision accepts?

Other suggestions on the work

parse_targets returns nothing once a list is present; load-bearing here, not obvious
at its call sites.

Fixes #14470

@chenmingwei23
chenmingwei23 requested a review from a team as a code owner September 27, 2026 13:22
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 27, 2026
@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 0b4d595956020b54f7c9beff8f06328268066213 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify the instruction-over-brief precedence: in the conflict shape the collector still drops every monitor reading, so the judge stays evidence-blind there (deliberate, documented, base-equal).

[DESIGN-REVIEWED] 0b4d595

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 0b4d595956020b54f7c9beff8f06328268066213 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify issue #14470 records the brief-armed no-monitor failure — the defect's provenance rests entirely on that link, which this lane cannot read.

What this change ships

Inventory (10 items) — 9 justified

Intent: a gated loop whose judge brief names its pull request must actually watch it — a FIX (linked issue #14470).

  1. A gated loop naming its PR only in judge.targets now gets a monitor (was none) — justified
  2. Replacing judge.targets now moves the watch to the new PR (was stuck on old) — justified
  3. A brief-only retarget now advances config_generation — justified
  4. Brief naming a different PR than the instruction: instruction keeps the watch, INFO logged — justified
  5. Brief naming two or more PRs leaves the instruction deciding — justified
  6. monitor_start ack now reports the gate a brief-armed loop actually gets — justified
  7. New public predicate targets.names_pull_request (1 consumer, infer_subject; non-generalized) — justified
  8. New public infer_subject/loop_subject/pr_targets_of replacing 5 independent message-only resolutions (0 targets.infer(loop.message) sites remain in src) — justified
  9. babysit-pr-watch.md rewritten from message-only inference to brief-first order, declared, same commit — justified
  10. Judge spec scrubbed once in add and reused for both store and monitor — rides along

[FIRST-PRINCIPLES-REVIEWED] 0b4d595

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 0b4d595956020b54f7c9beff8f06328268066213 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 0b4d595

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 0b4d595956020b54f7c9beff8f06328268066213: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 0b4d595956020b54f7c9beff8f06328268066213 — this comment is updated in place on each push.

Review details

Nothing blocks — one advisory finding: a judge-only update burns a config generation on a gated loop that has no resolvable subject.

FINDING — src/kiro_crew/autonudge.py:3533 — on a gated loop whose subject does not resolve (gate=True with a non-URL message, e.g. "Babysit the deploy…"), a monitor_update carrying only judge sets rebind_monitor = rebind_monitor or loop.gate, then same_subject is False on its inferred is not None clause alone even though current and inferred are both None, so the else arm runs loop.config_generation += 1 for a watch that moved nothing — and the expected_generation fence at autonudge.py:3327 then refuses the structural-terminal stop captured at fire time ("its config generation advanced while the malformed turn ran"), spending another doomed malformed turn that base would have stopped → Fix: treat "no subject before and no subject after" as unchanged, e.g. same_subject = (inferred is None and current is None) or (inferred is not None and current is not None and …), so a rebind that changes nothing spends no generation.

[OPUS-REVIEWED] 0b4d595

Verdict parsed from the review's SHA-scoped output markers for commit 0b4d595956020b54f7c9beff8f06328268066213.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 0b4d595956020b54f7c9beff8f06328268066213: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 27, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/judge-targets-monitor-inference branch from 2ad5b07 to 3a6dcfc Compare September 27, 2026 14:51
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/judge-targets-monitor-inference branch from 3a6dcfc to 036db6d Compare September 27, 2026 17:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 27, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 27, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/judge-targets-monitor-inference branch from 036db6d to 31d41ea Compare September 27, 2026 18:21
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
…hed PR

A gated loop resolved its subject from the message only, so a loop naming its
pull request in judge.targets and not in its own text got no monitor at all --
nothing was fetched, every tick fell back, and no PR comment reached the judge.

One resolver, infer_subject, now answers WHICH pull request a loop is about for
all five sites that used to ask independently: the brief's target list first, the
instruction second. parse_targets already reads the list first, so both halves
are now about one pull request instead of two.
@chenmingwei23
chenmingwei23 force-pushed the fix/judge-targets-monitor-inference branch from 31d41ea to 0b4d595 Compare September 27, 2026 22:22
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 27, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 27, 2026
@iamwhatever
iamwhatever merged commit f5239a3 into main Sep 28, 2026
97 checks passed
@iamwhatever
iamwhatever deleted the fix/judge-targets-monitor-inference branch September 28, 2026 02:01
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

wake judge: a loop naming its PR only in judge.targets gets no monitor, so no comment ever reaches the judge

2 participants