Skip to content

docs(rfc): wake judge -- probe fetches, judge decides - #13790

Merged
iamwhatever merged 1 commit into
mainfrom
docs/rfc-wake-judge-fetches
Sep 25, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
docs/rfc-wake-judge-fetches

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

What is the problem?

rfc-wake-judge.md records a placement the implementation no longer follows, and records it in a way that cannot work.

The document says the reader answers where it can and the judge runs only on the reader's quiet: "defers to the probe's own quiet return for a loop whose probe does run, because only the probe sees a merged or closed subject and ends the watch". Section 9 states the same thing as a composition rule: "probe first, judge on what the probe cannot type".

Why this issue matters to the user

Under that split the reader decides what is worth the owner's turn, and the criterion is exactly what differs between two loops watching one board. One owner wants to be woken when a reviewer asks a question; another only when the board goes red. A reader that answers first has already ruled for both of them.

It also leaves a whole class of watch unable to state its bar. "Wake me when a reviewer asks for a change" lives in prose, on a lane that reports success, so nothing typed sees it -- and a reader that returns quiet before the judge is consulted means the judge never gets the chance.

How our fix solves it

The symptom is a wake criterion an owner cannot express. Following it back: the criterion lives in prose, prose reaches only the judge, and the judge was reached only after a typed reader had already returned quiet. So the reader must stop answering.

This document now records that placement:

  • The gh-pr reader fetches and decides nothing. It hands over the whole reading -- state, mergeability, review decision, head, the check board deduped newest-per-name against the API's own total_count, and comment and review bodies bounded per item and in total.
  • The judge decides wake or quiet from that reading against the loop's own criteria, or the built-in brief when it names none.
  • One ending stays deterministic and stays out of both: the auto-nudge core maps a merged or closed pull request to TERMINAL. The judge can never end a watch, so no prose buys permanent silence.

Three consequences are recorded with it. The evidence kinds the pull-request collector produces are named (pr_state, pr_checks, pr_comment, pr_review). The char budget PINS those two typed summaries and sheds the oldest prose item when it binds: the built-in criteria are answered from the summaries, so shedding them would delete the evidence the question is asked against, and they are small enough (42 characters for a 90-lane board) never to be the pressure. The built-in brief names two clauses explicitly rather than leaving them to "a blocker": a remark that asks, which nothing typed can see, and a failing or short-of-whole check board, which is what an owner arming no criteria at all most expects. The clause says "failing" rather than "newly failing" because the state carries the current board and no prior one; last_verdict is what stops a red board waking its owner every tick.

The script-cron driver is recorded as retired. A script cron runs as a sandboxed subprocess with no gateway credential and no decisions provider, so a reading made there can reach no judge; keeping the adapter would leave those jobs polling on schedule, deciding nothing and reporting nothing.

What tests we did

Documentation only, so the checks are the documentation gates: scripts/docs_lint.py passes, and the comment-history gate passes with no source comments in scope. No CHANGELOG.md change. One commit, two files.

The index row and its audit paragraph are updated in the same commit, following the convention the last two revisions of this document used. The audit sentence names the branch sha the placement was read at and says so, and the front matter's own last-audited / audited-at are left alone: they record the last audit against main, which this change does not re-do.

Any other suggestions on the work

The document is split from the implementation deliberately. The First Principles lane on the implementation asks for the record to exist on the base branch before the code that follows it, which is the same path the last two revisions of this document took.

status stays partial, which is still accurate: the LLM lane is on main, and the core is on a branch the front matter names.

no linked issue: this records a design placement rather than fixing a reported defect. Refs #13759, which implements it and must not be closed by this merge.

@chenmingwei23
chenmingwei23 requested review from a team and dwu96 September 25, 2026 11:01
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of f0eb139839b40fb1bf844e8ecf5b555373e34192 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] f0eb139

False positive or not applicable? A repository writer can comment:
/ai-review override gpt f0eb139839b40fb1bf844e8ecf5b555373e34192: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of f0eb139839b40fb1bf844e8ecf5b555373e34192 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The revision drops the branch-vs-main demarcation the prior revision built, while adding more branch-read content and naming two different read SHAs.

Watch

The old Status section listed exactly which sections were read off an unmerged branch and instructed "Re-audit at merge" — this diff deletes that list, yet §3.1's shed rule, §3.2's collector, §3.3's brief clauses and §4's table now describe #13759's branch, not main. Meanwhile the README says the placement was "read at db5267bda9" and the doc says "Code references below were read at ccba8886cc", with no statement of which sections came from which. Mechanism: #13759 can drift in review or merge changed; consequence: the next auditor cannot tell which statements are main-facts, which is the one job this doc's audit apparatus exists to do.
Clears when: the Status paragraph names the sections read off #13759's branch, marks them for re-audit at merge (as the 2fdadc71ac revision did), and reconciles the two SHAs.

[DESIGN-REVIEWED] f0eb139

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed f0eb139839b40fb1bf844e8ecf5b555373e34192 — this comment is updated in place on each push.

Review details

Five doc-vs-code contradictions in the landed text; nothing blocks the merge.

FINDING — docs/request-for-change/rfc-wake-judge.md:104 — §3.1's shed rule is inverted against both trees: "the typed summaries pr_state and pr_checks are pinned, and when the budget binds the oldest PROSE item goes", while SHED_FIRST_KINDS = frozenset({KIND_PR_CHECKS, KIND_PR_STATE}) + _shed_index on the cited branch commit db5267bda9 sheds exactly those two FIRST and keeps the prose (and main's build_state sheds oldest-overall, pinning nothing), so a reader tuning the budget from the design of record drops the reviewer bodies the point exists to read → Fix: restore the implementing PR's own wording — the budget sheds a typed summary (pr_state, pr_checks) before any prose item, and among prose items the oldest.

FINDING — docs/request-for-change/rfc-wake-judge.md:514 — §7 Q6 ("DECIDED: it is auto-paused, not deleted … That is what PR G implements") and §8:567 ("AUTO-PAUSED rather than deleted … stays listed with auto_paused") record an outcome the cited path cannot produce: a copied pr_watch.py:watch tick reaches PrWatchProbe.identity, whose if not getattr(ctx, "in_process", False) raises ValueError, which irq.py:602 turns into Done, which slack/gateway.py's status == "done" branch treats as success — record_success() then _deliver_script_result(..., remove=True) — so the registration is deleted on the first tick and _AUTO_PAUSE_THRESHOLD is never approached → Fix: restate Q6 and §8's PR G entry as the outcome the code produces (the job is removed with one message) or as an outstanding mechanism change, not as what PR G implements.

FINDING — docs/request-for-change/rfc-wake-judge.md:211 — §3.3 states the shipped default brief as "a blocker, a failing check or one whose reading is not whole" and §3.3:228 argues 'says "failing", not "newly failing"', while DEFAULT_WAKE_WHEN at db5267bda9 reads "a check that is newly failing or whose reading is not whole" — opposite predictions about whether a board that stays red keeps waking its owner every tick → Fix: restore "newly failing" in §3.3 and drop the paragraph arguing against it, or change DEFAULT_WAKE_WHEN in the PR that lands this reading.

FINDING — docs/request-for-change/README.md:28 — the index's closing claim "only the reader placement and the Decisions-card consent switch remain outstanding" names shipped work and hides unshipped work: the nudge_evidence switch is on main (DecisionsCard.tsx:374/:411, pinned by decisionsCard.test.tsx:1875 "draws the nudge_evidence switch on the nudge.wake panel"), while §8's PR F skills are absent (wake_when/quiet_when has zero hits under src/kiro_crew/builtin_skills/), and the same PR's RFC status line says the outstanding items are the reader placement and the skills → Fix: replace the clause with the RFC's own reading — reader placement (PR G) and PR F's skills outstanding; the consent switch renders generically from needs_scope and is on main.

FINDING — docs/request-for-change/rfc-wake-judge.md:6 — the revision re-audits the main-side reading and bumps implementation-prs but leaves last-audited: 2026-09-23 / audited-at: 2fdadc71ac, against the directory's own rule to update all four in the same PR, and §3's new provenance line cites ccba8886cc, which resolves to no object in this repo or on any remote ref (the implementing branch origin/feat/gh-pr-probe-pure-fetch is db5267bda, the commit the index row names), so the field the README calls "the authoritative freshness signal" understates the audit and the body's claims cannot be re-verified at the commit it names → Fix: set last-audited: 2026-09-25 and audited-at to the main commit verified (2d5a800ee), and cite db5267bda9 for the branch reading in the body as the index row does.

[OPUS-REVIEWED] f0eb139

Verdict parsed from the review's SHA-scoped output markers for commit f0eb139839b40fb1bf844e8ecf5b555373e34192.

False positive or not applicable? A repository writer can comment:
/ai-review override fable f0eb139839b40fb1bf844e8ecf5b555373e34192: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-wake-judge-fetches branch from 68124ec to 7b3624c Compare September 25, 2026 11:27
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Sep 25, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • span=8a40c323f69f -- retired babysit jobs are deleted after one message to a reader-less surface

Right, and stronger than stated: the notice may reach nobody at all.

Checked rather than assumed. The refusal path raises out of the watch identity, and the kernel converts that to its own ending with a message. Done's contract in the cron layer says a script delivers a message by calling notify before raising, and the kernel does not call it -- so the ending carries a string the owner may never see, and then the registration goes. That is the unread death section 1 measures, arriving through a different door.

Recorded as section 7 question 6 rather than silently decided, because the cron layer already offers the alternative you name and choosing it is a behaviour change on the implementing branch, not a documentation one. A job whose run keeps failing crosses the auto-pause threshold and then stays LISTED with auto_paused, a last_error that can name the replacement, and an audit entry -- evidence kept, reason readable on the schedule page.

Section 8's entry no longer asserts deletion. It records refusing as the decided half, names disposal as unsettled, and requires the implementing change to settle question 6 before it lands. The conductor owns that choice and has it in front of them.

@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-wake-judge-fetches branch from 7b3624c to 9f490ba Compare September 25, 2026 11:45
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 25, 2026
@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-wake-judge-fetches branch from 9f490ba to e9a6d59 Compare September 25, 2026 11:58
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-wake-judge-fetches branch from e9a6d59 to ccba888 Compare September 25, 2026 13:02
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Sep 25, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • span=2d22c7a03284 -- a routine 5xx per tick would spend the turn this record exists to save

Right about the direction, and the transport already carries most of the floor you are asking for.

A single 5xx does not reach the clause. Every call goes through a bounded retry with exponential backoff and full jitter, and a refusal that names itself as non-transient is answered once instead of retried, so an isolated fault is absorbed inside the tick and the reading still comes back ok. The clause fires only when the retries are exhausted or the whole-tick budget runs out, which is a sustained outage rather than a routine fault.

What remains is the case you are pointing at: a window that stays unreadable across many ticks. The kernel's consecutive-failure backstop coalesces exactly that, but it counts a reading that could not reach the subject at all, not one that came back short -- so a persistently partial board is not coalesced today.

Not closed here, and not silently: this is the same residual the design lane raised against the implementing change, where the owner of that work ruled that a short reading fires because a quiet drawn from half a board is a quiet about the wrong half. Coalescing repeated partials like the failure backstop is the follow-up that would close it, and it needs its own round with a measurement of how often a board is persistently partial in practice.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • span=9b83636b5394 -- "pinned" contradicts SHED_FIRST_KINDS, which discards typed summaries first

The contradiction is real and the direction of the fix is the other way round: the code is what changes, not this record.

SHED_FIRST_KINDS does shed pr_state and pr_checks before any prose item, and those are exactly the two items the built-in criteria are answered from, so the budget can delete the evidence the question is asked against. The design lane found that on an earlier head of this document and it is why the rule was rewritten here.

Documenting the current behaviour instead would record a defect as the design. The summaries are also bounded and small -- one pr_checks item renders a 90-lane board in 42 characters -- so shedding them frees nothing worth having while the prose items at up to 1 000 characters each are the actual pressure.

This record therefore leads the code deliberately, and the code change is queued for the implementing pull request's single rebase push after this one merges: pin the two summaries, shed the oldest prose, and update the docstring and the tests that pin the two-tier order.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • span=9b83636b5394 -- "failing" contradicts DEFAULT_WAKE_WHEN, and last_verdict carries no fingerprint

Two claims. The first is the record leading the code on purpose; the second is correct and the record is wrong.

On the wording: the state handed to the judge carries the CURRENT board and no prior one, so a brief asking for newness asks a question nothing in the state can answer. Retaining "newly failing" would keep a criterion that cannot be evaluated. The code is what changes: the implementing pull request's single rebase push after this one merges replaces the clause and updates the test that pins the old wording.

On the fingerprint: you are right. verdict_record returns {outcome, evidence_items, at} and nothing else -- deliberately text-free -- so there is no evidence fingerprint anywhere in last_verdict. What it actually supports is weaker than this document claims: the judge learns it already answered, and on how many items, not that the evidence matched. This document's own description of last_verdict as the previous answer "and fingerprint" is wrong on the same point and predates this change.

That correction needs a further edit to this file rather than a code change, and it is raised to the owner of this work to fold into the rebase push, since this pull request takes no further pushes.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author
  • span=9b83636b5394 -- auto-pause contradicts the ValueError to Done to removal path

Accurate about today's path, and the removal is the half being changed rather than the half to write down.

Today the watch identity raises, the kernel ends the run, and the scheduler deletes the registration. That registration is the only durable record that a watch was ever armed, so deleting it leaves an owner who looks days later with nothing to see -- which is the unread death this record's problem section exists to fix, reached through a different door.

Worth correcting one thing in passing, because an earlier draft of this document had it wrong: the owner IS told. The runner reports the ending with its message and the scheduler delivers that message before removing the job. The loss is the registration, not the notice, and section 7 question 6 now says so.

The owner of this work ruled that a refused job is auto-paused rather than deleted: it stays listed with auto_paused and a last_error naming the replacement path, which the cron layer's own failure accounting already provides. The code change is queued for the implementing pull request's single rebase push after this one merges.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 25, 2026
Record the placement the wake judge rests on: the gh-pr reader fetches and
decides nothing, the whole reading reaches the judge as evidence, and the one
deterministic ending -- a merged or closed pull request -- is mapped by the
auto-nudge core.

The document previously recorded the opposite: the reader answered where it
could and the judge ran only on the reader's quiet. That split has the reader
deciding the owner's criterion for them, and the criterion is exactly what
differs between two loops watching one board. It also leaves a pull-request
watch whose bar is a sentence rather than a check conclusion with no way to
express it.

Also recorded: the evidence kinds the pull-request collector produces
(pr_state, pr_checks, pr_comment, pr_review), why the char budget sheds a
typed summary before any prose item, the two clauses the built-in brief names
explicitly and why, and the retirement of the script-cron driver -- a
sandboxed subprocess holds no decisions provider, so a reading made there can
reach no judge.

Refs #13759
@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-wake-judge-fetches branch from ccba888 to f0eb139 Compare September 25, 2026 15:06
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 25, 2026
@iamwhatever
iamwhatever merged commit da67439 into main Sep 25, 2026
129 of 133 checks passed
@iamwhatever
iamwhatever deleted the docs/rfc-wake-judge-fetches branch September 25, 2026 17:09
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants