Repository navigation
docs(rfc): wake judge -- a System One model screens auto-nudge ticks - #12735
Conversation
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify §7 Q2's ruling — LLM lane bypasses keystone consent — is ratified by a security owner, not just the doc's own author. Suggestions
[DESIGN-REVIEWED] 67f3a26 |
UX Review (Fable 5) — ⏭️ skippedRevision |
First Principles Review (Fable 5) — ⏭️ skippedRevision |
d9139a2 to
7612a8b
Compare
7612a8b to
68d0ccf
Compare
68d0ccf to
7349e20
Compare
7349e20 to
4ec5c39
Compare
An auto-nudge loop fires a full model turn on its owning session every interval. PrWatchProbe already makes an unchanged tick free for a loop that names one pull request; every other loop pays full price on every tick, because the only typed reader is a probe and a probe cannot interpret a transcript tail, a review comment or a log line. The design inserts a cheap reader instead of making the expensive one cheaper. Each tick the gateway scrubs and bounds the new evidence and asks a nudge.wake decision point, on the decisions seam that already ships, whether the owner must act now. Only a yes fires the main session, and every error path -- provider failure, timeout, invalid answer, low confidence, quiet-streak floor -- is the ungated timer, so the judge can remove turns but never silence a loop. The judge has exactly two verdicts, QUIET and WAKE. It cannot end a loop: only a typed probe does that, a merged or closed pull request or a work ledger with every item closed. If prose could mint a terminal verdict then a comment saying "this is merged" would buy permanent silence, which is worse than the compound bound below. A finished or broken reading therefore WAKES with the verdict attached and the woken session decides. Section 5 states the compounding bound, not just the per-answer cost. A QUIET answer sustained by untrusted evidence compounds, so an input crafted to read as nothing-to-do can suppress delivery for the floor times the interval: 7.5 hours at the default floor of 10 and a 45-minute interval. The floor enforces the bound because it counts every tick including QUIET ones, resets only on a delivered turn, and is configurable downward per point but never off. Section 6 now reconciles the two monitoring documents this borrows from. rfc-token-efficient-monitors owns the controller that adds fingerprints, budgets, terminal outcomes and completed-turn accounting, and it leaves the AutoNudge timer as the durable scheduling primitive, so terminal outcomes are its vocabulary rather than the judge's. rfc-consolidated-monitor merges three streams including autonudge.py; the judge is a field on an existing loop's record rather than a second loop, so there is nothing extra to merge, but its deletion list covers irq's quiet-streak floor on the grounds that structured budgets subsume it, so section 5's bound must be restated against whatever replaces the floor. The switch is not a new toggle. nudge.wake registers in gate.POINTS with a nudge_evidence scope in POINT_SCOPE_KEYS, so it draws its own row on the Decisions card with no frontend edit. auto resolves to the Jev lane only when the keystone consents and that scope is granted, otherwise to the LLM lane; a pinned jev without the scope stores the brief and runs the plain timer. Section 7 Q2 records that ruling. This also withdraws rfc-ledger-conductor-channel, which Design Review correctly found re-solved a wake gate rfc-conductor-work-ledger already owns, on a second store, naming no incumbent. That work belongs to that document's Phase 3 and Phase 4. The wake judge is the complementary half: typed probe for typed facts, judge for prose evidence. Verified at 80bd0a8: gate.py declares DECISION_POINT_NAMES, impl_jev.py holds JevOracle and _to_wire, points/ carries six live points, and types.py declares one wire type (class Choice, aliased Question) with no noul and no score. nudge_wake.py, impl_llm.py, the nudge_evidence scope and every decisions.nudge_wake.* key, quiet_streak_floor included, do not exist. The tick hook _monitor_tick_is_quiet exists and its floor _MAX_QUIET_STREAK is 10, the value 3.1 and 5 both defer to. PrWatchProbe exists. The Decisions-card shape 3.6 mounts its row on is #12598, open with auto-merge on and no approving review, pending at c181a6a. Docs only, status draft.
4ec5c39 to
67f3a26
Compare
GPT 5.6 Review —
|
…irodotdev#12735) An auto-nudge loop fires a full model turn on its owning session every interval. PrWatchProbe already makes an unchanged tick free for a loop that names one pull request; every other loop pays full price on every tick, because the only typed reader is a probe and a probe cannot interpret a transcript tail, a review comment or a log line. The design inserts a cheap reader instead of making the expensive one cheaper. Each tick the gateway scrubs and bounds the new evidence and asks a nudge.wake decision point, on the decisions seam that already ships, whether the owner must act now. Only a yes fires the main session, and every error path -- provider failure, timeout, invalid answer, low confidence, quiet-streak floor -- is the ungated timer, so the judge can remove turns but never silence a loop. The judge has exactly two verdicts, QUIET and WAKE. It cannot end a loop: only a typed probe does that, a merged or closed pull request or a work ledger with every item closed. If prose could mint a terminal verdict then a comment saying "this is merged" would buy permanent silence, which is worse than the compound bound below. A finished or broken reading therefore WAKES with the verdict attached and the woken session decides. Section 5 states the compounding bound, not just the per-answer cost. A QUIET answer sustained by untrusted evidence compounds, so an input crafted to read as nothing-to-do can suppress delivery for the floor times the interval: 7.5 hours at the default floor of 10 and a 45-minute interval. The floor enforces the bound because it counts every tick including QUIET ones, resets only on a delivered turn, and is configurable downward per point but never off. Section 6 now reconciles the two monitoring documents this borrows from. rfc-token-efficient-monitors owns the controller that adds fingerprints, budgets, terminal outcomes and completed-turn accounting, and it leaves the AutoNudge timer as the durable scheduling primitive, so terminal outcomes are its vocabulary rather than the judge's. rfc-consolidated-monitor merges three streams including autonudge.py; the judge is a field on an existing loop's record rather than a second loop, so there is nothing extra to merge, but its deletion list covers irq's quiet-streak floor on the grounds that structured budgets subsume it, so section 5's bound must be restated against whatever replaces the floor. The switch is not a new toggle. nudge.wake registers in gate.POINTS with a nudge_evidence scope in POINT_SCOPE_KEYS, so it draws its own row on the Decisions card with no frontend edit. auto resolves to the Jev lane only when the keystone consents and that scope is granted, otherwise to the LLM lane; a pinned jev without the scope stores the brief and runs the plain timer. Section 7 Q2 records that ruling. This also withdraws rfc-ledger-conductor-channel, which Design Review correctly found re-solved a wake gate rfc-conductor-work-ledger already owns, on a second store, naming no incumbent. That work belongs to that document's Phase 3 and Phase 4. The wake judge is the complementary half: typed probe for typed facts, judge for prose evidence. Verified at 80bd0a8: gate.py declares DECISION_POINT_NAMES, impl_jev.py holds JevOracle and _to_wire, points/ carries six live points, and types.py declares one wire type (class Choice, aliased Question) with no noul and no score. nudge_wake.py, impl_llm.py, the nudge_evidence scope and every decisions.nudge_wake.* key, quiet_streak_floor included, do not exist. The tick hook _monitor_tick_is_quiet exists and its floor _MAX_QUIET_STREAK is 10, the value 3.1 and 5 both defer to. PrWatchProbe exists. The Decisions-card shape 3.6 mounts its row on is kirodotdev#12598, open with auto-merge on and no approving review, pending at c181a6a. Docs only, status draft.
This PR was repurposed. It previously carried
rfc-ledger-conductor-channel.md,which is withdrawn in this revision. Design Review (Fable 5) pointed out that
document re-solved a wake gate
rfc-conductor-work-ledgeralready owns, on asecond store, and named no incumbent. That was correct, so the work moved to
where it belongs: that RFC's Phase 3 (the typed wake gate) and Phase 4 (the
surfaces) carry it, and a separate revision to that document records it.
What lands here is the complementary half, which has no incumbent: a typed probe
handles typed facts, and a judge handles the prose evidence no probe can read.
1. What is the problem?
An auto-nudge loop (
monitor_start) fires a full model turn on its owningsession every interval. When the loop names one pull request by URL,
PrWatchProbealready turns an unchanged tick into a free re-arm. Every otherloop pays full price on every tick: a conductor reading worker transcripts, a
babysitter reading bot comment bodies, a loop watching a log.
The typed wake gate removes the turn only for facts that arrive as typed fields,
such as an item status or a check conclusion. It cannot read a transcript tail, a
review comment, or a log line. Those need judgment, and today the only judge
available is the main session carrying its whole context.
Measured on this repository: a conductor watching three workers at 45 minutes
spends about 48 turns over two days to act on roughly five events. Ten patrol
loops also died together on a gateway restart and nobody noticed for hours,
because the only reader of the loop was the loop itself.
2. Why this issue matters to the user
Every idle tick is spend that buys nothing. The owner pays for a turn in which
the model wakes, reads evidence that did not need it, and goes back to sleep. The
cost scales with the number of loops, so the person running several conductors
pays the most.
It also shapes behaviour in the wrong direction. Because a tick is expensive,
intervals get stretched to keep the bill down, which is exactly what makes a real
event sit unattended for 45 minutes. A cheap screen lets the interval be short
and the wakes be rare, instead of trading one against the other.
3. How our fix solves it
This PR adds the design document only. No code changes.
The chain from symptom to root cause is: the loop wakes the main session because
nothing cheaper can read the evidence; nothing cheaper can read it because the
only typed reader is a probe, and a probe cannot interpret prose. So the design
inserts a cheap reader rather than making the expensive one cheaper. On each tick
the gateway collects the new evidence, scrubs and bounds it, and asks a decision
point
nudge.wakeone question: does the owner need to act now. Only a yes firesthe main session.
The judge sits on the
decisionsseam that already ships, so this is a new pointon an existing mechanism rather than new machinery. Section 3.1 fixes the
questions and puts the thresholds in code with tests, not in prompt text.
Section 3.4 gives two providers behind one interface, so the feature is real
without a Jev key. Section 3.6 puts the switch on the Decisions card as a scope,
so there is no second toggle to keep in sync.
The property that makes it safe to run live is that every error path is the
ungated timer. A provider failure, a timeout, an answer that fails validation, a
low-confidence answer, or a run of quiet verdicts hitting the floor all fire the
turn. The judge can only remove turns that were safe to remove; it cannot silence
a loop.
Rollout is three code PRs after this one: PR D is the judge core (the point, the
scope, the collectors, the tick hook, the
judgespec onmonitor_startandmonitor_update, Jev lane only), PR E is the LLM lane and its settings row, andPR F updates the
babysitandgoal-conductorskills to author the judge'sbrief when they arm a loop.
Convention followed: filed as
docs/request-for-change/rfc-<topic>.mdwith therequired YAML front matter,
status: draft, an H1# RFC:heading, an index rowin
docs/request-for-change/README.md, and that index's audit sentence. Thewithdrawn document and its index row are removed in the same commit.
4. What tests we did
Docs-only, so the doc track only. No pytest and no vitest.
./scripts/docs-lint.shexits 0 with zero findings across all four markdowntrees, including after the file swap, so no index points at the withdrawn doc.
scripts/local-gate.py --dry-runinspected, not executed: for a docs diff itplans a 279-file
vitestrun and apytest -n auto, neither of which applies.CHANGELOG.mduntouched.Every premise the document names was checked at base
80bd0a81fbeforeaudited-atwas filled, and the index row records the result:decisionsseam is real:gate.pydeclaresDECISION_POINT_NAMES,impl_jev.pyholdsJevOracleand_to_wire, andpoints/carries six livepoints including
skills_select,tool_riskandmodel_route.points/nudge_wake.py, noimpl_llm.py, nonudge_evidencescope.types.pydeclares exactly one wire type,class Choice, aliased asQuestion. There is nonouland noscore, which confirms section 3.1'sclaim and is why a sentence was added there naming PR D as the widening.
_monitor_tick_is_quietinautonudge.py, with aquiet-streak floor
_MAX_QUIET_STREAK.PrWatchProbeinprobes/gh_pr.py.merged and described correctly; feat(settings): give the Decisions card a list-and-detail shape #12598 (Decisions card list-and-detail) is
still open, which matches the document treating it as a dependency.
5. Any other suggestions on the work
Design Review's findings are all applied in this revision rather than disclosed. The judge now has two verdicts only, QUIET and WAKE: it cannot end a loop, because if prose could mint a terminal verdict a comment saying "this is merged" would buy permanent silence. Only a typed probe ends a loop. Section 6 also reconciles the two monitoring documents this borrows from, rfc-token-efficient-monitors and rfc-consolidated-monitor, and records one real dependency: the latter's deletion list covers irq's quiet-streak floor because structured budgets subsume it, so section 5's compounding bound has to be restated against whatever replaces the floor.
The section 3.1 floor claim is fixed: the bullet gave the quiet-streak floor a
default of 6 while calling it "the same floor the PR probe already has", and the
shipped constant is 10, so the bullet now defers to
_MAX_QUIET_STREAKitself,verified as 10 at base
80bd0a81f. Section 5 gained a Compounding bullet, becauseits Authority bullet stated only the per-answer cost while a QUIET answer
sustained by untrusted evidence compounds: a crafted input can suppress delivery
for the floor times the interval, 7.5 hours at the default floor and a 45-minute
interval, and the bullet now names that bound, what enforces it, and the
per-point key that tunes it down. Section 1 gained a second measurement, and
section 7's Q2 now records the ruling rather than a proposal:
autoresolves tothe Jev lane only when the keystone consents and the
nudge_evidencescope isgranted, otherwise to the LLM lane, and a pinned
jevwithout the scope storesthe brief and runs the plain timer.
One thing is still worth a look. Section 2 credits "PR #11490 and successors" and
then names the three successor use cases in prose (sampled skill selection,
risky-tool flags, model-tier routing). Those descriptions are accurate and all
three shipped, but only #11490 and #12267 are cited by number; the risky-tool
flag PR is #12253 and the skills-verdict follow-up is #11993, if the author wants
them named.
One dependency worth stating plainly: section 3.6 builds the switch on the card
shape from #12598, which is open. PR D does not need it, because the scope switch
comes from the point registry, but PR E's settings row does, so PR E is gated on
#12598 merging.
Per the repository's issue policy this is a proposal rather than a defect, so it
carries no tracking issue. This PR and the document are the context.