Skip to content

docs(rfc): wake judge -- a System One model screens auto-nudge ticks - #12735

Merged
iamwhatever merged 1 commit into
mainfrom
docs/rfc-ledger-conductor-channel
Sep 22, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
docs/rfc-ledger-conductor-channel

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

This PR was repurposed. It previously carried rfc-ledger-conductor-channel.md,
which is withdrawn in this revision. Design Review (Fable 5) pointed out that
document re-solved a wake gate rfc-conductor-work-ledger already owns, on a
second store, and named no incumbent. That was correct, so the work moved to
where it belongs: that RFC's Phase 3 (the typed wake gate) and Phase 4 (the
surfaces) carry it, and a separate revision to that document records it.

What lands here is the complementary half, which has no incumbent: a typed probe
handles typed facts, and a judge handles the prose evidence no probe can read.

1. What is the problem?

An auto-nudge loop (monitor_start) fires a full model turn on its owning
session every interval. When the loop names one pull request by URL,
PrWatchProbe already turns an unchanged tick into a free re-arm. Every other
loop pays full price on every tick: a conductor reading worker transcripts, a
babysitter reading bot comment bodies, a loop watching a log.

The typed wake gate removes the turn only for facts that arrive as typed fields,
such as an item status or a check conclusion. It cannot read a transcript tail, a
review comment, or a log line. Those need judgment, and today the only judge
available is the main session carrying its whole context.

Measured on this repository: a conductor watching three workers at 45 minutes
spends about 48 turns over two days to act on roughly five events. Ten patrol
loops also died together on a gateway restart and nobody noticed for hours,
because the only reader of the loop was the loop itself.

2. Why this issue matters to the user

Every idle tick is spend that buys nothing. The owner pays for a turn in which
the model wakes, reads evidence that did not need it, and goes back to sleep. The
cost scales with the number of loops, so the person running several conductors
pays the most.

It also shapes behaviour in the wrong direction. Because a tick is expensive,
intervals get stretched to keep the bill down, which is exactly what makes a real
event sit unattended for 45 minutes. A cheap screen lets the interval be short
and the wakes be rare, instead of trading one against the other.

3. How our fix solves it

This PR adds the design document only. No code changes.

The chain from symptom to root cause is: the loop wakes the main session because
nothing cheaper can read the evidence; nothing cheaper can read it because the
only typed reader is a probe, and a probe cannot interpret prose. So the design
inserts a cheap reader rather than making the expensive one cheaper. On each tick
the gateway collects the new evidence, scrubs and bounds it, and asks a decision
point nudge.wake one question: does the owner need to act now. Only a yes fires
the main session.

The judge sits on the decisions seam that already ships, so this is a new point
on an existing mechanism rather than new machinery. Section 3.1 fixes the
questions and puts the thresholds in code with tests, not in prompt text.
Section 3.4 gives two providers behind one interface, so the feature is real
without a Jev key. Section 3.6 puts the switch on the Decisions card as a scope,
so there is no second toggle to keep in sync.

The property that makes it safe to run live is that every error path is the
ungated timer. A provider failure, a timeout, an answer that fails validation, a
low-confidence answer, or a run of quiet verdicts hitting the floor all fire the
turn. The judge can only remove turns that were safe to remove; it cannot silence
a loop.

Rollout is three code PRs after this one: PR D is the judge core (the point, the
scope, the collectors, the tick hook, the judge spec on monitor_start and
monitor_update, Jev lane only), PR E is the LLM lane and its settings row, and
PR F updates the babysit and goal-conductor skills to author the judge's
brief when they arm a loop.

Convention followed: filed as docs/request-for-change/rfc-<topic>.md with the
required YAML front matter, status: draft, an H1 # RFC: heading, an index row
in docs/request-for-change/README.md, and that index's audit sentence. The
withdrawn document and its index row are removed in the same commit.

4. What tests we did

Docs-only, so the doc track only. No pytest and no vitest.

  • ./scripts/docs-lint.sh exits 0 with zero findings across all four markdown
    trees, including after the file swap, so no index points at the withdrawn doc.
  • scripts/local-gate.py --dry-run inspected, not executed: for a docs diff it
    plans a 279-file vitest run and a pytest -n auto, neither of which applies.
  • CHANGELOG.md untouched.

Every premise the document names was checked at base 80bd0a81f before
audited-at was filled, and the index row records the result:

  • The decisions seam is real: gate.py declares DECISION_POINT_NAMES,
    impl_jev.py holds JevOracle and _to_wire, and points/ carries six live
    points including skills_select, tool_risk and model_route.
  • What the design adds does not exist yet: no points/nudge_wake.py, no
    impl_llm.py, no nudge_evidence scope.
  • types.py declares exactly one wire type, class Choice, aliased as
    Question. There is no noul and no score, which confirms section 3.1's
    claim and is why a sentence was added there naming PR D as the widening.
  • The tick hook exists: _monitor_tick_is_quiet in autonudge.py, with a
    quiet-streak floor _MAX_QUIET_STREAK.
  • The probe it composes with exists: PrWatchProbe in probes/gh_pr.py.
  • Cited PRs: feat(decisions): use Jev for sampled skill selection #11490 (sampled skill selection) and feat(decisions): let Jev route a chat turn to a model tier #12267 (model-tier route) are
    merged and described correctly; feat(settings): give the Decisions card a list-and-detail shape #12598 (Decisions card list-and-detail) is
    still open, which matches the document treating it as a dependency.

5. Any other suggestions on the work

Design Review's findings are all applied in this revision rather than disclosed. The judge now has two verdicts only, QUIET and WAKE: it cannot end a loop, because if prose could mint a terminal verdict a comment saying "this is merged" would buy permanent silence. Only a typed probe ends a loop. Section 6 also reconciles the two monitoring documents this borrows from, rfc-token-efficient-monitors and rfc-consolidated-monitor, and records one real dependency: the latter's deletion list covers irq's quiet-streak floor because structured budgets subsume it, so section 5's compounding bound has to be restated against whatever replaces the floor.
The section 3.1 floor claim is fixed: the bullet gave the quiet-streak floor a
default of 6 while calling it "the same floor the PR probe already has", and the
shipped constant is 10, so the bullet now defers to _MAX_QUIET_STREAK itself,
verified as 10 at base 80bd0a81f. Section 5 gained a Compounding bullet, because
its Authority bullet stated only the per-answer cost while a QUIET answer
sustained by untrusted evidence compounds: a crafted input can suppress delivery
for the floor times the interval, 7.5 hours at the default floor and a 45-minute
interval, and the bullet now names that bound, what enforces it, and the
per-point key that tunes it down. Section 1 gained a second measurement, and
section 7's Q2 now records the ruling rather than a proposal: auto resolves to
the Jev lane only when the keystone consents and the nudge_evidence scope is
granted, otherwise to the LLM lane, and a pinned jev without the scope stores
the brief and runs the plain timer.

One thing is still worth a look. Section 2 credits "PR #11490 and successors" and
then names the three successor use cases in prose (sampled skill selection,
risky-tool flags, model-tier routing). Those descriptions are accurate and all
three shipped, but only #11490 and #12267 are cited by number; the risky-tool
flag PR is #12253 and the skills-verdict follow-up is #11993, if the author wants
them named.

One dependency worth stating plainly: section 3.6 builds the switch on the card
shape from #12598, which is open. PR D does not need it, because the scope switch
comes from the point registry, but PR E's settings row does, so PR E is gated on
#12598 merging.

Per the repository's issue policy this is a proposal rather than a defect, so it
carries no tracking issue. This PR and the document are the context.

@chenmingwei23
chenmingwei23 requested review from a team and dwu96 September 22, 2026 06:46
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 22, 2026
@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 67f3a268826b949d839dd7f46663ed0c5eb185ec — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 67f3a26

Verdict parsed from the review's SHA-scoped output markers for commit 67f3a268826b949d839dd7f46663ed0c5eb185ec.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 67f3a268826b949d839dd7f46663ed0c5eb185ec: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 67f3a268826b949d839dd7f46663ed0c5eb185ec and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 67f3a26

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 67f3a268826b949d839dd7f46663ed0c5eb185ec: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 67f3a268826b949d839dd7f46663ed0c5eb185ec — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify §7 Q2's ruling — LLM lane bypasses keystone consent — is ratified by a security owner, not just the doc's own author.

Suggestions

  • §3.1's thresholds assume calibrated probabilities; the LLM lane's self-reported numbers are not — note per-provider thresholds (or per-lane tuning) before PR E ships them shared.

[DESIGN-REVIEWED] 67f3a26

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision labels Sep 22, 2026
@chenmingwei23 chenmingwei23 changed the title docs(rfc): the session ledger as the conductor-worker channel docs(rfc): wake judge -- a System One model screens auto-nudge ticks Sep 22, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Sep 22, 2026
@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Fable 5) — ⏭️ skipped

Revision 67f3a268826b949d839dd7f46663ed0c5eb185ec touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 22, 2026
@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ⏭️ skipped

Revision 67f3a268826b949d839dd7f46663ed0c5eb185ec ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 22, 2026
@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-ledger-conductor-channel branch from d9139a2 to 7612a8b Compare September 22, 2026 09:20
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 22, 2026
@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-ledger-conductor-channel branch from 7612a8b to 68d0ccf Compare September 22, 2026 09:56
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Sep 22, 2026
@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-ledger-conductor-channel branch from 68d0ccf to 7349e20 Compare September 22, 2026 10:20
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 22, 2026
@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-ledger-conductor-channel branch from 7349e20 to 4ec5c39 Compare September 22, 2026 12:07
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 22, 2026
An auto-nudge loop fires a full model turn on its owning session every
interval. PrWatchProbe already makes an unchanged tick free for a loop
that names one pull request; every other loop pays full price on every
tick, because the only typed reader is a probe and a probe cannot
interpret a transcript tail, a review comment or a log line.

The design inserts a cheap reader instead of making the expensive one
cheaper. Each tick the gateway scrubs and bounds the new evidence and
asks a nudge.wake decision point, on the decisions seam that already
ships, whether the owner must act now. Only a yes fires the main
session, and every error path -- provider failure, timeout, invalid
answer, low confidence, quiet-streak floor -- is the ungated timer, so
the judge can remove turns but never silence a loop.

The judge has exactly two verdicts, QUIET and WAKE. It cannot end a
loop: only a typed probe does that, a merged or closed pull request or a
work ledger with every item closed. If prose could mint a terminal
verdict then a comment saying "this is merged" would buy permanent
silence, which is worse than the compound bound below. A finished or
broken reading therefore WAKES with the verdict attached and the woken
session decides.

Section 5 states the compounding bound, not just the per-answer cost.
A QUIET answer sustained by untrusted evidence compounds, so an input
crafted to read as nothing-to-do can suppress delivery for the floor
times the interval: 7.5 hours at the default floor of 10 and a
45-minute interval. The floor enforces the bound because it counts
every tick including QUIET ones, resets only on a delivered turn, and
is configurable downward per point but never off.

Section 6 now reconciles the two monitoring documents this borrows from.
rfc-token-efficient-monitors owns the controller that adds fingerprints,
budgets, terminal outcomes and completed-turn accounting, and it leaves
the AutoNudge timer as the durable scheduling primitive, so terminal
outcomes are its vocabulary rather than the judge's.
rfc-consolidated-monitor merges three streams including autonudge.py;
the judge is a field on an existing loop's record rather than a second
loop, so there is nothing extra to merge, but its deletion list covers
irq's quiet-streak floor on the grounds that structured budgets subsume
it, so section 5's bound must be restated against whatever replaces the
floor.

The switch is not a new toggle. nudge.wake registers in gate.POINTS with
a nudge_evidence scope in POINT_SCOPE_KEYS, so it draws its own row on
the Decisions card with no frontend edit. auto resolves to the Jev lane
only when the keystone consents and that scope is granted, otherwise to
the LLM lane; a pinned jev without the scope stores the brief and runs
the plain timer. Section 7 Q2 records that ruling.

This also withdraws rfc-ledger-conductor-channel, which Design Review
correctly found re-solved a wake gate rfc-conductor-work-ledger already
owns, on a second store, naming no incumbent. That work belongs to that
document's Phase 3 and Phase 4. The wake judge is the complementary
half: typed probe for typed facts, judge for prose evidence.

Verified at 80bd0a8: gate.py declares DECISION_POINT_NAMES, impl_jev.py
holds JevOracle and _to_wire, points/ carries six live points, and
types.py declares one wire type (class Choice, aliased Question) with no
noul and no score. nudge_wake.py, impl_llm.py, the nudge_evidence scope
and every decisions.nudge_wake.* key, quiet_streak_floor included, do
not exist. The tick hook _monitor_tick_is_quiet exists and its floor
_MAX_QUIET_STREAK is 10, the value 3.1 and 5 both defer to. PrWatchProbe
exists. The Decisions-card shape 3.6 mounts its row on is #12598, open
with auto-merge on and no approving review, pending at c181a6a.

Docs only, status draft.
@chenmingwei23
chenmingwei23 force-pushed the docs/rfc-ledger-conductor-channel branch from 4ec5c39 to 67f3a26 Compare September 22, 2026 13:34
@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ⚠️ review incomplete

GPT 5.6 did not produce a complete verdict for 4ec5c391a1fc13a850feb30eee28bdeda4af109c; inspect the workflow logs and re-run it.

This comment is updated in place on each push.

See the GPT 5.6 Review job logs; this commit has no completed GPT verdict.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 4ec5c391a1fc13a850feb30eee28bdeda4af109c: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 22, 2026
@iamwhatever
iamwhatever enabled auto-merge (squash) September 22, 2026 16:10
@iamwhatever
iamwhatever merged commit 0d77dd7 into main Sep 22, 2026
96 of 102 checks passed
@iamwhatever
iamwhatever deleted the docs/rfc-ledger-conductor-channel branch September 22, 2026 16:10
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 22, 2026
wknight1 pushed a commit to wknight1/KiroCrew that referenced this pull request Oct 5, 2026
…irodotdev#12735)

An auto-nudge loop fires a full model turn on its owning session every
interval. PrWatchProbe already makes an unchanged tick free for a loop
that names one pull request; every other loop pays full price on every
tick, because the only typed reader is a probe and a probe cannot
interpret a transcript tail, a review comment or a log line.

The design inserts a cheap reader instead of making the expensive one
cheaper. Each tick the gateway scrubs and bounds the new evidence and
asks a nudge.wake decision point, on the decisions seam that already
ships, whether the owner must act now. Only a yes fires the main
session, and every error path -- provider failure, timeout, invalid
answer, low confidence, quiet-streak floor -- is the ungated timer, so
the judge can remove turns but never silence a loop.

The judge has exactly two verdicts, QUIET and WAKE. It cannot end a
loop: only a typed probe does that, a merged or closed pull request or a
work ledger with every item closed. If prose could mint a terminal
verdict then a comment saying "this is merged" would buy permanent
silence, which is worse than the compound bound below. A finished or
broken reading therefore WAKES with the verdict attached and the woken
session decides.

Section 5 states the compounding bound, not just the per-answer cost.
A QUIET answer sustained by untrusted evidence compounds, so an input
crafted to read as nothing-to-do can suppress delivery for the floor
times the interval: 7.5 hours at the default floor of 10 and a
45-minute interval. The floor enforces the bound because it counts
every tick including QUIET ones, resets only on a delivered turn, and
is configurable downward per point but never off.

Section 6 now reconciles the two monitoring documents this borrows from.
rfc-token-efficient-monitors owns the controller that adds fingerprints,
budgets, terminal outcomes and completed-turn accounting, and it leaves
the AutoNudge timer as the durable scheduling primitive, so terminal
outcomes are its vocabulary rather than the judge's.
rfc-consolidated-monitor merges three streams including autonudge.py;
the judge is a field on an existing loop's record rather than a second
loop, so there is nothing extra to merge, but its deletion list covers
irq's quiet-streak floor on the grounds that structured budgets subsume
it, so section 5's bound must be restated against whatever replaces the
floor.

The switch is not a new toggle. nudge.wake registers in gate.POINTS with
a nudge_evidence scope in POINT_SCOPE_KEYS, so it draws its own row on
the Decisions card with no frontend edit. auto resolves to the Jev lane
only when the keystone consents and that scope is granted, otherwise to
the LLM lane; a pinned jev without the scope stores the brief and runs
the plain timer. Section 7 Q2 records that ruling.

This also withdraws rfc-ledger-conductor-channel, which Design Review
correctly found re-solved a wake gate rfc-conductor-work-ledger already
owns, on a second store, naming no incumbent. That work belongs to that
document's Phase 3 and Phase 4. The wake judge is the complementary
half: typed probe for typed facts, judge for prose evidence.

Verified at 80bd0a8: gate.py declares DECISION_POINT_NAMES, impl_jev.py
holds JevOracle and _to_wire, points/ carries six live points, and
types.py declares one wire type (class Choice, aliased Question) with no
noul and no score. nudge_wake.py, impl_llm.py, the nudge_evidence scope
and every decisions.nudge_wake.* key, quiet_streak_floor included, do
not exist. The tick hook _monitor_tick_is_quiet exists and its floor
_MAX_QUIET_STREAK is 10, the value 3.1 and 5 both defer to. PrWatchProbe
exists. The Decisions-card shape 3.6 mounts its row on is kirodotdev#12598, open
with auto-merge on and no approving review, pending at c181a6a.

Docs only, status draft.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants