feat(autonomy): deterministic lane-stop gate + operator STOP notification#969
feat(autonomy): deterministic lane-stop gate + operator STOP notification#969kyle-sexton wants to merge 4 commits into
Conversation
…tion Ship #535 members 3 and 4 (the ratified leave-week safety set); members 1, 2, and 5 remain deferred and out of scope. Member 3 — deterministic lane-stop gate. autonomy's first hook, a `Stop` event handler (hooks/lane-stop-gate.sh) that structurally intercepts every stop attempt of an opted-in lane and runs a deterministic completion self-check, converting "a lane that stops itself before its goal is met is a bug" from a prompt admonition into a mechanism. Unless completion is explicitly signaled — the exact sentinel token (default LANE-STOP-OK, standalone-matched) in the agent's final message, or a configured marker file — the first stop is blocked with a re-injected self-check (decision:"block" + reason). This counters the fabricated-context-percentage premature-stop failure (#576/#577): the reason states a self-estimated "~50% context" is not a completion condition. It is the settings-scoped, cross-session sibling of /goal's session-only condition (#481); a shell hook cannot re-run the /goal evaluator model, so completion is checked deterministically. Default OFF (a Stop-blocker must never engage for an interactive session), fail-open on unreadable stdin / missing jq / a non-Stop event (a SubagentStop never trips it), and bounded against runaway by the stop_hook_active one-block guard plus Claude Code's consecutive-block cap. Member 4 — operator notification. When a lane stops anyway after the one structural nudge, the gate treats it as a down/stuck lane, allows the stop (never wedges it), and alerts the operator via a self-contained hooks/lane-notify.sh (OS toast + terminal bell/OSC 9). Reach is local-machine only (no remote/Slack primitive exists). It reimplements rather than sources desktop-notification: a Stop hook's stdout is parsed for decision/reason and cannot use the terminalSequence field that plugin's Notification hook relies on, and cache-isolated plugins cannot source each other. No separate repeated-failure counter — a lane that keeps stopping re-fires this notification, and StopFailure API-error telemetry is already owned by claude-ops. Adds six userConfig options, the shared hooks/hook-utils.sh copy, README docs, and black-box tests (25 cases). shellcheck/shfmt/markdownlint clean; silent-skip, cross-plugin-drift, changelog-parity, and plugin-manifest gates pass locally. ## Related Refs #535 (members 3+4 per the ratified 2026-07-20T02:17:15Z triage-routing decision — members 1/2/5 remain deferred, not in scope). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 5m 40s —— View job PR Review:
|
|
Claude finished @kyle-sexton's task in 5m 20s —— View job Security Review — PR #969
Reviewed files changed in this PR. No GitHub Actions files are in scope. No SQL, template injection, or supply-chain concerns found. Three findings follow, ordered by severity. IMPORTANT — Gate bypass: sentinel revealed in the block reasonConfidence: HIGH The block reason string includes the exact sentinel token the agent must emit to pass the gate on a future attempt: An agent that reads its block reason (the documented purpose of a
This collapses the gate from "require deliberate standalone token" into "repeat the token we just gave you." The PR description calls this "residual risk," but revealing the token in the denial message makes the false-negative scenario trivial rather than residual — any agent that processes its block output will naturally exercise this path. Mitigation options (not required for this review, raising for awareness):
SUGGESTION —
|
|
Orchestrator review pass (rule 10 full-diff-read + post-green review). Reviewed the full diff personally (Stop-hook gate = high blast radius). Logic review: fail-open paths correct (empty stdin, missing jq, non-Stop event all allow-and-exit), default-OFF verified, one-nudge-then-notify bounded correctly against One real finding, fix requested before flipping ready: README.md's example lane-launcher invocation is not valid Claude Code CLI syntax: Verified against https://code.claude.com/docs/en/plugins-reference.md: The correct per-session-override mechanism per the same doc page is Please fix the README example to a verified-working invocation (test it empirically if the docs are ambiguous on the exact |
The README showed `claude --config lane_stop_gate_enabled=true ...`, which is invalid: `--config <key=value>` is an option of `claude plugin install` (persisting to pluginConfigs in user settings), not a flag on the session-launch command — a launcher copying it would fail. Replace it with the correct per-lane, session-scoped mechanism: `--settings` carrying a `pluginConfigs` override (inline JSON, applies to one session only, no persistence). Verified against the official docs (plugins-reference.md line "pluginConfigs[<plugin-id>].options ... reads it back from ... the --settings flag", cli-reference.md "--settings ... Path to a settings JSON file or an inline JSON string") and empirically against this machine's live settings.json, whose existing `pluginConfigs` entry is keyed by the marketplace-qualified id `source-control@melodic-software` — confirming the key is `autonomy@melodic-software`, not the bare name. Also note that the persistent `--config` install path would enable the gate for every session, defeating the default-OFF design. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 2m 38s —— View job PR Review —
|
|
Claude finished @kyle-sexton's task in 4m 26s —— View job Security Review — PR #969
This review covers the full diff (both commits on this branch). The latest commit ( IMPORTANT — Gate bypass: sentinel disclosed in the block reasonConfidence: HIGH REASON="...declare completion by emitting the exact token ${SENTINEL} on its own line..."The block reason interpolates the literal sentinel value and delivers it to the agent as the
This collapses the gate's semantic intent from "require deliberate completion signaling" into "repeat the token we handed you." The standalone-only match (the ERE boundary check) still holds technically, but it provides no meaningful lift: an agent cannot accidentally emit the sentinel, but one that reads the denial message will naturally do so deliberately. The block reason is operator-authored copy (not agent-supplied), so there is no injection surface here — this is a logic-level flaw only. Mitigation options:
SUGGESTION — Unvalidated CWD used to construct marker-file pathConfidence: MEDIUM
CWD=$(printf '%s' "$INPUT" | jq -r '.cwd // ""' 2>/dev/null | tr -d '\r')
[[ -n "$CWD" ]] && MARKER="${CWD%/}/$MARKER"
Impact is limited to false-negative (gate bypassed, lane stops early) — not privilege escalation. The absolute-path branch of the
SUGGESTION —
|
…on example The activation example literally named this org's marketplace (`autonomy@<org>`), which the autonomy plugin's portability contract forbids (validate-plugin-contracts.mjs: the plugin must stay org/fleet-agnostic — binding-seam owns org-specific instance data; cf. #531). plugin-gate caught it. Replace the literal with a `<marketplace>` placeholder and state the key format explicitly (`<plugin-name>@<marketplace-name>`). The verified fact — that the pluginConfigs key IS marketplace-qualified, not the bare plugin name — is preserved; only this org's specific marketplace name is removed. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 44s —— View job PR Review —
|
|
Claude finished @kyle-sexton's task in 2m 58s —— View job Security Review — PR #969 (HEAD
|
|
Full CI green (24/24 checks) after both fix rounds. Flipping to ready. Summary for the record: rule-10 full-diff-read found the core logic (Stop-hook gate, notification, tests) correct with no DEFER-FORBIDDEN-class issue. Post-green review surfaced one doc-accuracy defect (invalid |
|
Claude finished @kyle-sexton's task in 3m 6s —— View job Final Review —
|
|
Claude finished @kyle-sexton's task in 3m 19s —— View job Security Review — PR #969 (HEAD
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 73035c6b02
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| LAST=$(printf '%s' "$INPUT" | jq -r '.last_assistant_message // ""' 2>/dev/null) | ||
| # Escape any regex metacharacters in the (configurable) sentinel before use. | ||
| SENTINEL_RE=$(printf '%s' "$SENTINEL" | sed 's/[][\.^$*+?(){}|/]/\\&/g') | ||
| if printf '%s' "$LAST" | grep -qE "(^|[^A-Za-z0-9-])${SENTINEL_RE}([^A-Za-z0-9-]|$)"; then |
There was a problem hiding this comment.
Require a line-level completion token
When the final lane message merely mentions or negates the sentinel, this regex still treats it as completion because it matches the token anywhere as a standalone word. In particular, after the hook’s own nudge tells the agent about LANE-STOP-OK, a premature follow-up such as “I should not emit LANE-STOP-OK yet” will set SIGNALED=1 and allow the stop without a marker or verified completion. Require the sentinel to appear alone on a line (matching the prompt text) or use an unambiguous structured signal so discussion of the token cannot bypass the gate.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| # | Finding | Classification | Evidence | Reacted |
|---|---|---|---|---|
| 1 | Require a line-level completion token — inline mention/negation of the sentinel should not fire the gate | VALID — fixed in a6335f4a |
Detection at lane-stop-gate.sh:71 matched the sentinel as a standalone word anywhere in the final message, so "I should not emit LANE-STOP-OK yet" set SIGNALED=1. Changed the match to require the token alone on its own line (^[[:space:]]*<sentinel>[[:space:]]*$), which is exactly the "emit the exact token … on its own line" instruction the block reason already gives — detection and the documented contract now agree. New regression tests: Case 13 ("I should not emit LANE-STOP-OK yet" → blocked) and Case 14 (token alone on an indented line amid other lines → allowed). Full suite 17/17 green. |
👍 |
Scope note (so this reply does not over-claim): the line-level fix closes the inline mention/negation false-positive you flagged. It does not address the separate concern that the block reason discloses the literal token, letting an agent emit it on its own line with no further work — that is the security-review IMPORTANT finding, a distinct design tradeoff (hiding the token from the nudge would degrade a legitimately-complete lane's ability to discover the completion protocol), classified separately and left for the author to decide. The gate remains a bounded one-nudge-then-allow+notify speed-bump by design, not a hard stop.
The completion-token detection matched the sentinel as a standalone word *anywhere* in the final message, so a turn that merely mentioned or negated the token inline (e.g. "I should not emit LANE-STOP-OK yet") set SIGNALED=1 and authorized the stop — a false positive in the very mechanism the gate exists to make reliable. The block reason already instructs the agent to emit the token "on its own line"; detection now enforces that same contract (the token alone on a line, surrounding whitespace allowed), so inline discussion of the token can no longer bypass the gate. Erring toward a missed genuine signal over a spurious one is the correct bias here: a missed signal is bounded (one nudge, then allow + notify), whereas a spurious match defeats the gate outright. Tests: the sentinel-present cases (3, 11) now place the token on its own line — a contract tightening, not a weakening-to-pass. Adds Case 13 (inline mention/negation → blocked) and Case 14 (token alone on an indented line amid other lines → allowed).
|
Claude finished @kyle-sexton's task in 2m 37s —— View job PR Review —
|
|
Claude finished @kyle-sexton's task in 3m 16s —— View job Security Review — PR #969 (HEAD
|
|
Classification of the HEAD (
This pass fixed the one core-purpose correctness finding (#1 — the gate firing on a mention of its own token). Findings #2–#5 are a genuine design tradeoff (#2) plus defense-in-depth/doc follow-ups (#3–#5) with no |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a6335f4ac1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # completion condition. Emitted as the documented Stop stdout decision. | ||
| REASON="Autonomy lane-stop gate: you attempted to stop, but this lane's completion condition is not yet signaled. A lane that stops itself before its stated goal is met is a bug. Do NOT stop on a self-estimated context percentage, a turn count, or a vague sense that enough was done — none of those is completion. Either (1) continue working toward the lane's stated goal, or (2) if the goal is genuinely and verifiably met, declare completion by emitting the exact token ${SENTINEL} on its own line (or by creating the configured completion-marker file), then stop. This is your one automated nudge; if you stop again without signaling completion, the operator will be alerted that the lane went down." | ||
|
|
||
| jq -nc --arg r "$REASON" '{decision:"block", reason:$r}' |
There was a problem hiding this comment.
Emit telemetry for lane-stop outcomes
When HOOK_TELEMETRY_SINK is configured, this new wired hook never calls hook::emit_telemetry, so both the meaningful blocked outcome here and the subsequent allowed-with-operator-notification outcome are invisible to the fleet’s hook observability pipeline. This prevents operators from measuring premature lane stops or correlating the local alert with lane activity; add fire-and-forget telemetry for the gate’s evaluated outcomes.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| # | Finding | Classification | Evidence | Reacted |
|---|---|---|---|---|
| 1 | Gate never calls hook::emit_telemetry, so lane-stop outcomes are invisible to HOOK_TELEMETRY_SINK |
VALID (defer) — real convention gap; implementation is an author design call, not a mechanical consistency fix | Telemetry emission IS an established, consistent convention: 26 non-test call sites across 12+ plugins (formatters, guardrails, claude-ops audits, desktop-notification), and hook::emit_telemetry is already present in plugins/autonomy/hooks/hook-utils.sh:391. But no autonomy hook emits yet, and every one of the 26 emitters ships a published docs/conventions/hook-telemetry/data/<hook>.schema.json — there is none for this gate. Adding it here is therefore not a one-line trailer; it requires new design. |
👍 |
Why deferred to the author rather than fixed in this round (same posture as this PR's other design-tradeoff finding). Wiring this correctly means making, not guessing, these decisions:
- New published data schema. The convention (
docs/conventions/hook-telemetry/README.md) makesdata/<hook>.schema.jsonthe source of truth, discovered by the envelope'shookvalue. Alane-stop-gateproducer needs its own schema artifact authored and committed. - Outcome→status mapping has a gap. The documented status set is
ok | error | skipped | blocked.blockedcovers the first-attempt block (L123) cleanly, but the outcome this finding most wants to measure — the nudged, still-stopping down-lane that is allowed with an operator alert (L105–114) — has no home in that set. Distinguishing it from an ordinary completion-signaled allow (L95,ok) requires a designeddata.outcomefield, which is a schema decision. - Fail-open exit-path restructure. The gate has multiple early
exit 0paths (default-off L42, fail-open stdin/non-Stop L46/L58, silent allow L95, notify+allow L114, block L123) and captures noEPOCHREALTIMEstart. Emitting per-outcome needs either an EXIT trap over an outcome variable or emit calls at each branch, plus aSTARTsnapshot — a deliberate restructure of a fail-open, security-sensitive hook, not a drop-in. - Security-conscious field selection. This exact file carries three open security findings (sentinel disclosure, CWD/marker path traversal,
git -CCWD). Thedatapayload must be designed to NOT surface the sentinel token or an unsanitized CWD into the sink envelope.
Net: the convention gap is real and worth closing, but it is a scoped design task (schema + status vocabulary + exit-path/telemetry restructure + security review of the payload), so it is left for the author to design rather than have a review-round worker invent a schema.
|
🔒 merge-drive lane claiming this PR for on-branch fix work this cycle (amendment-round: 16, worker tier). Will fix clear branch-owned CI/bot findings, defer non-DEFER-FORBIDDEN items with reply+resolve, push, then re-check the merge gate. |
What
Ships the two cheap, additive hooks the operator ratified as the leave-week safety set for the
"operational resilience" umbrella (#535): member 3 (deterministic lane-stop gate) and member 4
(operator notification on a genuine lane stop). Both land inside the
autonomyplugin — its firsthooks/directory — and touch nothing else.Member 3 — deterministic lane-stop gate (
hooks/lane-stop-gate.sh,Stopevent)"A lane that stops itself before its goal is met is a bug" was only a prompt admonition. This hook
fires on every stop attempt of an opted-in lane and structurally intercepts it:
/goalevaluatormodel) — either the exact sentinel token (default
LANE-STOP-OK, matched standalone-only) inthe agent's final message, or the existence of a configured marker file. This is the
settings-scoped, cross-session sibling of
/goal's session-only completion condition (feat: /goal prompt-crafting skill — docs-conformant goal shape + mechanical length check #481); itsown docs name "a prompt-based Stop hook" as exactly this alternative.
(
decision:"block"+ reason). The reason directly counters the fabricated-context-percentagepremature-stop failure (loop skill / babysit-prs: rule 15's '~50% context indicator' has no real instrument behind it — self-reported estimates are fabricated guesses, not measurements #576/work-items(work): rule-20 context-estimate is a guess, not a measurement — caused a premature session stop (30% actual vs 55% claimed) #577): a self-estimated "~50% context", a turn count, or a vague sense
of "enough" is explicitly stated to not be completion.
Safety properties: default OFF (a Stop-blocking hook must never engage for an interactive
session); fail-open on unreadable stdin, missing
jq, or a non-Stopevent (aSubagentStopnever trips it); bounded against runaway by the
stop_hook_activeone-block guard plus Claude Code'sown consecutive-block cap. Scope: catches a graceful self-stop only — a closed laptop, a
killed process, or
/loopexpiry emit noStopevent and are members 1/2's job.Member 4 — operator notification (
hooks/lane-notify.sh)Fired by the gate at the genuine-stop juncture. OS-native toast (macOS/Linux) + best-effort terminal
bell + OSC 9, local machine only (no remote/Slack/push primitive exists in the marketplace yet).
It reimplements rather than sources
desktop-notificationbecause aStophook's stdout is parsedfor
decision/reasonand cannot use theterminalSequencefield that plugin'sNotificationhookrelies on (a genuinely different emission path: direct
/dev/tty), and because cache-isolated pluginscannot source each other at runtime. No separate repeated-failure counter was built — a lane that
keeps stopping simply re-fires this notification, and
StopFailureAPI-error telemetry is alreadyowned by
claude-ops(avoids that plugin's in-flight PRs entirely).Files
plugins/autonomy/hooks/{lane-stop-gate.sh, lane-notify.sh, hooks.json, hook-utils.sh}(new)plugins/autonomy/hooks/{lane-stop-gate.test.sh, lane-notify.test.sh}(new, 25 black-box cases)plugins/autonomy/.claude-plugin/plugin.json(0.9.0 → 0.10.0, sixuserConfigoptions)plugins/autonomy/{CHANGELOG.md, README.md}Verification (local)
shellcheck (0.11.0) + shfmt (v3.13.1) + markdownlint clean; both test suites pass (15 + 10);
sync-hook-utils --check(13 copies),check-silent-skips,check-cross-plugin-source-drift,check-changelog-parity(--check+--check-bump), andvalidate-pluginsall pass.For the reviewer — please double-check
to emit the sentinel / touch the marker gets exactly one blocked turn, then is allowed to stop with
a notification. Bounded and non-fatal, and only for opted-in lanes. Acceptable?
hallucinating completion. Mitigated by requiring a deliberate standalone token (much higher bar
than a passive stop) and by the marker-file alternative; residual risk documented.
stop_hook_activereset semantics decide whether the gate re-arms for another nudge on a laternatural stop (per-attempt reset) or nudges only once per session (latched). The code and safety
bound are correct under either reading; the docs are worded to not over-claim a per-cluster reset.
If you know the authoritative semantics, confirm the wording.
StopvsSubagentStopfiring: the hook early-allows unlesshook_event_name == "Stop", so aTask-tool worker finishing can never trip the lane gate regardless of how the platform routes the
registration.
Issue linkage
No linked issue — this PR partially addresses the umbrella #535 (members 3+4 only) and closes nothing.
The umbrella stays open with members 1/2/5 deferred; see Related.
Related
Refs #535 (members 3+4 per the ratified 2026-07-20T02:17:15Z triage-routing decision — members 1/2/5
remain deferred, not in scope).