From 68d41fbe4da5d6c8b30e00055b9c97fa1553226e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 12 Nov 2026 23:58:02 -0800 Subject: [PATCH 1/2] quest: plan asking the maintainer only what their answer changes From transcripts on three hosts and five repositories' PR timelines: about 500 answers a day, 74% taken as recommended, and 40% of PR lead time spent waiting for a merge pick. Plans wider standing picks, picks recorded on the PR, a landed list, milestone placement by rule, a headless loop with a push, one cross-repository decisions session, and a quest messages fix. Co-Authored-By: Claude Opus 5.5 --- quest/m0/ask-less/README.md | 122 ++++++++++++++++++++++++ quest/m0/ask-less/decisions-session.md | 30 ++++++ quest/m0/ask-less/headless-loop.md | 44 +++++++++ quest/m0/ask-less/landed-list.md | 30 ++++++ quest/m0/ask-less/messages-codex-tui.md | 31 ++++++ quest/m0/ask-less/milestone-by-rule.md | 20 ++++ quest/m0/ask-less/recorded-picks.md | 33 +++++++ quest/m0/ask-less/standing-picks.md | 53 ++++++++++ quest/m0/mistake-audit.md | 7 ++ 9 files changed, 370 insertions(+) create mode 100644 quest/m0/ask-less/README.md create mode 100644 quest/m0/ask-less/decisions-session.md create mode 100644 quest/m0/ask-less/headless-loop.md create mode 100644 quest/m0/ask-less/landed-list.md create mode 100644 quest/m0/ask-less/messages-codex-tui.md create mode 100644 quest/m0/ask-less/milestone-by-rule.md create mode 100644 quest/m0/ask-less/recorded-picks.md create mode 100644 quest/m0/ask-less/standing-picks.md diff --git a/quest/m0/ask-less/README.md b/quest/m0/ask-less/README.md new file mode 100644 index 00000000..ac94f02e --- /dev/null +++ b/quest/m0/ask-less/README.md @@ -0,0 +1,122 @@ +# Ask the maintainer only what their answer changes + +## Goal + +Agents ask the maintainer only the decisions their answer is likely to +change. Everything else proceeds on a standing rule the maintainer set and +recorded, and stays reviewable after the fact. Each repository's work loop +runs without being launched by hand, and the maintainer opens one session +when they choose, or when a push says decisions wait. + +This epic's own work is its closing re-measure: rerun the baseline below +over a week of sessions on every host, and record the comparison in its +closing PR. + +Non-goals: releases and deploys, which keep their own approvals; the +online reviewers; changing what reviews a PR before it is ready. + +## Plan + +Decided 2026-11-12 (maintainer, /quest-plan), from the transcripts on the +maintainer's three Linux hosts and the forge records of moq-dev/moq, +moq-dev/moq.pro, moq-dev/quest, kixel-ai/rust2 and kixel-ai/rind: + +- Goal? ✅ As above (rejected: keep every merge pick and stop only + repeated and rubber-stamped questions; agents merge whatever they + recommend and the maintainer only sets direction). +- Scope? ✅ All four: merge picks by change class, never asking twice, + the loop running without the maintainer, and the machine-side blockers + (rejected: none dropped). +- Which merges land on a standing pick? ✅ Every PR the coordinator would + merge, except a protected list + ([standing picks](/quest/m0/ask-less/standing-picks.md)) (rejected: + plan PRs and bug fixes only; plan PRs only). The maintainer's override + rate barely varies by kind of change, and the pick does not catch + escapes: nearly every later culprit had been picked. +- Review after the fact? ✅ A list of what landed on a standing pick, + acted on by naming one ([landed list](/quest/m0/ask-less/landed-list.md)) + (rejected: session summaries only; the list plus a weekly second-model + audit). Either way the [mistake audit](/quest/m0/mistake-audit.md) + counts escapes by standing and maintainer pick, and proposes narrowing a + class whose escapes rise. +- Which planning questions stop? ✅ Milestone placement only + ([milestone by rule](/quest/m0/ask-less/milestone-by-rule.md)); which + follow-ups to plan is still asked (rejected: both; neither). +- The loop? ✅ Each repository's `/quest-run` runs headless on a schedule + and never asks, a push goes out when only decisions remain, and one + session asks them across every repository + ([headless loop](/quest/m0/ask-less/headless-loop.md), + [decisions session](/quest/m0/ask-less/decisions-session.md)) + (rejected: the same without a push; keep launching by hand, with one + cross-repository `/quest-complete`). +- Push channel? ✅ The Discord webhook kixel-ai/dotfiles' storage-alert + quest adds, so there is one place to look (rejected: a separate Discord + channel; ntfy; decide later). Quest runs a configured command; the + host owns the channel. +- How long does a recorded skip hold? ✅ Until the PR's head changes or the + maintainer lifts it + ([recorded picks](/quest/m0/ask-less/recorded-picks.md)) (rejected: + until lifted; until the next day). +- Which repositories? ✅ Every repository, opting out or adding protected + paths in its `CONTRIBUTING.md` as today (rejected: opt in per + repository; all but those running production services). +- Milestone? ✅ m0 (rejected: m1; fixes in m0 and the loop in m1). + +Host work, decided the same day, goes to kixel-ai/dotfiles' m0, since its +consolidate-hosts quest makes it the one repository for the maintainer's +hosts and archives kixel-ai/nixos: + +- The service running each loop and posting to Discord. +- The dev-shell guard, which refused 601 agent session starts since 11-03. + ✅ Keep its HEAD-on-origin check and accept the flake files at any version + merged on the default branch, not only its tip. First picked without the + HEAD check, then reversed once it turned out to reopen the fork-PR hole + kixel-ai/dotfiles@7202fdf closed (rejected: drop the HEAD check too; also + accept the agent's own flake changes; keep the guard as is). + +Baseline. Typed messages and answers come from `quest messages --json` run +on each host, 2026-10-20 to 11-13; sessions whose Codex TUI turns it +rejects were skipped ([messages fix](/quest/m0/ask-less/messages-codex-tui.md)). +Blocking time comes from the session logs since 11-03; PR stages from the +forge timelines of PRs created and merged 11-03 to 11-13. + +- 12,900 answers to agents' blocking questions, about 500 a day, with the + maintainer active 7 to 12 hours every day. 74% took the recommendation + unchanged, 8% picked another listed option, 12% typed something else + (9% in the first week, 15% in the last), and 5% had no recommendation. +- 2,752 answers where the agent recommended merging, 5.1% overridden; 4.2% + where the question named no API, breaking, decision, outside-author, or + release signal. Since the narrow standing pick landed on 11-07, about + 127 such answers a day. +- 345 merge-pick answers and 80 skips repeated an answer already given for + the same PR; 325 PRs were asked about three or more times. +- Milestone placement was asked 902 times (85% accepted), and which + follow-ups to plan 602 times (86%). +- Typed overrides that were not just the recommendation retyped: 13% asked + what something is or why, 12% cut scope or deleted, and 8% were about API + shape or naming. Those, goal and scope, and product knowledge are where + the maintainer's answer changed outcomes. +- Typed messages: 192 hand-run `/quest-complete`, 112 `/afk`, and status + nudges such as "any new prs?", "finish up", and "continue" among the + commonest short messages. +- 2,954 blocking questions since 11-03 kept their agents waiting 389 hours + in total: median 2.4 minutes, p90 16 minutes, 8 over six hours. +- 2,354 merged PRs: median lead time 5.8 hours, p90 25.6. Of the summed + lead time, implementation and local review took 38%, waiting from ready + to the first pick 40% (median 1.0 hour, p90 12.2), and the queue 21%. + A PR readied between 23:00 and 04:00 Pacific waited a median 5 to 9 + hours for its pick. 96% were first picked from the maintainer's account. +- 213 merged PRs (9%) are linked under a later PR's `## Caused by`, an + upper bound, since the mistake audit found most such links are not real + escapes. + +## Related + +- [Mistake audit](/quest/m0/mistake-audit.md) - tallies escapes by pick and + the answers taken as recommended each week +- [Late signals](/quest/m0/late-signals.md) - holds a head on a marked + decision comment, which recorded picks can share +- [Agent admission](/quest/m0/agent-admission.md) - limits what an + unattended loop starts to what the host and account can carry +- [Queue watcher](/quest/m0/queue-watcher/README.md) - re-enqueues the + merge queue's drops, 21% of merged PRs' lead time diff --git a/quest/m0/ask-less/decisions-session.md b/quest/m0/ask-less/decisions-session.md new file mode 100644 index 00000000..4005784a --- /dev/null +++ b/quest/m0/ask-less/decisions-session.md @@ -0,0 +1,30 @@ +# [L] Settle every repository's decisions in one session + +## Goal + +One `/quest-complete` session covers every repository the user works in: +it first says how many decisions wait, and where, then shows the +[landed list](/quest/m0/ask-less/landed-list.md), then asks the decisions +in its usual rounds, ranked across repositories by what each unblocks. It +edits each repository's quests in a checkout of that repository, one PR per +repository, as it does for one today. + +## Plan + +Decided 2026-11-12 (maintainer, /quest-plan, +[the epic](/quest/m0/ask-less/README.md)), with the +[headless loop](/quest/m0/ask-less/headless-loop.md): one session asks the +decisions every loop deferred. + +Agent's choices, for the implementer to revisit: + +- The repositories come from one list in the user's own Quest settings, + which the headless loop's host service reads too, rather than a scan of + checkouts. +- Reuse `quest waiting --json` per repository and merge the rankings, + rather than a cross-repository command, unless the merge needs one. + +## Related + +- [Recorded picks](/quest/m0/ask-less/recorded-picks.md) - keeps it from + asking what an earlier session settled diff --git a/quest/m0/ask-less/headless-loop.md b/quest/m0/ask-less/headless-loop.md new file mode 100644 index 00000000..b7cf8373 --- /dev/null +++ b/quest/m0/ask-less/headless-loop.md @@ -0,0 +1,44 @@ +# [L] Run a repository's quest loop headless and push when decisions wait + +## Goal + +One command runs a repository's `/quest-run` as a headless agent that never +asks: it takes the standing picks, starts ready quests, iterates and +merges, and defers every decision the maintainer owns, as `/quest-afk` +does. When it goes idle with nothing left but the maintainer's decisions, +it runs a command the user configured once, with how many decisions wait +in which repositories. A host service runs it on a schedule, and the +maintainer answers in the +[decisions session](/quest/m0/ask-less/decisions-session.md). + +Non-goal: the service and the channel, which the host owns: the +maintainer's are a kixel-ai/dotfiles quest posting to Discord. + +## Plan + +Decided 2026-11-12 (maintainer, /quest-plan, +[the epic](/quest/m0/ask-less/README.md)): ✅ headless loops with a push +when decisions wait (rejected: the same without a push; keep launching by +hand). The core runs a configured command, so it names no service. + +From 2026-10-20 to 11-13 the maintainer hand-ran `/quest-complete` 192 +times and `/afk` 112 times, each in one repository, and PRs readied +between 23:00 and 04:00 Pacific waited a median 5 to 9 hours for a pick. + +Agent's choices, for the implementer to revisit: + +- Build on `quest agent start`, which already runs a headless turn on the + CLI's own login and reports `limited` on a usage limit, rather than a + second runner. A limit ends the run cleanly; the next scheduled run + resumes it. +- Push once per transition to "only decisions wait", never per run, so a + schedule that finds nothing new says nothing. +- Keep the configured command in the user's own Quest settings, where + `quest setup` writes, never in a repository. + +## Related + +- [Agent admission](/quest/m0/agent-admission.md) - what an unattended + loop may start +- [Standing picks](/quest/m0/ask-less/standing-picks.md) - what it may + merge without the maintainer diff --git a/quest/m0/ask-less/landed-list.md b/quest/m0/ask-less/landed-list.md new file mode 100644 index 00000000..86e4e4d1 --- /dev/null +++ b/quest/m0/ask-less/landed-list.md @@ -0,0 +1,30 @@ +# [M] List what landed on a standing pick + +## Goal + +The maintainer sees, in one list across their repositories, every PR that +merged on a standing pick since they last looked, grouped by the class it +qualified under, one line each. Naming one starts its revert or its redo, +as the maintainer says, as a new PR. The +[decisions session](/quest/m0/ask-less/decisions-session.md) shows the +list first. + +## Plan + +Decided 2026-11-12 (maintainer, /quest-plan, +[the epic](/quest/m0/ask-less/README.md)): ✅ a landed list the maintainer +acts on (rejected: session summaries only; the list plus a weekly +second-model audit of a sample). + +Agent's choices, for the implementer to revisit: + +- A CLI command reading the [recorded picks](/quest/m0/ask-less/recorded-picks.md) + of merged PRs, with `--since` and `--json`, so the skill step stays + prose-light. +- "Since they last looked" is the time the list was last shown, kept in the + user's own state, never in a repository. + +## Required + +- [Recorded picks](/quest/m0/ask-less/recorded-picks.md) - which PRs merged + on a standing pick, and under which class diff --git a/quest/m0/ask-less/messages-codex-tui.md b/quest/m0/ask-less/messages-codex-tui.md new file mode 100644 index 00000000..2c8433f1 --- /dev/null +++ b/quest/m0/ask-less/messages-codex-tui.md @@ -0,0 +1,31 @@ +# [S] quest messages reads Codex TUI turns that carry only content kinds + +## Goal + +`quest messages` reads Codex 0.162 TUI sessions, whose user turns carry +`content_item_kinds`, such as `user.text` for typed text and +`agents_md.instructions` or `environments.environment_context` for +injected context, with no `content_item_metadata` provenance. It counts +`user.text` parts as typed, sets the injected kinds aside, and still fails +on a kind it does not know. + +## Plan + +Found 2026-11-12 while measuring [the epic](/quest/m0/ask-less/README.md)'s +baseline: the command failed on two of the maintainer's three hosts, 4 +sessions on one and 21 on the other, every one `codex-tui` 0.162.1, so the +baseline was taken with those sessions removed from a copy of +`~/.codex/sessions`. + +`readCodexMessages` in `src/cli/messages.ts` accepts `content_item_kinds` +only when every kind is `unknown`, a compaction's replay, and +`test/messages.test.ts` expects `["user.text", "unknown"]` to throw. Keep +that refusal for parts whose kind says nothing. + +Caused by [moq-dev/quest#626](https://github.com/moq-dev/quest/pull/626), +which required provenance on every typed Codex turn. + +## Related + +- [Mistake audit](/quest/m0/mistake-audit.md) - reads typed messages from + every host each week diff --git a/quest/m0/ask-less/milestone-by-rule.md b/quest/m0/ask-less/milestone-by-rule.md new file mode 100644 index 00000000..4891ee40 --- /dev/null +++ b/quest/m0/ask-less/milestone-by-rule.md @@ -0,0 +1,20 @@ +# [S] Place new quests by rule instead of asking + +## Goal + +`/quest-plan` places each new quest in the milestone a recorded rule or its +siblings' placement gives: a milestone rule the maintainer stated, in this +session or recorded in the tree, or the milestone of the epic or quests it +joins. It records the placement and the reason in the quest's Plan, lists +it in the summary, and asks only when neither covers the quest. + +## Plan + +Decided 2026-11-12 (maintainer, /quest-plan, +[the epic](/quest/m0/ask-less/README.md)): ✅ milestone placement stops +being asked; which follow-ups to plan is still asked (rejected: also plan +every recommended follow-up unasked; keep asking both). + +Milestone placement was asked 902 times from 2026-10-20 to 11-13, 85% taken +as recommended. `assets/skills/plan.md` asks it of every new quest today, +unless a rule from the same session covers it. diff --git a/quest/m0/ask-less/recorded-picks.md b/quest/m0/ask-less/recorded-picks.md new file mode 100644 index 00000000..8410acb5 --- /dev/null +++ b/quest/m0/ask-less/recorded-picks.md @@ -0,0 +1,33 @@ +# [M] Record merge picks and skips on the PR + +## Goal + +A merge pick or skip, the maintainer's or a standing one, is recorded on +the PR itself, so no session asks it again. `quest status` prints it, and +every coordinator honors it: a pick holds until the PR lands or hands back +a decision, and a skip holds until the PR's head changes or the maintainer +lifts it. A skip whose head changed is asked once more, saying what +changed. A standing pick records the class it qualified under. + +## Plan + +Decided 2026-11-12 (maintainer, /quest-plan, +[the epic](/quest/m0/ask-less/README.md)): ✅ a skip holds until the head +changes (rejected: until the maintainer lifts it; until the next day). + +Today a pick lives in the asking session and its sub-agents' briefs, and a +skip "stands for the session" (`assets/skills/run.md`, `complete.md`), so +from 2026-10-20 to 11-13 345 merge-pick answers and 80 skips repeated one +already given for the same PR, and 325 PRs were asked about three or more +times. + +Agent's choices, for the implementer to revisit: + +- Record each as a PR comment opening with a fixed marker naming the kind, + the head it was given on, and for a standing pick its class, beside + [late signals](/quest/m0/late-signals.md)' "Maintainer decision" + marker, rather than a label, which cannot carry the head. Only comments + from accounts with write access count. +- GitHub auto-merge enabled on the current head stays the pick's record + while it lasts; the comment covers the gaps, such as a PR waiting for + `/quest-iterate`. diff --git a/quest/m0/ask-less/standing-picks.md b/quest/m0/ask-less/standing-picks.md new file mode 100644 index 00000000..aca76bc9 --- /dev/null +++ b/quest/m0/ask-less/standing-picks.md @@ -0,0 +1,53 @@ +# [M] Merge everything outside a protected list on a standing pick + +## Goal + +A coordinator that would recommend `/quest-merge` for a PR runs it without +asking, unless the PR is on the protected list: + +- a public API or wire change, or a breaking (`!`) title +- a release, deploy, or backport, or anything that ships one +- a new dependency +- shipped agent rules or skills, such as Quest's own `assets/` +- an author without write access +- open decisions: a `## Waiting` section, or a plan PR holding decisions + the maintainer did not make in its interview + +A plan PR that records only the maintainer's own interview answers +qualifies, so adding or expanding quests no longer needs a pick by itself. +A repository's `CONTRIBUTING.md` can opt out, as today, or add protected +paths, such as deploy or billing configuration. The summary names each +PR merged this way, with the class it qualified under. + +## Plan + +Decided 2026-11-12 (maintainer, /quest-plan, +[the epic](/quest/m0/ask-less/README.md)): ✅ protected list (rejected: +plan PRs and bug fixes only; plan PRs only), in every repository with +`CONTRIBUTING.md` opt-out as today (rejected: opt in per repository; all +but production services). + +The standing pick of `assets/skills/run.md` (landed in +[moq-dev/quest#534](https://github.com/moq-dev/quest/pull/534)) covers only +PRs with no user-facing behavior change that add no quests. This +replaces that rule there, in `assets/skills/afk.md` and `merge.md` where +they restate it, and in this repository's `AGENTS.md`, whose merge line +quotes the old rule. + +Agent's choices, for the implementer to revisit: + +- Judge the diff, not the title, as today, and ask when unsure. A pick + stays standing only while each new head still qualifies. +- Record the class each merge qualified under, through + [recorded picks](/quest/m0/ask-less/recorded-picks.md) once it lands, + so the [landed list](/quest/m0/ask-less/landed-list.md) and the + [mistake audit](/quest/m0/mistake-audit.md) can tell standing picks from + the maintainer's. +- Before review, replay the rule over a sample of last week's merge-pick + questions and report how many it would have taken and how many of those + the maintainer overrode, from `quest messages --json`. + +## Related + +- [Mistake audit](/quest/m0/mistake-audit.md) - proposes narrowing a class + whose escapes rise diff --git a/quest/m0/mistake-audit.md b/quest/m0/mistake-audit.md index 4e4f792c..7e110db3 100644 --- a/quest/m0/mistake-audit.md +++ b/quest/m0/mistake-audit.md @@ -110,6 +110,13 @@ Check first next run: [late signals](/quest/m0/late-signals.md) lands. - Whether moq-dev/moq and moq-dev/moq.pro planned the runtime follow-ups of moq-dev/quest's 2026-11-11 run PR, and whether their escapes recurred. +- Once [standing picks](/quest/m0/ask-less/standing-picks.md) widen and + [recorded picks](/quest/m0/ask-less/recorded-picks.md) record their + class, count escapes separately for standing and maintainer picks, by + class, and propose narrowing a class whose escapes rise (maintainer, + 2026-11-12, [ask less](/quest/m0/ask-less/README.md)). Compare the + answers taken as recommended with that epic's baseline, read from + `quest messages` on every host. Baseline from the 2026-11-11 run (PRs updated 2026-11-09 to 11-11; typed messages 2026-10-26 to 11-11, classed by one sub-agent, approximate): From 87520846a6a2e090d4b83ee0f5cd57a468f7b0f6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 13 Nov 2026 13:34:01 -0800 Subject: [PATCH 2/2] quest: drop links to the finished agent-admission quest Co-Authored-By: Claude Opus 5.5 --- quest/m0/ask-less/README.md | 2 -- quest/m0/ask-less/headless-loop.md | 5 ++--- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/quest/m0/ask-less/README.md b/quest/m0/ask-less/README.md index ac94f02e..1ca14d26 100644 --- a/quest/m0/ask-less/README.md +++ b/quest/m0/ask-less/README.md @@ -116,7 +116,5 @@ forge timelines of PRs created and merged 11-03 to 11-13. the answers taken as recommended each week - [Late signals](/quest/m0/late-signals.md) - holds a head on a marked decision comment, which recorded picks can share -- [Agent admission](/quest/m0/agent-admission.md) - limits what an - unattended loop starts to what the host and account can carry - [Queue watcher](/quest/m0/queue-watcher/README.md) - re-enqueues the merge queue's drops, 21% of merged PRs' lead time diff --git a/quest/m0/ask-less/headless-loop.md b/quest/m0/ask-less/headless-loop.md index b7cf8373..b901544b 100644 --- a/quest/m0/ask-less/headless-loop.md +++ b/quest/m0/ask-less/headless-loop.md @@ -30,7 +30,8 @@ Agent's choices, for the implementer to revisit: - Build on `quest agent start`, which already runs a headless turn on the CLI's own login and reports `limited` on a usage limit, rather than a second runner. A limit ends the run cleanly; the next scheduled run - resumes it. + resumes it. `quest agent start` admits itself, so the loop starts only + what the host and account can carry. - Push once per transition to "only decisions wait", never per run, so a schedule that finds nothing new says nothing. - Keep the configured command in the user's own Quest settings, where @@ -38,7 +39,5 @@ Agent's choices, for the implementer to revisit: ## Related -- [Agent admission](/quest/m0/agent-admission.md) - what an unattended - loop may start - [Standing picks](/quest/m0/ask-less/standing-picks.md) - what it may merge without the maintainer