Skip to content

feat(records): record what each session gated and the CPU its shell calls used - #747

Merged
moq-bot[bot] merged 15 commits into
mainfrom
quest/m0/benchmark/gating
Nov 17, 2026
Merged

moq-bot[bot] merged 15 commits into
mainfrom
quest/m0/benchmark/gating

Conversation

@kixelated

@kixelated kixelated commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Claims quest/m0/benchmark/gating.md and completes it.

Summary

Each session record now shows both kinds of gating mistake: a gated command that barely used the CPU, and a CPU-heavy command run outside the gate.

  • gated: one entry per outermost quest gate run, with its command (program names only, see below), when it asked to start, when it was admitted, when it ended, and the CPU milliseconds its systemd scope used. The gate now logs one gate line per run under $XDG_STATE_HOME/quest/waits. The record takes its gate-queue and gate-run waits from that line, and older two-line logs still read.
  • Reading a scope's CPU reliably: a probe showed systemd removed a finished scope's cgroup before the gate saw the exit in 1 of 20 runs. A measured scope (measuredRun in supervise.ts) now keeps a double-forked holder reading descriptor 5 until the gate has read cpu.stat. quest worktree run keeps the unmeasured handshake.
  • shell: the session's shell calls, grouped where they overlapped (in parallel, in the background, or across sub-agents in one cgroup). Each group's cpu is what the session's own cgroup used from the first call's start to the last call's end. Gate scopes sit outside that cgroup, so gated commands never count there.
    • Quest's PreToolUse and new PostToolUse hooks sample the cgroup's usage_usec around each Bash call into $XDG_STATE_HOME/quest/calls. That covers Claude Code (plus PostToolUseFailure) and Codex. Claude Code's Stop and SubagentStop hooks also record which background shell tasks still run, which closes background calls.
    • A sample counts only where the cgroup holds the session alone: one process tree whose top is the harness (the process CLAUDE_PID names, where Claude Code sets it), every other branch hanging from it, with daemonized leftovers allowed. A terminal's scope and macOS record null.
  • quest usage --json prints both lists, and parseRecord checks them.
  • Commands keep program names only: a record's command is each program the command ran, in order and once, with the subcommand of programs that take one (git, cargo, just, nix, npm, gh, quest, …), such as cd; cargo build; tail. Arguments, assignments (GH_TOKEN=…), redirections, quoted text, and here-document bodies never leave the host (src/core/command.ts). A shell's -c script under quest gate is read the same way.
    • The shell parse is not what keeps secrets in: a program counts only as a shell builtin or one the host can run (on PATH, or by its path after the script's literal cds, runs in src/cli/program.ts), else ?. A subcommand counts only from a fixed list (VERBS: build, check, test, …), else it is dropped. parseRecord refuses any other shape, so a misread script costs the audit detail, never a secret.
  • The mistake audit Plan now reads both lists each week, keeps the gating baseline, and counts nix build apart.
  • The README's quest setup section had a paragraph garbled into the PATH note; this restores it.

Public API and wire impact

  • SessionRecord (in quest/core) gains required gated: GatedRun[] | null and shell: ShellGroup[] | null. parseRecord and parseUpload refuse a record without them. A store must take this quest/core before it accepts uploads from this version (see Follow-ups).
  • New exports: GatedRun, ShellCall, ShellGroup, COMMAND_LIMIT, commandName, argvName, isCommandName.
  • quest setup installs PostToolUse (Bash) for both agents and PostToolUseFailure (Bash) for Claude Code. Rerun it after upgrading.
  • The wait log's gate lines change shape. Nothing is in production, so there is no migration or shim, in either direction.
  • Version skew: every host must upgrade. A dev build of this PR already wrote a "kind":"gate" line into one host's shared waits log, and main's readWaits refuses it, breaking quest usage and the installed record sender on that host until it runs this version. So every host takes this version together and reruns quest setup for the new hooks, and Kixel Code's store (kixel-ai/kixel.quest) takes this quest/core before any host sends from it. The host's waits log was left as it is.

Decisions

Taken by the coordinator while the user was away, as recommendations, not the user's decisions: each is a choice for review.

  • A --since window splitting a group (the round-3 checkpoint): ✅ read one extra day before the window (rejected: per-cgroup group IDs assigned in the hooks, as locked state on every shell call that killed sessions leave stale; merging as is with the split documented). readCalls now always reads the UTC day before the window's, then earlier days while a call it read began on one. The edge left: a background call handed off and finished within a single turn of over 24 hours, before the window's day-before, can still split its group at the window's start. It only touches quest usage --since; uploaded records start at the session's first transcript line.
  • What leaves the host of a command: ✅ program names, plus a subcommand from a fixed list for programs known to take one, each program checked against what the host can run (rejected: the first line cut to 200 characters, which still carries an inline token; pattern redaction, which is brittle; program plus any lowercase first argument, which keeps cd's directory or echo's word; trusting the shell parse alone, which two review rounds showed misreading here-documents and substitutions into recorded words). just and make recipes outside the list record as plain just or make, which loses some audit detail. The audit judges a command's class by what it ran, and cargo build against cargo check, just test against just check, or nix build stay apart, so names are enough. Each program is kept once, so cd a && cargo build && cd b is cd; cargo build. The store refuses any other shape, which bounds what leaves the host even from a buggy sender.
  • Orphaned processes in a session's cgroup (found in review): ✅ an orphan tree counts toward the session's CPU only where init or a systemd manager adopted it and its environment carries this session's CLAUDE_CODE_SESSION_ID or CODEX_THREAD_ID (rejected: counting any adopted orphan, which can charge another harness's work; refusing every orphan, which nulls the CPU of any session that daemonized a server).
  • Version skew: ✅ no compatibility shim (rejected: making main's readWaits skip unknown kinds). AGENTS.md says skip backwards compatibility while nothing is in production, so every host upgrades, as Public API says.

Choices for review

  • No ranked text report: the plan suggested reporting ranked lists. The audit reads --json, so ranking stays in its Plan rather than a new quest usage section.
  • Process-group gates record cpu: null: this covers macOS and Linux without a systemd user manager. Node exposes no reaped-children CPU, and Bun's would mean rewriting the group owner, for hosts where the session side is null anyway.
  • One group across sessions: a group shared by sessions in one cgroup, such as a parent's with its sub-agent's, is in each of their records. Each record lists only its own session's calls (so a sub-agent routed to another endpoint never leaks commands into its parent's record), with shared: true saying the CPU covers others' calls too.
  • When background calls end: a Claude Code background call runs until the first turn end that no longer lists its task, else its cgroup's last sample. So a long-running background server folds that session's later calls into one group. Codex calls end when the tool returns.
  • Identifying the harness: the cgroup's top must be the process Claude Code names in CLAUDE_PID, or a process named codex, since Codex names no PID. Without either, the sample is null.
  • A CPU reading never fails a gated run (from the OpenAI review): an unreadable or malformed cpu.stat warns on stderr and records cpu: null; the command's exit status stands. Containment and cleanup failures stay fatal.
  • Shared groups ranked apart (from the OpenAI review): the records keep each shared group's full CPU with shared: true, and the mistake audit ranks only unshared groups per command, listing shared ones as CPU no command owns. Splitting a shared group's CPU per session in the record would guess at an attribution the cgroup cannot give.
  • Window: gated counts runs that ended since the window's start, like waits, and shell counts groups that started since then. readCalls reads earlier retained days while a call in view began before them.

Validation

  • just check and just test pass: 1,392 tests, including new test/calls.test.ts, new record, hook, and gate tests, and 20 short real-scope runs that must all keep their CPU reading.

  • Before the fix: 20 short systemd-run --scope --collect commands lost their cpu.stat once (.tmp/race.ts). After it: 40 measured runs lost none, reading 33 to 35 ms for the same busy loop.

  • Real surface in a T3 Code session on Linux:

    • I piped real PreToolUse and PostToolUse JSON into the CLI's quest hook around a busy loop. /usr/bin/time said 3.65 CPU-s; the session cgroup's delta was 3,719 ms (the extra is the hook's startup and the harness).
    • quest gate /usr/bin/time ... of the same loop logged cpu: 3682 against time's 3.67 s, plus a 40 s queue wait under load.
    • quest usage --pr 747 --json then printed this session's record with that gated run, its gate-queue/gate-run waits, and the shell group (cpu: 3719). The printed records pass parseRecord.
  • sessionSample here returned this session's t3code-claude-*.scope. Given a harness PID that isn't the scope's top, or run inside a scope started by systemd-run with an inherited CLAUDE_PID, it returned null.

  • After the iterate on ace8dec: just check and just test pass (1,450 tests), including a new real-scope test that a command exiting 0 still returns 0 with cpu: null when the CPU read throws EACCES.

  • After the decisions: just check and just test pass (1,449 tests). New test/command.test.ts covers tokens in assignments, quotes, and arguments, here-documents and here-strings inside substitutions, wrappers, cd before relative programs, Windows paths, sh -c scripts, and a real PATH lookup. test/calls.test.ts adds the round-3 window boundary and orphan ownership.

  • After the queue's handback: merged main, dropped gate-record-cpu.md's links to the deleted gating quest, and gave the hooks test's sampled() fixture stub cat and ls on its own PATH. Before, the test failed as on the queue's NixOS runner with /usr/bin hidden by a tmpfs in a user namespace (command: "?" at test/hooks.test.ts:279); after, it passes there. just check, just test (1,491 tests), and quest check --base origin/main pass.

Caused by

  • moq-dev/quest#600 - spliced the PATH note into the footer-check paragraph of the README's quest setup section (found by reviewer)

Follow-ups

  • kixel-ai/kixel.quest: Kixel Code's record store takes the quest/core with gated, shell, and program-name commands, so it accepts and keeps records from this version, before any host sends from it. That work is in Kixel Code's quest tree, for /quest-plan there.

Models

Size: L
Implementer: Claude, claude-opus-5-5, high effort
Implementer: Claude Code, claude-opus-5-5 (iterate, S)
Implementer: Claude Code, claude-opus-5-5 (iterate, M)
Implementer: Claude Code, claude-opus-5-5, low effort (iterate, XS, arm B)

Review

Rewritten by quest review after each round.

  • Head: 73aefc3

  • Base: origin/main at 4249857

  • Risk: high, Adding required gated and shell fields changes the exported record schema and how parseRecord reads it, potentially breaking Kixel Code and existing stored records; reverting Quest cannot undo records already uploaded.

  • Classified: high by Codex, gpt-6.1-sol, low effort, session 01a1e8ef-9d6a-7450-a2be-07b6a488e3bf

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort

  • Session: 01a1e094-d0eb-7020-a5ae-34b91821aa48, round 6

  • Score: 10/10, Code inspection confirms the Windows fixture fix and finds no new issues; builds and tests were not run.

  • Rounds: 12; first round 3 (high 1, medium 2); findings 11 (high 2, medium 9): fixed 11, refuted 0, open 0

  • Findings: none

  • The last round's findings, at 77a1cd2:

    • Fixed: medium, test/hooks.test.ts:228: Create Windows-compatible executable fixtures; test/hooks.test.ts:228-229 now creates cat.cmd and ls.cmd on Windows, matching the default PATHEXT lookup in src/cli/program.ts:56.
  • Reviewer: Codex, gpt-6.1-sol, high effort, shadow

  • Session: 01a1e094-d0eb-7e43-aefa-ae0fdbea10ec

  • Rounds: 6; first round 1 (medium 1); findings 4 (high 3, medium 1): fixed 4, refuted 0, open 0

kixelated and others added 2 commits November 15, 2026 09:50
…alls used

Each session record gains `gated`, its outermost `quest gate` runs with
command, queue, run, and the CPU milliseconds the gate read from its
systemd scope, and `shell`, its shell calls grouped where they overlapped
with the CPU the session's own cgroup used meanwhile. The gate logs one
line per run, from which the record also takes its gate-queue and gate-run
waits. A measured scope keeps a holder process until the gate reads its
cpu.stat, since systemd otherwise removed the cgroup first about one run in
twenty. Quest's PreToolUse, new PostToolUse, and PostToolUseFailure hooks,
and Claude Code's Stop, sample the session cgroup's counter into
$XDG_STATE_HOME/quest/calls, only where that cgroup holds the session
alone. The mistake audit reads both lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Records show what each session gated, and what it should have feat(records): record what each session gated and the CPU its shell calls used Nov 15, 2026
kixelated and others added 3 commits November 15, 2026 10:27
…e the harness

A shared shell group now lists only the recording session's calls, with a
shared flag, so a sub-agent routed elsewhere never sends its commands
inside its parent's record. A sample counts only when the cgroup's top is
the harness Claude Code names in CLAUDE_PID, or a process named codex, so a
terminal-run Codex session no longer counts its terminal's other jobs.
Shell groups count from their start, which the window's logs always hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ting

# Conflicts:
#	quest/m0/benchmark/gating.md
#	quest/m0/gate-quick.md
…efore a window

A command line can carry a secret such as an inline token, and records leave
the host. Records now keep each program's name, and the subcommand of those
that take one, never arguments; the parser refuses anything else.

`quest usage --since` now reads the call log from the day before the window's,
so a background call handed off then and finished before the window's first
turn end no longer splits its group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: after the round-3 checkpoint, the coordinator's recommendations (taken in the user's absence, listed under the PR body's ## Decisions for review) are applied at cb19102:

  • readCalls always reads the UTC day before a --since window's day (option 1); the >24-hour single-turn edge stays documented.
  • A record's command keeps only program names, plus the subcommand of programs known to take one (src/core/command.ts), and parseRecord refuses any other shape, so no arguments or inline tokens leave the host.
  • No compatibility shim for the waits log's new gate lines: every host upgrades and reruns quest setup, and Kixel Code's store takes the new quest/core first.

The next review restarts the round count with quest review --new.

(Written by Opus 5.5)

kixelated and others added 3 commits November 15, 2026 10:49
…h here-document delimiters exactly

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p here-documents finding a substitution's end

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…relative programs, and skip here-strings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: the round-3 checkpoint of review session 01a1e083 found parse errors could still carry data into a recorded subcommand (git password from a misread here-string). Under the coordinator's recommendation to bound what leaves the host (PR body ## Decisions), the parse is no longer what keeps a secret on the host:

  • a program counts only as a shell builtin or one the host can run, on PATH or by its path (resolved after the script's literal cds), else ?;
  • a subcommand counts only from a fixed list in src/core/command.ts (build, check, test, …), and parseRecord refuses any other.

A misread script now costs the audit detail, never a secret. The round's two parse findings (here-strings, cd before a relative program) are fixed too. The next review restarts the count with quest review --new.

(Written by Opus 5.5)

kixelated and others added 2 commits November 15, 2026 11:10
…sewhere, and read Windows program paths

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he session

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 15, 2026 19:15
@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 27 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 95446e09-cb21-42f7-8001-dd5484de251e
Commits

Reviewing files that changed from the base of the PR and between 4249857 and 73aefc3.

Files selected for processing: 25

  • README.md
  • quest/m0/benchmark/gate-record-cpu.md
  • quest/m0/benchmark/gating.md
  • quest/m0/mistake-audit.md
  • src/cli/calls.ts
  • src/cli/gate.ts
  • src/cli/hooks.ts
  • src/cli/main.ts
  • src/cli/program.ts
  • src/cli/records.ts
  • src/cli/send.ts
  • src/cli/supervise.ts
  • src/cli/usage.ts
  • src/cli/waits.ts
  • src/core/command.ts
  • src/core/index.ts
  • src/core/record.ts
  • test/calls.test.ts
  • test/command.test.ts
  • test/gate.test.ts
  • test/hooks.test.ts
  • test/records.test.ts
  • test/send.test.ts
  • test/split.test.ts
  • test/usage.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — the README's example record fails parseRecord; one small README nit.

Reviewed changes

Reviewed the full diff at ace8dec: session records gain gated and shell, commands are reduced to program names, and new hooks and gate plumbing feed both lists.

  • Program-name commands — src/core/command.ts keeps each program and a fixed-list subcommand. A program counts only as a builtin or something runs finds on the host, else ?. parseRecord enforces the shape with isCommandName.
  • Record schema — SessionRecord requires gated: GatedRun[] | null and shell: ShellGroup[] | null, validated by parseGated and parseGroup.
  • Call sampling — PreToolUse, PostToolUse, PostToolUseFailure, and Claude's Stop/SubagentStop log samples under $XDG_STATE_HOME/quest/calls. sessionSample counts a cgroup only when the harness is its top and orphans carry the session's env. shellGroups charges overlapping calls the cgroup's CPU delta.
  • Gate CPU — measured scopes keep a double-forked holder on fd 5, so cpu.stat survives until the gate reads it. The gate logs one kind: "gate" line per run, and records derive gate-queue/gate-run waits from it.
  • Docs and quests — README session-record and quest setup sections, the mistake audit Plan, and removal of the finished gating.md quest.

I checked the external hook contracts against Codex's hooks reference (PostToolUse and tool_use_id on both tool events) and Claude Code's (tool_use_id on PreToolUse). The holder process is reaped by the existing kill(group) after the CPU read, so nothing outlives the run.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread README.md Resolved
Comment thread README.md Resolved

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: ace8dec5c667d8a3e1e6b24caec77d9e4e583236.

Host-side summarization and distinct gates are a good direction. The existing README example finding is separate. Static review only; no hook integration or systemd tests run.

…be read, and rank shared groups apart

A failed cpu.stat read now warns and records cpu: null instead of failing a
command that exited 0. The mistake audit ranks only unshared shell groups per
command and lists shared ones apart. The README's example record now parses,
and the footer-check paragraph names its subject.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Addressed the OpenAI review in 1682969:

  • CPU sampling never changes the gate's exit status: in contain (src/cli/supervise.ts), a failed cpu.stat read now warns on stderr and leaves cpu: null. Containment and cleanup failures stay fatal. New test in test/gate.test.ts ("keeps a successful command's exit status when its CPU cannot be read") runs a real measured scope whose reader throws EACCES and expects { status: 0, cpu: null }.
  • Shared groups ranked apart: quest/m0/mistake-audit.md now ranks only unshared shell groups per command and lists shared groups apart, as CPU no command owns. The record keeps the group's full CPU flagged shared: true. The cgroup can't split that CPU by session, so the consumer has to separate it.

(Written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI) · 1682969732f2e666ed4ed647b5be22a01bf1e802

Both findings are fixed. A CPU-sampling error now warns and leaves cpu: null without changing the command’s status; the regression checks an exit-0 command. The audit now ranks only unshared groups by command and reports shared groups separately rather than attributing the whole group to a parent or child. No new blocker in this delta. Static re-review only; I did not run the systemd tests.

pullfrog[bot]
pullfrog Bot previously approved these changes Nov 17, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the delta since ace8dec: the prior README findings and the OpenAI review's two findings are addressed.

  • CPU read never fails a gated run — contain in src/cli/supervise.ts now warns on stderr and leaves cpu: null when reading the cgroup's usage throws, instead of setting failure. RunOptions.usage lets a test inject the reader, and a new test/gate.test.ts case checks that an exit-0 command still returns { status: 0, cpu: null }.
  • Shared groups ranked apart — the mistake audit Plan ranks only unshared shell groups per command and lists shared groups as CPU no command owns.
  • README fixes — the example record's command is git status, which parseRecord accepts, and the restored quest setup paragraph now opens with "The footer check reads…".

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 17, 2026
@moq-bot

moq-bot Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

quest land watch: the merge queue removed this pull request at 1682969 because its merge_group build failed (failed_checks).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

Queue / Queue build / Build (job) failure at step Check that the PR leaves its quest accurate:

…in/git config --local --get-all includeif.gitdir:/github/workspace/.git/worktrees/*.path
/github/runner_temp/git-credentials-8780be33-c056-4b8d-9dab-a8dfbff65a8a.config
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --unset includeif.gitdir:/github/workspace/.git/worktrees/*.path \/github\/runner_temp\/git\-credentials\-8780be33\-c056\-4b8d\-9dab\-a8dfbff65a8a\.config
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
Removing credentials config '/var/cache/github-runner-work/_temp/git-credentials-8780be33-c056-4b8d-9dab-a8dfbff65a8a.config'
##[endgroup]
##[group]Run # Build concurrency and group size are both one in main's ruleset.
�[36;1m# Build concurrency and group size are both one in main's ruleset.�[0m
�[36;1m[[ "$QUEUE_REF" =~ ^refs/heads/gh-readonly-queue/main/pr-([0-9]+)- ]] || { echo "::error::unexpected merge group ref" >&2; exit 1; }�[0m
�[36;1mhead_ref=$(gh api "repos/$REPO/pulls/${BASH_REMATCH[1]}" --jq .head.ref)�[0m
�[36;1m# The base's skills would reject a PR that adds, removes, or changes a skill's stub. The PR's�[0m
�[36;1m# own binary checks the stubs against the PR's skills in `just check` below, in this same job.�[0m
�[36;1m./quest-base/bin/quest --root "$GITHUB_WORKSPACE" check --no-stubs --base "$BASE" --branch "$head_ref"�[0m
shell: /nix/store/lfm00fnq66xfgbn099vqckabyl5jllz6-bash-interactive-5.3p20/bin/bash -e {0}
env:
  BASE: 42498572e41fc3638dce256abc0c0ab76ee5c7ce
  GH_TOKEN: ***
  REPO: moq-dev/quest
  QUEUE_REF: refs/heads/gh-readonly-queue/main/pr-747-42498572e41fc3638dce256abc0c0ab76ee5c7ce
##[endgroup]
quest: quest/m0/benchmark/gate-record-cpu.md:20: link does not resolve: /quest/m0/benchmark/gating.md
quest: quest/m0/benchmark/gate-record-cpu.md:46: link does not resolve: /quest/m0/benchmark/gating.md
quest: quest/m0/benchmark/gate-record-cpu.md:46: requires /quest/m0/benchmark/gating.md, which is not a quest document; make an outside condition its own quest
##[error]Process completed with exit code 1.
Post job cleanup.
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git version
git version 2.56.0
Temporarily overriding HOME='/var/cache/github-runner-work/_temp/5e2f89d2-4ab3-4965-b5ef-eaab32425380' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --global --add safe.directory /var/cache/github-runner-work/quest/quest
Removing SSH command configuration
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
Post job cleanup.
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git version
git version 2.56.0
Temporarily overriding HOME='/var/cache/github-runner-work/_temp/4b4c6325-d737-4f6b-8480-ea913234f0bb' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --global --add safe.directory /var/cache/github-runner-work/quest/quest
Removing SSH command configuration
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
A job completed hook has been configured by the self-hosted runner administrator
##[group]Run '/nix/store/qrn6ds64bzvwbm5nwf245ifz2c6jb9xw-ci-cache-gc.sh/bin/ci-cache-gc.sh'
shell: /nix/store/lfm00fnq66xfgbn099vqckabyl5jllz6-bash-interactive-5.3p20/bin/bash --noprofile --norc -e -o pipefail {0}
##[endgroup]
evicted 0 objects and 0 action results (0 B logical); 49.9 GiB logical remain
Cleaning up orphan processes

Queue / Queue (job) failure at step Require the build to pass:

Current runner version: '2.338.2'
##[group]Runner Image Provisioner
Hosted Compute Agent
Version: 20261103.629
Commit: 15dbd9ccb5dbf34fae002b6ed1a2670f52df37bc
Build Date: 2026-11-03T17:26:59Z
Worker ID: {bafc3b5e-5643-4984-9b23-26dee503d3a3}
Region: eastus2
Cloud: Azure
##[endgroup]
##[group]Operating System
Ubuntu
26.04.1
LTS
##[endgroup]
##[group]Runner Image
Image: ubuntu-26.04
Version: 20261111.196.1
Included Software: https://github.com/actions/runner-images/blob/ubuntu26/20261111.196/images/ubuntu/Ubuntu2604-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu26%2F20261111.196
##[endgroup]
##[group]GITHUB_TOKEN Permissions
Contents: read
Metadata: read
PullRequests: read
##[endgroup]
Secret source: Actions
Cache mode: write
Prepare workflow directory
Prepare all required actions
Complete job name: Queue
##[group]Run [[ "$RESULT" == success ]] || { echo "::error::the queue build ended $RESULT" >&2; exit 1; }
�[36;1m[[ "$RESULT" == success ]] || { echo "::error::the queue build ended $RESULT" >&2; exit 1; }�[0m
shell: /usr/bin/bash -e {0}
env:
  RESULT: failure
##[endgroup]
##[error]the queue build ended failure
##[error]Process completed with exit code 1.
Cleaning up orphan processes

(written by quest land watch)

@moq-bot

moq-bot Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

codex, gpt-6.1-sol, medium effort land fixer note:

The queue log reports:

273: quest: quest/m0/benchmark/gate-record-cpu.md:20: link does not resolve: /quest/m0/benchmark/gating.md
274: quest: quest/m0/benchmark/gate-record-cpu.md:46: link does not resolve: /quest/m0/benchmark/gating.md
275: quest: quest/m0/benchmark/gate-record-cpu.md:46: requires /quest/m0/benchmark/gating.md, which is not a quest document; make an outside condition its own quest
276: Process completed with exit code 1.

Merged base and committed the mechanical stale-reference repair as dc14873cb6bd05c7f70af2551bef6eec817f1786 on pr-747. The surviving quest now links to src/cli/gate.ts and no longer requires the completed quest. No runtime code was changed by the repair.

Passed: nix develop --command just install; the base CLI checking this checkout with check --no-stubs --base base --branch pr-747; nix develop --command just check; nix flake check; nix develop --command bun run build; nix develop --command bun run compile.

The full nix develop --command just test run had 1490 tests pass, 5 skip, and one fail. A focused rerun with nix develop --command bun run vitest run test/hooks.test.ts -t "the tool hooks sample each shell call" reproduced the failure at line 279: expected command "cat" and "ls", received "?". The new sampled() fixture hard-codes PATH=/bin:/usr/bin; neither directory contains cat or ls on this Nix runner. Adjusting that test setup needs a change outside the permitted mechanical categories, so this remains a handback. The shell and lifecycle stages after vitest were not reached by just test.

(written by codex, gpt-6.1-sol, medium effort)

@moq-bot

moq-bot Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

codex, gpt-6.1-sol, medium effort land fixer: Queue quest validation failed on links to the deleted gating quest; after repairing them, nix develop --command just test fails at test/hooks.test.ts:279: expected command "cat"/"ls", received "?" bec. Recommendation: Keep the stale-reference repair at dc14873cb6bd05c7f70af2551bef6eec817f1786; make the hook test use portable executable fixtures or the dev-shell PATH, then rerun just check and just test in nix devel

quest: quest/m0/benchmark/gate-record-cpu.md:20: link does not resolve: /quest/m0/benchmark/gating.md
quest: quest/m0/benchmark/gate-record-cpu.md:46: link does not resolve: /quest/m0/benchmark/gating.md
quest: quest/m0/benchmark/gate-record-cpu.md:46: requires /quest/m0/benchmark/gating.md, which is not a quest document; make an outside condition its own quest
##[error]Process completed with exit code 1.

(written by codex, gpt-6.1-sol, medium effort)

kixelated and others added 3 commits November 17, 2026 01:50
…ok test's programs

gate-record-cpu.md linked to and required quest/m0/benchmark/gating.md,
which this PR deletes; it now names measuredRun instead. The hooks test's
sampled() fixture put /bin:/usr/bin on PATH, where a NixOS runner has no
cat or ls; it now builds its own stub programs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the delta since 1682969: a merge of main, one quest doc fix, and one test fixture fix.

  • Dropped links to the deleted gating quest — quest/m0/benchmark/gate-record-cpu.md now names measuredRun in src/cli/supervise.ts in place of the link to gating.md, and drops its ## Required entry for it. No other file in the tree links to gating.md.
  • Stubbed the hook test's programs — sampled() in test/hooks.test.ts writes executable cat and ls stubs into a temp directory and makes it the whole PATH. Hosts that keep neither in /bin or /usr/bin, such as NixOS, now resolve them. On Windows the stubs end in .cmd, which which in src/cli/program.ts finds through its default PATHEXT.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@moq-bot
moq-bot Bot added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 9986a8c Nov 17, 2026
19 checks passed
@moq-bot
moq-bot Bot deleted the quest/m0/benchmark/gating branch November 17, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant