Repository navigation
feat!: review locally with the other vendor's model (quest review, /quest-review) - #271
Conversation
Quest-Claim: 6186cc2eca218f7e
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The skill reviews the branch locally with the other vendor's model, records the round in the PR body's ## Review section, pushes, and marks the PR ready. Merge addresses online findings already posted and never waits for a review. Deletes the finished local-review quest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er directory; give REVIEW.md every round Claude Code now loads no settings files, since a user's own allow rule (such as Bash(gh pr merge *)) ran under dontAsk, and Codex ignores execpolicy rules. Identical instruction files in sibling directories each keep their scope, and later rounds carry the base's current REVIEW.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… later rounds A resumed session keeps the system or developer instructions it started with, so a later round now restates instruction files that changed since the session was given them, and says when the base dropped REVIEW.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Neither CLI refreshes system or developer instructions on resume, and a user-turn restatement cannot override them, so a changed set of instruction files starts a new session whose follow-up still weighs the author's responses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Claude reviewer loads no settings, so a CLI default could not pick its model. quest review now runs claude-opus-5-5 or gpt-6.1-sol at medium effort, and with --strong, for a high-stakes change, claude-fable-5-1 or gpt-6-astra. The ids live in one MODELS table; --model and --effort override them. The reported model is the one passed, so the Codex rollout scraping and the stored model go away, and a new session can no longer inherit the old one's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The command's level sets its own effort and overrode --effort, so a medium review ran its first turn at high. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
Security findingsAdvisory findings (2)
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aff3bfa9cd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: aff3bfa
I agree with the independently reported P2 actual-base finding: assets/skills/review.md:17 invokes quest review without the PR's base, while src/cli/review.ts defaults to origin's default branch. A release/backport PR can therefore be recorded as reviewed against the wrong diff. Resolve and pass the PR's base on every round; add a nondefault-base workflow regression. Referencing that thread rather than duplicating an inline comment.
No additional actionable bugs found in this static pass. Direction is sound: a shared findings schema, explicit reviewer selection, and restarting sessions when base instructions change keep the local loop understandable. The skill rename/migration and optional-online-review policy are documented; the self-reported Review record is a policy choice, not an independently enforced attestation.
Verification: inspected CLI orchestration, instruction sourcing, resume/error handling, fake-CLI tests, and workflow changes. No tests or live reviewer CLI/security probes were run; reported model, cache, and sandbox behavior was not independently reproduced.
There was a problem hiding this comment.
🛡️ Codex Security Review · Automatically triggered
Here are some automated security review suggestions for this pull request.
Reviewed commit: aff3bfa9cd
ℹ️ About Codex security reviews in GitHub
This is an experimental Codex feature. Security reviews are triggered when:
- You comment "@codex security review"
- A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review
Once complete, Codex will leave suggestions, or a comment if no findings are found.
…cords and checks A PR against a release branch was reviewed against the default branch. A PR's author writes its body, so a fork author's own `## Review` record counted, and reviewing their head ran its checks with the reviewer's credentials. Only a record on a PR whose author has write access now counts; anyone else's head is reviewed locally by the merger, with its checks left to CI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In a repository whose `quest` runs from the checkout, reviewing an outsider's head ran its code before the reviewer's sandbox started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 8da37a2
The actual-base P2 referenced in my previous review is addressed: the shared guide now requires the PR's base via --base, and the new fake-CLI test covers fetching a release branch and using its instructions. No new actionable bugs found in this two-commit update; the base is unchanged.
The independent untrusted-checks and review-record trust findings are addressed at the workflow-policy level: outsider heads must run no local code, including their own quest, and their self-written review records do not count. Reusing the shared guide avoids divergent skill rules. These remain agent-followed rules, not a new CLI sandbox or authenticated attestation.
Verification: static inspection of both commits, the affected guide/skills, and the nondefault-base regression test. No tests or live CLI/security probes run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: c1c0ff6
No actionable bugs found in the CLI API change since 8da37a2. The --strong → --deep rename is consistent through option parsing, option forwarding, tier selection, model-conflict handling, tests, and usage guidance. Model/effort selection itself is unchanged; no additional compatibility machinery is needed for this still-unmerged command.
The prior actual-base fix and workflow trust guidance are unchanged. Verification: static inspection of this commit and surrounding dispatch/selection code; no tests or live reviewer CLIs run.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c1c0ff69fb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ad the head's config A worktree of a trusted clone is trusted, so Codex loaded the reviewed head's .codex/config.toml and started its MCP servers outside the sandbox. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: aca8b3b
No new actionable bugs found in this update since c1c0ff6. The independently reported P1 head-controlled Codex configuration issue is addressed at both launch sites: the shared codexFlags now passes --ignore-user-config to initial review and resume, and the fake-CLI test asserts it for both. Centralizing the isolation flags is the right direction; the earlier actual-base and workflow-policy fixes are unchanged.
Verification: static inspection of the one-commit delta, shared flag wiring, and test assertions. No tests or live Codex probes run; the reported trusted-worktree/MCP isolation and retained-login behavior were not independently reproduced.
|
Merge summary for aca8b3b:
(written by Claude Opus 5.5) |
* quest: plan the follow-ups from quest review (#271) A caller plans a background merge's follow-ups, quest status drops review-rounds, quest init prunes retired stubs, a live test checks the real reviewer CLIs run nothing a head plants, a condition quest tracks Codex's project-config skip, and a recurring check keeps the pinned review models current. land-review requires an App check run for runner heads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * quest: watch #49333 without posting upstream Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
quest init and quest uninstall recognized a stub only by exact match with a current skill's stub, so a repository that installed /quest-ready kept it after #271 removed the skill, and any stub whose description or setup link changed made init refuse to run. Both now recognize a stub of any version by its shape: frontmatter naming the skill, then the `quest skill <name>` pointer. Init removes such stubs for skills this version no longer ships and leaves older stubs of current skills as they are; uninstall removes stubs of any version. Symlinked entries and edited stubs stay untouched. Deletes the finished init-prune quest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(quest): claim quest/m0/init-prune.md Quest-Claim: c30a38f3e6af77c6 * fix(init): remove stale skill stubs by their shape quest init and quest uninstall recognized a stub only by exact match with a current skill's stub, so a repository that installed /quest-ready kept it after #271 removed the skill, and any stub whose description or setup link changed made init refuse to run. Both now recognize a stub of any version by its shape: frontmatter naming the skill, then the `quest skill <name>` pointer. Init removes such stubs for skills this version no longer ships and leaves older stubs of current skills as they are; uninstall removes stubs of any version. Symlinked entries and edited stubs stay untouched. Deletes the finished init-prune quest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(init): treat a stub with extra frontmatter fields as edited Skill frontmatter has only ever held `name` and `description`, so a stub with any other field, such as `allowed-tools`, was edited by a user and stays put on init and uninstall. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Closes the local review quest, as planned in kixelated/quest#267 with trial findings from kixelated/quest#270.
Summary
Before a PR goes up for review, the authoring session now has the other vendor's model review the committed branch read-only on this machine, fixes or refutes each finding, and only then pushes and marks the PR ready. GitHub's online reviewers become optional input that merge addresses but never waits for.
quest review(new CLI,src/cli/review.ts): it detects the running harness (CLAUDECODE/CODEX_THREAD_ID) and runs the other one.--harnessoverrides that; an unknown or ambiguous harness, or a missing or logged-out CLI, is an error. It pins each reviewer's model and effort in oneMODELStable:claude-opus-5-5orgpt-6.1-solat medium effort, and with--deep, for a high-stakes change,claude-fable-5-1orgpt-6-astra;--model/--effortoverride them. Round 1 runs the vendor's own review (codex exec review --baseorclaude -pwith/code-review <effort> <base>...HEAD), then resumes that session for one turn that applies the base'sREVIEW.mdand returns findings in one JSON schema (--output-schema/--json-schema). Later rounds resume the same session with the new head and--responses. A refutation stands only with evidence the reviewer cannot rebut. Session state is kept per branch in the clone's common Git dir.read-onlysandbox withapproval_policy="never",--ignore-rules, and--ignore-user-config(the user config's project trust would load the head's.codex/config.tomland start its MCP servers). Claude runs indontAskmode with--tools Read,Grep,Glob,Bash,Agent,--strict-mcp-config, and no settings files (--setting-sources ""), so only Bash that Claude Code itself classifies read-only runs.AGENTS.md/CLAUDE.md. Claude loads no settings sources; Codex gets-c project_doc_max_bytes=0. Instead, the base'sAGENTS.md/CLAUDE.mdfrom the root and each directory holding a changed path arrive via--append-system-prompt/-c developer_instructions. Neither CLI refreshes those on resume, so a change to the applicable files starts a new session, which still weighs--responses. The base'sREVIEW.md, read withgit show <base>:REVIEW.md, goes into each round's user turn and is revoked if the base drops it.CLAUDE_CODE_PROMPT_CACHE_TTL=1h; Codex keeps its default 30-minute retention (the Plan's 24h request is dropped, see Decisions). Every turn passes identical flags. Each turn's usage prints as ausageline./quest-reviewreplaces/quest-ready. It runs the checks and the review loop (with--deeponly for a high-stakes change such as security, a wire format or public API, or a risk of data loss; or records a trivial skip), rewrites## Review, pushes, and marks the PR ready with the existingquest readyhold, then offers/quest-merge./quest-start,/quest-iterate,/quest-complete,/quest-spawn,/quest-afk,/quest-merge, the README, anddocs/getting-started.mdnow call it.--base; the## Reviewrecord names the reviewer's model and effort),/quest-merge's waits, andCONTRIBUTING.md's review policy were rewritten: the## Reviewrecord naming the head satisfies the required review (counted only on a PR whose author has write access; anyone else's head is reviewed by the merger, running no code from it, with checks left to CI and aquestthe PR cannot modify), online findings are addressed at merge and never waited for or requested, and the every-fourth-round design checkpoint countsquest reviewrounds (--newrestarts the count after a design change). Land only reviewed heads now names the## Reviewrecord as kixelated/quest's policy example.codex-cloud-off.mdandhelpers/review-rules.mdwere updated.Impact
quest review(plain and--jsonoutput, each naming the reviewer's model and effort;--deepfor the stronger tier; exit 0, 2, or 1).quest-readyskill is renamedquest-review. Migrate by runningquest initto add thequest-reviewstub and deleting the old.claude/skills/quest-readyand.agents/skills/quest-readystubs, which now fail with "no skill named ready". Its precedent, takeover→iterate, did the same. This repository's stubs are renamed here.quest readyCLI hold command is unchanged. No wire or data format changes.Validation
just checkandjust testpass (618 + 426 tests).test/review.test.tsadds 14 tests using fakeclaude/codexCLIs. They cover harness detection, unknown harness, missing and logged-out CLIs, dirty or detached checkouts, read-only flags, and base-only instructions (head canary absent, nested scoping, identical siblings, symlink dedupe). They also cover the pinned models,--deep, and its conflict with--model, a nondefault--base(fetched, with that branch's instructions),REVIEW.mdin the follow-up, resume with responses,--responsesrules, revoking a removedREVIEW.md, a new session on instruction change, and schema parsing failures.claude2.1.312 andcodex-cli0.160.0:claude-opus-5-5andclaude-fable-5-1at--effort mediumanswered with that model inmodelUsage;gpt-6.1-solandgpt-6-astraatmodel_reasoning_effort="medium"answered with that model and effort in the rollout'sturn_context. Bogus ids fail on both (unrecognized_model/ 400). A probe with--effort mediumand/code-review lowran its review subagent at"effort":"low", so the command's level overrides--effortand now follows it.claude -pwith/code-review high <base>...HEADworks headless as a built-in command. Its first turn reports usage only inmodelUsage, and--resume <id> --json-schemareturnsstructured_output. Prompts must go on stdin, since--toolsis variadic and swallows a positional prompt.dontAskdeniedgit diff --output=<file>(no file written). With--setting-sources user, the maintainer's ownBash(gh pr merge *)allow rule rangh pr merge --help; with--setting-sources ""it was denied. Hidden-instruction probes confirmed the head'sCLAUDE.md(root and nested) is not loaded and the appended base text is.codex exec reviewthread shows the head'sAGENTS.mdinjected asproject-docby default and absent withproject_doc_max_bytes=0, whiledeveloper_instructionsreaches the review thread. A resume with a differentdeveloper_instructionskeeps only the first, which motivated the new-session rule.AGENTS.mdrule ("titles start withBASE:") was followed over the head's injected canary, andREVIEW.mddropped style findings. Round 2 dropped a fixed finding and kept an uncited refutation with a rebuttal./code-reviewturn runs as a local command), and its round 2 read 26,882/28,262 (95%).Review
codex-cli 0.160.0), modelgpt-6.1-solat medium effort, viaquest review --base origin/mainfrom this Claude Code session.aca8b3bdd5dccbf51ea66baaf23ae07e6c6b4b9b, round 10 of the session: no findings.c1c0ff6, the--deeprename, with the maintainer's decisions 5 and 6 in its responses): no findings. Codex online then posted a P1, reproduced and fixed inaca8b3b: in a worktree of a clone Codex trusts, the head's.codex/config.tomlloaded and its MCP servers ran outside the sandbox. Codex now skips the user's config (--ignore-user-config), whose project trust caused it;-c projects={}did not.8da37a2): no findings.e9f3eeb), high: on an outsider's head,questin this repository runs from the checkout, so its code ran before the reviewer's sandbox. The guide now runs no code from such a head, using aquestthe PR cannot modify with--root.aff3bfa): no findings. Online reviewers then posted three on that head, all fixed ine9f3eeb:/quest-reviewreviewed against the default branch instead of the PR's base (Codex P2, agreed by the maintainer account's automated review); it ran an outsider's checks locally (Codex security P1); and this PR had dropped "the author's own review does not count", so a fork author's self-written record counted (Codex security P1).7716a68), medium: Claude's review turn sent/code-review high, whose level overrides--effort, so a medium review ran its first turn at high. Verified by probe (above); the command now takes the selected effort.c60367d), low: a new session inherited the old session's recorded model. Fixed by removal: the round reports the model it passed, and the stored model and Codex rollout scraping are gone.967585c): restating changed instructions in the user turn can't override system-level ones, so a change now starts a new session.1636fe2): changed instructions never reached a resumed session. AREVIEW.mdremoved from the base was never revoked.de3d4e5): user allow rules widened Claude's read-only access, so it now loads no settings files and Codex gets--ignore-rules. Identical siblingAGENTS.mdfiles lost their scope. A staleREVIEW.mdstayed on later rounds.Decisions
Settled by the maintainer on 2026-10-28 (5 and 6 confirmed at merge):
Codex prompt-cache retention. Codex 0.160 has no setting for it, and current models offer only
30m(default) or12h, not the Plan's 24h.prompt_cache_options.ttl, then request12h.Claude reviewer settings.
--setting-sources ""), so the reviewer stays read-only whatever the user's allow rules, hooks, or plugins;--model/--effortpick a model.Round-4 design checkpoint.
/quest-reviewto mark ready.Reviewer model and effort. This supersedes the Plan's "quest hardcodes no model ids" (model and effort from the CLI's defaults): with no settings loaded, Claude had no configurable default.
claude-opus-5-5/gpt-6.1-solat medium effort by default, plus a stronger tier,claude-fable-5-1/gpt-6-astra, that the authoring agent picks for high-stakes changes (security, wire/API, data-loss risk), never routine work. Both tiers live in one table, selected by a flag rather than typed ids;--model/--effortoverride.Trust line.
## Reviewrecord counts and whose head may run code locally, matching who can edit a PR body.Stronger-tier flag name.
--deepeverywhere (CLI, help, tests,/quest-review, README, guide, this body), renamed from--stronginc1c0ff6.Choices made without asking
Instruction mechanism: CLI flags instead of a scratch worktree with restored files, because both CLIs hide project docs cleanly and the flags keep sessions in the real checkout. "Applicable" means the base's
AGENTS.md/CLAUDE.mdat the root and in every directory holding a changed path; a same-directory symlink is given once.--base <remote>/<branch>, defaulting toorigin's default branch fromls-remote --symref, fetched into its remote-tracking ref with an explicit refspec.Strict checkout inputs: a dirty or detached checkout is refused.
--responsesis required after a round with findings and refused when there is no earlier round.--responses -reads stdin.State is keyed by branch in
<git-common-dir>/quest/review/. Switching the reviewer harness starts a new session.Environment: the reviewer gets the user's environment minus
quest agent's stripped variables (CLAUDE*exceptCLAUDE_CONFIG_DIR,ANTHROPIC*,CODEX_*exceptCODEX_HOME,OPENAI_API_KEY,OP_*). So it uses each CLI's stored login, and an env-var-only API key login is reported as logged out.Model tiers:
--deepconflicts with--model(exit 2). The stronger tier also runs at medium effort, since only the model was specified. The round reports the model and effort quest passed rather than what the CLI reports. Claude's/code-reviewlevel follows the selected effort, as its level overrides--effort. Usage lines are omitted for turns that report all zeros (Codex's review turn).Guide: the local review runs before marking ready and before each push to a ready PR; pushes to a draft that still waits on decisions need none.
quest status'sreview-roundsis left as is but is no longer referenced by the guide.Codex user config: skipped (
--ignore-user-config) for the P1 above, mirroring decision 2's choice for Claude; a custommodel_provideror profile in it no longer applies to the reviewer.Suggested follow-ups
peer-reviewskill once this lands (per the Plan).MODELSinsrc/cli/review.ts; a recurring quest could check the pinned ids against the newest models each milestone.quest init(or an upgrade path) remove stubs of skills that no longer exist, such asquest-ready.quest status'sreview-roundscounts GitHub review rounds, which the guide no longer uses: drop it, or report local rounds from the## Reviewrecord.## Reviewrecord is written by the author's session, so kix's gate should decide how far it trusts a self-reported record.🤖 Generated with Claude Code
(written by Claude Opus 5.5)