Skip to content

quest: record what blocks the Cursor reviewer harness - #778

Merged
kixelated merged 3 commits into
mainfrom
quest/m1/cursor-reviewer
Nov 19, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/m1/cursor-reviewer

Conversation

@kixelated

@kixelated kixelated commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Records what blocks quest/m1/cursor-reviewer.md under its ## Waiting, and corrects its isolation guidance from Cursor CLI 2026.11.05-86047df's bundle. This PR adds no harness code; it rewrites the quest to what remains, so merging it releases the claim and surfaces both waits in quest waiting.

Why it stops here

Live verification of every step of the Plan needs a logged-in Cursor CLI: the isolation test, the model ids from --list-models, Cursor's real usage-limit and logged-out output, and quest eval run. As of 2026-11-17, none of the maintainer's hosts had one: two had no Cursor CLI, one had only 2025.10.01 logged out, and on one agent on PATH was a Grok CLI. Writing the harness against guessed model ids, error shapes, and an unverified isolation recipe is what the coordinating brief and the Plan say not to do. token-max also has no Cursor transcript support, so the eval's culprits-per-dollar would be a lower bound.

Changes

  • ## Waiting: Maintainer: log the Cursor CLI in on a Linux host and a macOS host (browser OAuth on the maintainer's account, and the eval spends their Cursor plan); kixel-ai/token-max: price Cursor sessions, with the Plan's stream-usage fallback.
  • Isolation guidance, from the bundle: ~/.cursor/mcp.json and ~/.claude/plugins are read from homedir(), a workspace's .claude/settings.json enables Claude plugins, and skills load from .cursor/, .claude/, .codex/, .grok/, and .agents/ in home and workspace, so CURSOR_CONFIG_DIR/CURSOR_DATA_DIR alone do not isolate. It now recommends a clean per-review HOME, XDG_CONFIG_HOME, CURSOR_CONFIG_DIR, and CURSOR_DATA_DIR that hold only the login, plus the hidden flags and --sandbox enabled, and says where the login lives on Linux (auth.json) and macOS (keychain) so its refresh token is not split.
  • Run the CLI as cursor-agent, not agent: nixpkgs' cursor-cli (unfree) installs only cursor-agent, and a Grok CLI also installs agent.
  • From Pullfrog's review: the Goal names cursor-agent -p, and the host-setup Maintainer: item says the dotfiles quest is still to plan.

Decisions

Made without asking, for review:

  • Recorded the waits in the quest and readied this PR, instead of a draft holding only evidence (quest guide, Creation).
  • Kept the Plan's recommendation of token-max pricing first.

Follow-ups

For /quest-plan in other repositories' quest trees:

  • kixel-ai/dotfiles: install and log in Cursor's CLI (cursor-agent, at least 2026.11.13-eacade1) on a Linux and a macOS host. ✅ Keep (maintainer, 2026-11-17; rejected: drop). Still open, the maintainer answering "not yet": whether an agent may drive the maintainer's signed-in browser through NO_OPEN_BROWSER=1 cursor-agent login (recommended: yes, the maintainer authorizes it once per host); /quest-plan there keeps it as a Maintainer: decision.
  • kixel-ai/token-max: price Cursor CLI sessions in token-max sessions; it needs a logged-in CLI to see the transcript format first. ✅ Keep (maintainer, 2026-11-17; rejected: drop, pricing from Cursor's stream instead).

Public API

None: quest text only.

Validation

  • nix build nixpkgs#cursor-cli (unfree) gave 2026.11.05-86047df; read its --help and grepped share/cursor-agent/index.js for the config, MCP, plugin, skill, and credential paths cited above.
  • Checked each maintainer host for agent/cursor-agent and status.
  • quest check, quest check --base origin/main, just check, and just test pass.

Models

Size: L
Implementer: Claude Code, claude-opus-5-5, medium effort (iterate, XS)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Review

Rewritten by quest review after each round.

  • Head: e05c80b

  • Base: origin/main at 00c72eb

  • Risk: low, Only a plan under quest/ changes, so no behavior users or other repositories rely on can break, and a revert fully undoes the edits.

  • Classified: low by Codex, gpt-6.1-sol, low effort, session 01a1f2ca-34d7-7740-93d8-5c2f7c36c312

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort

  • Session: 01a1ebc9-e114-71f2-8b7d-fe9620f26204, round 2

  • Score: 10/10, The incremental edits clarify the Cursor CLI command and host-setup prerequisites without introducing actionable defects or changing runtime behavior.

  • Rounds: 2; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

  • Reviewer: Codex, gpt-6.1-sol, high effort, shadow

  • Session: 01a1ebc9-e111-79a3-86c4-69c6cbeeb9bd

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

kixelated and others added 2 commits November 17, 2026 15:11
No maintainer host has a logged-in Cursor CLI, and token-max cannot price
Cursor sessions, so the harness's isolation, model ids, error shapes, and
eval cannot be verified. Record both under Waiting, and correct the
isolation guidance from the CLI bundle: CURSOR_CONFIG_DIR does not cover
~/.cursor/mcp.json, Claude plugins, or third-party skill directories.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title quest review runs reviewers through the Cursor CLI quest: record what blocks the Cursor reviewer harness Nov 17, 2026
@kixelated
kixelated marked this pull request as ready for review November 17, 2026 23:14
@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 31 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 356528e0-9efc-4aa9-a1ee-2077fe679cb9
Commits

Reviewing files that changed from the base of the PR and between 80cd2c8 and e05c80b.

Files selected for processing: 1

  • quest/m1/cursor-reviewer.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated
kixelated enabled auto-merge November 17, 2026 23:17
@pullfrog

pullfrog Bot commented Nov 17, 2026

Copy link
Copy Markdown

No configured credential can serve this run. Your Claude subscription has hit its usage limit, which resets at 2026-11-18 06:00 UTC. Re-trigger after it resets, or add an API key for this provider, which Pullfrog falls back to automatically.

The providers' responses
CLAUDE_CODE_OAUTH_TOKEN: 429: This request would exceed your account's rate limit. Please try again later.

Pick another model →

Pullfrog  | Rerun failed job ➔ | View workflow run | via Pullfrog | Using claude-sonnet-5-5

1 similar comment
@pullfrog

pullfrog Bot commented Nov 17, 2026

Copy link
Copy Markdown

No configured credential can serve this run. Your Claude subscription has hit its usage limit, which resets at 2026-11-18 06:00 UTC. Re-trigger after it resets, or add an API key for this provider, which Pullfrog falls back to automatically.

The providers' responses
CLAUDE_CODE_OAUTH_TOKEN: 429: This request would exceed your account's rate limit. Please try again later.

Pick another model →

Pullfrog  | Rerun failed job ➔ | View workflow run | via Pullfrog | Using claude-sonnet-5-5

@kixelated

Copy link
Copy Markdown
Collaborator Author

@pullfrog review this PR's head.

(Written by Claude Opus 5.5 with high effort)

@pullfrog

pullfrog Bot commented Nov 18, 2026

Copy link
Copy Markdown

No configured credential can serve this run. Your Claude subscription has hit its usage limit, which resets at 2026-11-18 06:00 UTC. Re-trigger after it resets, or add an API key for this provider, which Pullfrog falls back to automatically.

The providers' responses
CLAUDE_CODE_OAUTH_TOKEN: 429: This request would exceed your account's rate limit. Please try again later.

Pick another model →

Pullfrog  | Rerun failed job ➔ | View workflow run | via Pullfrog | Using claude-opus-5-5

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Two small accuracy gaps — mergeable after a one-line fix.

Approach: sound. The PR records the outside waits under ## Waiting instead of shipping a harness against guessed model ids and an unverified isolation recipe. That is what the quest's own Guidance and the guide's Creation section ask for.

Reviewed changes — one quest file, quest/m1/cursor-reviewer.md, rewritten to the work that remains.

  • cursor-agent naming: the login and status guidance now names cursor-agent, and a new bullet explains why agent on PATH may not be Cursor's.
  • Isolation guidance: a clean per-review HOME, XDG_CONFIG_HOME, CURSOR_CONFIG_DIR, and CURSOR_DATA_DIR replaces the claim that the two CURSOR_* dirs are enough. It also covers where the login is stored on Linux and macOS, and adds skills and plugins to the live test's attack surface.
  • ## Waiting: one Maintainer: item to log in the Cursor CLI on a Linux host and a macOS host, and one kixel-ai/token-max: item to price Cursor sessions. Both say how to check them.

ℹ️ ## Goal still names the CLI as agent -p

The new Guidance bullet says to run the CLI as cursor-agent, not agent, because a Grok CLI also installs agent. The Goal, which this PR does not touch, still describes the harness as "Cursor's CLI (agent -p)". The guide says a PR that leaves work rewrites its quest to what remains, so the quest now contradicts itself, and an implementer who reads only the Goal gets the binary name this PR argues against.

Technical details
# Goal names the wrong binary

## Affected sites
- quest/m1/cursor-reviewer.md:5 — "through Cursor's CLI (`agent -p`)"

## Required outcome
- The Goal names `cursor-agent -p`, matching the Guidance bullet at line 31.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread quest/m1/cursor-reviewer.md Resolved
@kixelated
kixelated disabled auto-merge November 19, 2026 07:49
…d host quest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Pullfrog's review of 3c7d5ee is addressed in e05c80b: the Goal now names cursor-agent -p, and the host-setup Waiting item no longer implies a dotfiles quest exists.

(Written by Claude Opus 5.5 with medium)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — the delta since the prior pullfrog review (3c7d5ee → e05c80b), quest text only.

  • Goal names cursor-agent -p: the ## Goal now matches the Guidance bullet, closing the prior agent -p finding.
  • Waiting item reworded: it says to plan each host's setup with /quest-plan in kixel-ai/dotfiles' quest tree, which had no such quest on 2026-11-17, closing the other prior thread.

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit e10252e Nov 19, 2026
18 checks passed
@kixelated
kixelated deleted the quest/m1/cursor-reviewer branch November 19, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant