Repository navigation
feat(usage)!: price sessions through token-max, across SSH hosts - #704
Conversation
Quest-Claim: 54484a76d642b96e
`quest usage` reads each session's harness, model, effort, branches, repositories, and cost from `token-max sessions`, which treats each sub-agent as a session of its own, and drops ccusage and quest's own transcript readers. `--ssh HOST` (repeatable) adds other hosts' sessions, matched through the repositories resolved on their own host; a failing host fails the report. A session found on several hosts counts once, at its costliest copy, and the report says how many copies it folded. Sessions whose cost is a lower bound are listed. `fixedBy` counts only merged PRs, and only a `## Caused by` bullet that opens with the culprit's link. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ence/usage-token-max
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l name repositories Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
There was a problem hiding this comment.
ℹ️ No critical issues — one minor inconsistency inline.
Reviewed changes
This review covers the full PR at 133eeb6, which moves quest usage from ccusage and quest's own transcript readers to token-max sessions.
- token-max export:
runTokenMaxandparseSessionsread token-max's JSONL. A session'scomplete: falsenow applies to each of its models, so every cost it feeds readsor more. - Cross-host fold:
foldkeeps the costliest copy of each[harness, id]and leaves sessions without an ID alone. Attribution still draws repository evidence from every exported copy. - Attribution: a session matches a PR only through the repositories token-max resolved on the session's own host. Failing that, it uses those resolved for another session in the same directory on that host. Each session row now carries its
host, and effort moves to each model entry. fixedBy: only merged PRs count. Only a## Caused bybullet that opens with- [owner/repo#N](…/pull/N) -counts, and only when the link text and URL name the same PR.githubRepository: accepts an scp-like remote with no user, which is how token-max exports SSH remotes.- Dev shell: drops ccusage and pins token-max at
83ab616as a flake input whosequestinput follows the root.
I checked token-max's src/sessions.rs at the pinned revision, since the fold's correctness depends on it:
- A Claude sub-agent's
sessionis its ownagentId, with the parent ID inparent, sofolddoes not merge sub-agents into their coordinator. - A Codex
sessionis the thread ID, which matches whatquest reviewrecords for the reviewer role.
claude-opus-5-5 | 𝕏
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
This review covers the new commit 4f20790 since the prior pullfrog review at 133eeb6.
- Narrowed
foldedcount:usagenow keeps only the dropped copies whose harness and ID match a session inreported, so a--pror--questreport no longer counts folded copies of sessions it doesn't show. This is the same scope asincomplete, and sessions without an ID never fold, so they can't be miscounted. - Test update:
test/usage.test.tsnow expects{ copies: 0, hosts: [] }for the narrowed--pr 10report and{ copies: 1, hosts: ["box"] }for the unnarrowed one. This addresses the earlier inline thread, which is already resolved.
claude-sonnet-5-5 | 𝕏
|
quest land watch: the merge queue removed this pull request at 4f20790 because it conflicts with The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs. (written by quest land watch) |
|
codex, gpt-6.1-sol, medium effort land fixer note:
(written by codex, gpt-6.1-sol, medium effort) |
|
quest land watch: armed auto-merge on this pull request at 743fc0d, so it joins the back of the merge queue once its required checks pass, with the land fixer's fix ( (written by quest land watch) |
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
This review covers merge commit 743fc0d since the prior pullfrog review at 4f20790.
- Reconciled imports with base module moves:
src/cli/usage.tsnow importsrecordedfrom./models-record, andrepositoryPath,GitHub, andlinkfrom../github/client.src/cli/main.tsimportspullRequestStatusfrom../github/status. No behavior changed.tsc --noEmitis clean andtest/usage.test.tspasses 11/11.
claude-opus-5-5 | 𝕏
|
quest land watch: the merge queue removed this pull request at 743fc0d because its The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.
…b7zp09bssia-git-2.56.0/bin/git config --local --unset includeif.gitdir:/github/workspace/.git.path \/github\/runner_temp\/git\-credentials\-b1e4f9a9\-8dfa\-435e\-8493\-12effcad638b\.config
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --get-all includeif.gitdir:/github/workspace/.git/worktrees/*.path
/github/runner_temp/git-credentials-b1e4f9a9-8dfa-435e-8493-12effcad638b.config
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --unset includeif.gitdir:/github/workspace/.git/worktrees/*.path \/github\/runner_temp\/git\-credentials\-b1e4f9a9\-8dfa\-435e\-8493\-12effcad638b\.config
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
Removing credentials config '/var/cache/github-runner-work/_temp/git-credentials-b1e4f9a9-8dfa-435e-8493-12effcad638b.config'
##[endgroup]
##[group]Run # Build concurrency and group size are both one in main's ruleset.
�[36;1m# Build concurrency and group size are both one in main's ruleset.�[0m
�[36;1m[[ "$QUEUE_REF" =~ ^refs/heads/gh-readonly-queue/main/pr-([0-9]+)- ]] || { echo "::error::unexpected merge group ref" >&2; exit 1; }�[0m
�[36;1mhead_ref=$(gh api "repos/$REPO/pulls/${BASH_REMATCH[1]}" --jq .head.ref)�[0m
�[36;1m# The base's skills would reject a PR that adds, removes, or changes a skill's stub. The PR's�[0m
�[36;1m# own binary checks the stubs against the PR's skills in `just check` below, in this same job.�[0m
�[36;1m./quest-base/bin/quest --root "$GITHUB_WORKSPACE" check --no-stubs --base "$BASE" --branch "$head_ref"�[0m
shell: /nix/store/lfm00fnq66xfgbn099vqckabyl5jllz6-bash-interactive-5.3p20/bin/bash -e {0}
env:
BASE: fd385d028fa7f4c6420063141ba9b05c8f1c2d41
GH_TOKEN: ***
REPO: moq-dev/quest
QUEUE_REF: refs/heads/gh-readonly-queue/main/pr-704-fd385d028fa7f4c6420063141ba9b05c8f1c2d41
##[endgroup]
quest: quest/m0/benchmark/records.md:52: link does not resolve: /quest/m0/routing-evidence/usage-token-max.md
##[error]Process completed with exit code 1.
Post job cleanup.
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git version
git version 2.56.0
Temporarily overriding HOME='/var/cache/github-runner-work/_temp/3c751fa3-8fe3-4bce-8fca-71f89afdef96' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --global --add safe.directory /var/cache/github-runner-work/quest/quest
Removing SSH command configuration
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
Post job cleanup.
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git version
git version 2.56.0
Temporarily overriding HOME='/var/cache/github-runner-work/_temp/6ca9c31d-66ca-415c-b04b-f1a7994467b4' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --global --add safe.directory /var/cache/github-runner-work/quest/quest
Removing SSH command configuration
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
A job completed hook has been configured by the self-hosted runner administrator
##[group]Run '/nix/store/3wq9f25mz2ksqrx6zs5nq522386djzw1-ci-cache-gc.sh/bin/ci-cache-gc.sh'
shell: /nix/store/lfm00fnq66xfgbn099vqckabyl5jllz6-bash-interactive-5.3p20/bin/bash --noprofile --norc -e -o pipefail {0}
##[endgroup]
evicted 0 objects and 0 action results (0 B logical); 49.6 GiB logical remain
Cleaning up orphan processes
Current runner version: '2.338.2'
##[group]Runner Image Provisioner
Hosted Compute Agent
Version: 20261103.629
Commit: 15dbd9ccb5dbf34fae002b6ed1a2670f52df37bc
Build Date: 2026-11-03T17:26:59Z
Worker ID: {0957e4e5-df30-45f0-92c0-75f25cc43524}
Region: westus3
Cloud: Azure
##[endgroup]
##[group]Operating System
Ubuntu
26.04.1
LTS
##[endgroup]
##[group]Runner Image
Image: ubuntu-26.04
Version: 20261111.196.1
Included Software: https://github.com/actions/runner-images/blob/ubuntu26/20261111.196/images/ubuntu/Ubuntu2604-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu26%2F20261111.196
##[endgroup]
##[group]GITHUB_TOKEN Permissions
Contents: read
Metadata: read
PullRequests: read
##[endgroup]
Secret source: Actions
Cache mode: write
Prepare workflow directory
Prepare all required actions
Complete job name: Queue
##[group]Run [[ "$RESULT" == success ]] || { echo "::error::the queue build ended $RESULT" >&2; exit 1; }
�[36;1m[[ "$RESULT" == success ]] || { echo "::error::the queue build ended $RESULT" >&2; exit 1; }�[0m
shell: /usr/bin/bash -e {0}
env:
RESULT: failure
##[endgroup]
##[error]the queue build ended failure
##[error]Process completed with exit code 1.
Cleaning up orphan processes
(written by quest land watch) |
|
codex, gpt-6.1-sol, medium effort land fixer note:
(written by codex, gpt-6.1-sol, medium effort) |
|
quest land watch: armed auto-merge on this pull request at 1a70998, so it joins the back of the merge queue once its required checks pass, with the land fixer's fix ( (written by quest land watch) |
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
This review covers the changes since the prior pullfrog review at 743fc0d: a merge of main and one docs fix.
- Retargeted a deleted-quest link:
quest/m0/benchmark/records.mdnow links to this PR instead ofquest/m0/routing-evidence/usage-token-max.md, which the PR deletes. Nothing else in the tree still links to that file.
claude-opus-5-5 | 𝕏
|
quest land watch: the merge queue removed this pull request at 1a70998 because it conflicts with The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs. (written by quest land watch) |
|
quest land watch: armed auto-merge on this pull request at c5c1c77, so it joins the back of the merge queue once its required checks pass, with the land fixer's fix ( (written by quest land watch) |
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
This review covers merge commit c5c1c77 since the prior pullfrog review at 1a70998. Apart from what main brought in, the only change is to the tests.
- Isolated fixture paths from the host: the session fixtures in
test/usage.test.tsno longer use/work,/other, and their subdirectories as working directories. They use directories fromremoved(), which creates each one withtempDir()and then deletes it.src/cli/usage.tsis unchanged and still usescwdonly as a lookup key alongside the host, never reading the filesystem there. All 11 tests intest/usage.test.tspass.
claude-opus-5-5 | 𝕏

Finishes quest/m0/routing-evidence/usage-token-max.md and deletes it.
Summary
quest usagenow gets every session fromtoken-max sessionsinstead of ccusage plus quest's own transcript readers. Each session's harness, model, effort, branches, working directory, repositories, and cost come from token-max. Claude sub-agents and Codex review or spawned threads are separate sessions, so a quest a/quest-spawnor/quest-runsub-agent worked on carries its own cost. That was ~$1,600 of Claude sub-agent spend on this host since 2026-11-12, previously folded into coordinators.--ssh HOST(repeatable) passes through totoken-max sessions --ssh … --include-local. A session matches a PR only through the repositories token-max resolved on the session's own host, or (once its checkout is deleted) those resolved for another session in the same directory on the same host. Nothing is read from a checkout on the host running the report.folded: { copies, hosts }, and the text saysFolded N copies … left out from <hosts>.Lower boundswith its unreadable-line count, and every cost it feeds readsor more. A malformed line no longer stops the report, since token-max skips and counts it.fixedBycounts only merged fix PRs, and only a## Caused bybullet that opens with- [owner/repo#N](…/pull/N) - why. Context links, "No single PR"/"None" lines, "(fixed by …)" asides, notes after the bullets, and closed duplicates no longer count. The tests use the shapes from test(net): a refused replacement keeps a withdrawn incumbent open moq#8471, #8426, #8483, #8427, #8371, and #8493.githubRepositoryaccepts an scp-like remote without a user (github.com:owner/repo.git), which is how token-max exportsgit@github.com:…after removing user info. Without this, every session in an SSH-cloned checkout went unattributed.ccusageand pins token-max as a flake input at83ab616, the first revision whosetoken-max sessions --helpdocuments both--sshandrepositories. That clears the quest's Waiting item (kixel-ai/token-max#92 merged). Itsquestinput follows this flake (follows = ""), so the lock never chains older quest pins.eachLine,parseLine, andclaudeLogsmoved intosrc/cli/messages.ts, their only remaining user, and are no longer exported.quest/m0/messages-robustness.md; removed the deleted quest'sRequiredlinks fromwriter-ab.mdandcharts.md(both now unblocked), unlinked it in the epic README, and updatedmistake-audit.md's note onfixedBy.Public API
Breaking (
!) forquest usage --jsonconsumers:pulls[].sessions[]loseeffortand gainhost; eachmodels[]entry carries its owneffort, a nullablemodel, andcomplete.spend[],unattributed) gainscomplete.foldedandincomplete.pricednow means token-max ran.fixedByis narrower, as described above.To migrate: install token-max (the dev shell provides it), and read effort from each model entry. New flag:
--ssh <host>.Validation
just checkandjust testpass on the head after merging main (1342 tests).quest usage --since 2026-11-11with the pinned token-max: exits 0, reports 86 PRs, and attributes 80 sessions. Before the scp-remote fix it attributed 72.--ssh nixos(no token-max installed there) exits 1 withnixos: token-max is not installed on PATH, ….--ssh boxwith a stand-insshin.tmp/that runs the remote command on this host. Every session then exists twice. The report folded 652 copies "left out from box", and every PR's lines matched a local-only run apart from live drift. The real SSH transport is token-max's and is covered by its own tests.claude-fable-5-2entries) or live, unfinished transcripts.Choices made without asking
--ssh, a missing token-max is an error.cargo install --locked --git https://github.com/kixel-ai/token-max, since token-max has no release yet.start), not the transcript's first record, since token-max does not export the creation time it tracks. A session that began before--sinceon a PR opened before--sincenow reads as iterating instead of implementing; README and--helpsay so.Decisions
Decided 2026-11-13 (user, /quest-run):
quest messages? ✅ One robustness quest covering both causes (rejected: two quests; only the Codex one).quest messagesrobustness quest: ✅ keep (rejected: drop).Caused by
causes()took every link in## Caused byand counted unmerged fixes towardfixedByFollow-ups
quest messages reads transcripts it fails on today, added in this PR: it skips and counts malformed Claude Code lines, and reads Codex 0.162 TUI turns that carry
content_item_kindswithout provenance (salvaged from the closed moq-dev/quest#696).To plan in kixel-ai/token-max's quest tree (user picked):
token-max sessionsexports each session's creation time (itsFacts.created, already tracked), soquest usagecan classify a session that crosses--sinceas implementing. Not linked here, since it is another repository's quest.Models
🤖 Generated with Claude Code
(Written by Claude Opus 5.5 with medium effort)
Review
mechanical fix by codex, gpt-6.1-sol, medium effort:
Merged base and re·solved test/usage.test.ts conflicts, preserving exported session fixtures and base temporary-path isolation; ran nix develop --command just check and nix develop --command just test (1217 passed, 5 skipped; build, compile, installer and lifecycle checks passed).(head c5c1c77)mechanical fix by codex, gpt-6.1-sol, medium effort:
Merged base and retargeted a stale link to the completed PR; ran nix develop --command just install, nix develop --command just check, nix develop --command just test (gateway environment failure), nix develop --command env -u OPENAI_BASE_URL bun run vitest run test/review.test.ts -t "Codex reaches a gateway", nix develop --command env -u OPENAI_BASE_URL just test, nix flake check, nix build /tmp/704-base-tree#quest --out-link /tmp/704-base-quest, and /tmp/704-base-quest/bin/quest --root /work c(head 1a70998)mechanical fix by codex, gpt-6.1-sol, medium effort:
Merged base and re·solved import conflicts following base module moves, preserving token-max usage changes; ran nix develop --command just install, nix develop --command just check, nix develop --command just test, nix develop --command env -u OPENAI_BASE_URL bun run vitest run test/review.test.ts -t "Codex reaches a gateway", and nix develop --command env -u OPENAI_BASE_URL just test (passed).(head 743fc0d)Rewritten by
quest reviewafter each round.Head: 4f20790
Base: origin/main at b79569f
Reviewer: Codex, gpt-6.1-sol, high effort
Session:
01a1d70a-5341-72b3-a927-8d15048d4b87, round 2Score: 9/10,
The incremental change correctly limits folded-copy counts to reported sessions and preserves the earlier fixes; static review found no actionable defects, but builds and tests were not run as instructed.Rounds: 3; findings 3 (high 1, medium 2): fixed 2, refuted 1, open 0
Findings: none
Reviewer: Codex, gpt-6.1-sol, medium effort, shadow
Session:
01a1d70a-5341-72d1-b8f6-ffae57045dc0Rounds: 2; findings 2 (medium 2): fixed 1, refuted 1, open 0