Repository navigation
test: run the m1 tests audit - #179
Conversation
Quest-Claim: c24e65f2dac09dc1
…pans as requests are The T3 schema fixture mis-copied two indexes from the pinned c4ce7b0 and left out 13 more over the columns it keeps, so the query-plan test ran against a schema no real database has. It now holds every index that revision creates over those columns, verbatim. prefix_spans priced a Codex write whose lifetime split names one lifetime as part write and part ordinary input, and priced 12h writes outside the Responses API, where price_components leaves both unknown. It now agrees with price_components. New tests cover it and the premium-tier multipliers on gpt-5.4 and gpt-5.5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…otals and SSH protocol guards These paths had no test: a write failing after another file was already written, the one-by-one walk through turned-off MCP servers, a Codex total before the window starting the first delta inside it, and the coordinator rejecting a destination that changes its window or result, sends more after it, or fails after sending it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex writes its requested service tier in thread_settings_applied, never in turn_context, and starts each turn with task_started; none of 6,097 turn_context records on a heavy user's host carries a tier, and upstream's TurnContextItem has no such field. The fixture now records the turn's start, its context without a tier, and its applied settings with the model, provider, effort and the tier label Codex writes, so the tests read Codex's real paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
✅ No new issues found.
Approach: sound. The only behavior change makes prefix_spans follow the same rules as price_components: a partial lifetime split stays unknown, and so does a 12h write outside the Responses API. The rest brings fixtures in line with what they copy and adds tests for the gaps that would cost the most if they broke.
Reviewed changes
I reviewed all 13 files. I also ran the new and changed lib tests locally (codex_prefix_spans_*, premium_tiers_*, a_write_that_fails_after_*, turned_off_mcp_*, legacy_totals_*, nested_codex_settings_*, codex_response_usage_*) and they all pass. I did not run tests/ssh.rs.
prefix_spansCodex arm: it now returnsNonein exactly the cases whereprice_components(src/prices.rs~L1424–1452) marks the write split or the 12h lifetime unknown. An unsplit count still takes the default 30m lifetime.- T3 schema fixture: it gains
ordinal INTEGER NOT NULLand the full set of pinned indexes, including the correctedUNIQUE (provider_thread_id, ordinal). Everyprovider_turnsINSERT int3.rsandguards.rsnow supplies an ordinal, and the duplicate-row test usesordinal+1so the unique index still accepts it. - Codex fixture: the tier moves out of
turn_contextintothread_settings_applied("default"), and atask_startedrecord is added. All three consumers are re-indexed: thereader.rsinsert moves to index 4,tests/ssh.rsusesfixture[4]for usage, andfast.rsnow expects"default". Both labels price as standard (prices.rs:1175). - Doctor write tests: the blocker path
.config.toml.token-max-{pid}matchesTarget::write's staging name. The early?increate_copycorrectly leaves the blocker in place, and the uncommittedStagedbackup is dropped. - SSH protocol guards and pre-window totals: these are new coverage for the window/kind mismatch, trailing output, failure after the result, and a malformed earlier cumulative total.
- Quest files: the m1 stub becomes the m2 stub with a baseline and a measurement method, and adds the
ssh-from-windowsfollow-up with its open decision.
claude-opus-5-5 | 𝕏
|
Maintainer decision, in the merge pick round for this PR:
(Written by Claude Opus 5.5) |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Reviewed the one commit since the prior pullfrog review (docs only).
- Windows SSH decision recorded:
quest/m1/ssh-from-windows.mdnow states the maintainer's pick (port the harness) and drops the resolved## Waitingsection.
claude-sonnet-5-5 | 𝕏

Runs the m1
testsaudit (quest/m1/audit-tests.md) and plans its m2 run.Summary
prices::prefix_spanspriced a Codex write whose lifetime split names only one lifetime (sayttl_30m_tokenswithoutttl_12h_tokens) as part write, part ordinary input. It also priced 12h writes outside the Responses API.price_componentsleaves both of those requests unpriced, so the context check's figures for them were invented.prefix_spansnow agrees withprice_components.test/fixtures/t3-schema.sqlclaimed T3c4ce7b0's indexes but mis-copied two (provider_turns_thread_ordinal_idxisUNIQUE (provider_thread_id, ordinal), andturn_items_active_run_idxcovers(thread_id, run_id)). It also left out 13 more indexes over the columns it keeps. It now holds all 24 verbatim from the pinned migrations, plus theordinalcolumn the unique index needs. Its header states the copy rule. The query-plan test's scan counts hold against the real index set.test/fixtures/codex.jsonlhad the requested tier inturn_context. Codex never writes it there: none of 6,097turn_contextrecords on this host has one, and upstream'sTurnContextItemhas no such field. The tier lives inthread_settings_applied, and every turn starts withtask_started, neither of which the fixture had. It now records both, using the tier label Codex actually writes (default). The reader'stask_startedandthread_settingseffort paths now run in tests.quest/m1/audit-tests.mdbecomesquest/m2/audit-tests.md, carrying the baseline, how to measure it, the mirrors to recheck and what to check first.quest/m2/README.mdmatches the one the other m1 audit PRs add.Findings by kind
prices.rs, settings writes indoctor.rs, Codex cumulative totals inreader.rs, and untrusted SSH streams inssh.rs. All are covered above. Left for the next run, as its Plan says:report.rsandidle.rsrendering lines, and two guards no test can reach without new seams.#[ignore]and no env-gated suites. Everycfggate matches a CI platform, and every test file compiles into a target CI runs. One platform gap, a follow-up below: every--sshtest iscfg(unix), so a Windows coordinator has none.fixPR onmain, and every PR listing a## Caused by, added or changed tests, except kixel-ai/token-max#45, which only edits a workflow.nowas a value. The SSH and pty integration tests use deadlines only as hang guards around real child processes.prices.jsonmatches models.dev's live shape, includingtiersbesidecontext_over_200k.claude.jsonl's one key absent locally,compactMetadata.precomputed, is a real (feature-gated) Claude Code 2.1.325 field.Choices made without asking
prefix_spansreturns None, asprice_componentsdoes, rather than guessing a lifetime for a partial split.default). One fast test now expects that label instead ofstandard; both price as standard.config.tomlleaves the file starting with a blank line (TOML decor from the next table). It is cosmetic and valid TOML, so I left it alone.Caused by
prefix_spans' Codex arm priced a write with a partial lifetime split, and 12h writes off the Responses API, whichprice_componentsleaves unknown (found by check)Public API
None.
prefix_spansreturns None in two more cases; no signature or wire format changes.Validation
just checkpasses: fmt, clippy, all suites andquest check.prefix_spanstest fails on the old code: it prices 10,000 of 30,000 written tokens as writes and the rest as ordinary input, where the request total is unknown.cargo-llvm-cov0.9.1, folding in the profiles of the binary the integration tests spawn (theyenv_clear()it). The next stub's Plan records the method.statev2.sqliteschema (schema only) and the pinned migrations. I checked the transcript fixtures' key paths and types against this host's 2,703 Claude and 4,144 Codex transcripts, comparing key names and types only, never values. I checkedprices.jsonagainst models.dev's liveapi.json.Follow-ups
--sshrun from Windows. Settled in the merge pick (comment): ✅ keep it (rejected: drop it); ✅ port the SSH test harness to the Windows CI runner using Git for Windows'sh, size M (rejected: refuse--sshon Windows with a WSL hint). No open decisions.None of the other m1 audit PRs (kixel-ai/token-max#174, #175, #176, #177, #178) adds an overlapping quest.
Models
Size: M
Implementer: Claude Code, claude-opus-5-5, unknown effort (arm B)
Implementer: Claude Code, claude-opus-5-5, low effort (iterate, XS, arm B)
🤖 Generated with Claude Code
(Written by Claude Opus 5.5)
Review
Rewritten by
quest reviewafter each round.Head: e01769a
Base: origin/main at 309b526
Risk: high,
The change affects billing calculations, so incorrect whole-request pricing could misstate users’ costs; reverting the code would not correct cost decisions already made.Classified: medium by default
Reviewer: Codex, gpt-6.1-sol, high effort
Session:
01a1fa51-d45f-7a83-9285-816765e32f28, round 1Score: 9/10,
Static review found no actionable regressions in the pricing fix, fixtures, or added tests; builds and tests were not run, as instructed.Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0
Findings: none
Reviewer: Codex, gpt-6.1-sol, medium effort, shadow
Session:
01a1fa51-d45f-7d91-9751-f953d8b4cabc, round 1Score: 9/10,
No actionable regressions were found: the pricing correction matches whole-request billing logic and the added coverage is coherent, but builds and tests were not run as instructed.Raised the risk to high:
The change affects billing calculations, so incorrect whole-request pricing could misstate users’ costs; reverting the code would not correct cost decisions already made.Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0
Findings: none
Commit 7a4d54a, after the reviewed head e01769a, only records the merge pick's follow-up answers in
quest/m1/ssh-from-windows.md, so no new round ran.The
duplicatelinesquest statusprints forquest/m2/README.mdstay on purpose: every m1 audit PR adds the same m2 milestone README for its next stub, with identical content, so the merges resolve cleanly.Summary by CodeRabbit