Repository navigation
Implement local TypeScript replay and Claude TTL recommendations - #3
Conversation
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Quest-Claim: b0f3be4d4f7c4abd
Replace the Rust scaffold with the local streaming reader, public price adapter, shared report, and aggregate TTL replay. Preserve unknown billing and attribution, shared-history provenance, frozen snapshots, and strict safe failures. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
|
Design change: The maintainer selected option 1 for API attempts without linked token usage: estimate where recorded evidence supports it, show sensitivity, and leave unsupported cases and affected comparisons unpriced. Do not silently exclude these attempts or treat missing usage as zero billing. This supersedes the unresolved policy choice from round 2. Implement defensible estimates with explicit provenance and uncertainty, retain unknowns where estimation is unsupported, then run the final independent review and checks. Recommend the estimated aggregate net winner without an all-scenarios veto; unknown charges that demonstrably cancel between scenarios need not block their known difference. The existing strict malformed-complete-record failure policy remains in force. No malformed record is skipped and no complete real-host audit total is claimed from the partial samples. (Written by GPT-6.1 Sol with xhigh) |
Apply the maintainer policy recorded in PR 3 comment 6709040500; preserve original usage and unsupported comparisons as unknown. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Use measured neighboring input geometry within the minimum cache lifetime, require matching client provenance, and retain conditional sensitivity for estimated cache seeds outside the report window. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Exclude the following request’s own new writes from estimated input, preserve ordinary input as an explicit proxy, and reject inconsistent warm-prefix evidence. Restrict boundary inspection to Claude compaction events. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Keep growing-context attempts unsupported when no observed cache read supports their geometry, instead of assigning a conditional zero-cache estimate. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Preserve the remaining unsupported pre-window cache-chain finding, concrete reproduction, and recommended evidence-state redesign without changing implementation after the mandatory checkpoint. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
|
Design change: At the third-round checkpoint, the maintainer selected B1: rethink the cache replay chain as explicit known, estimated, and unknown states with predecessor provenance. Carry unsupported attempts and missing-speed ambiguity across possible cohorts and the selected window boundary instead of dropping or silently clearing their evidence. Reject the targeted flag-only patch. The evidence-backed estimation policy and aggregate estimated net winner remain settled; unsupported affected comparisons stay unpriced. The maintainer also authorized configuring token-max's repository auto-merge like moq and continuing quest-complete/quest-spawn until done. This is not permission to merge this draft before the remaining correctness fix, checks, and independent review are complete. (Written by GPT-6.1 Sol with xhigh) |
Keep replay’s current remaining work and audit limitations while integrating the squash-merged plan. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Record the maintainer's B1 design, preserve ambiguous pre-window attempts across compatible cohorts, and price only established cache geometry. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Retain unbilled pre-window requests and compaction boundaries, keep invalid unselected usage unknown, and share all part evidence before selecting comparison charges. Separate prefix evidence from cohort completeness and verify reader/CLI behavior. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Replay all boundary claims before emitting selected charges, apply observed-TTL guards only to measured parts, and keep known subagents outside main. Retain the scan billing bound for library checks. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Implement the local replay core and Claude main/outside-main cache-TTL comparison in Node-compatible TypeScript. Replace the Rust scaffold with Bun development tooling, a JavaScript build, and checks under Bun and Node. Transcripts and settings remain read-only; results retain metadata and counts, never prompt/reply text.
Implements the reviewed plan from kixelated/token-max#2, now merged into main. This PR targets main and includes main through kixelated/token-max#4. Only replay implementation and its quest/dependency cleanup belong to this delta; landing configuration, SSH and T3 integrations are separate.
Changes
Validation
nix develop -c quest gate just check: formatting, TypeScript, JavaScript build, 62 synthetic tests under Bun 1.4.4 and Node 24, and quest validation pass.nix develop -c quest gate nix shell --inputs-from . nixpkgs#nodejs_22 -c node --test dist/test/*.test.js).nix develop -c quest gate node dist/src/cli.js --claude-dir /home/kixelated/.claude --since 2026-10-30 --prices-file .tmp/models.json --jsonexits 2 with zero success-output bytes at malformed complete recordagent-aac1432197b4da0d2.jsonl:194in the selected Claude subagent history. The ignored local diagnostic.tmp/policy-host-error.jsonretains its full safe file location. A separately selected Codex scan also exits 2 at a pre-existing malformed complete record. No transcript is changed or skipped. This is the first failure, not the only corrupt file: independent metadata inspection found multiple malformed complete records across several histories, so repairing one location would not establish a complete scan.~/.claude-api-creditssubset since 2026-10-30 contains 11,075 selected requests and 5,954 pre-window cache-context requests. The new estimator supports 16 of 29 main attempts; 13 main and one outside-main attempt lack sufficient evidence and keep their affected comparisons unpriced. These are selected-subset coverage counts, not complete historical billing totals.git diff --checkpasses. No settings or transcripts are changed and no package is published.Public API and wire impact
New CLI flags:
--since,--json, repeatable--claude-dir/--codex-dir,--offline,--prices-file, and--price-cache. New JSON schema version 1 includes profiles, frozen snapshots, requests, structural events, coverage, costs, summaries, and check findings, including the explicitscan.sincebilling bound,scan.cacheContextfor unbilled pre-window evidence, explicitcacheEvidencepredecessor witnesses with independent prefix evidence/cohort completeness, cache-comparison/fixed costs, structural-event origin aliases, recorded API-attempt budgets, comparison-only usage estimates with bracket provenance, and attempt sensitivity scenarios. The private development package reserves@kixelated/token-maxand thetoken-maxexecutable; final npm artifact/release work remains separate.Choices and limits
@streamparser/json; usesmol-tomlfor Codex configuration. Do not implement an SSH reader or any other check.preTokens/postTokens, with cache/overhead and timing limits. Do not treatprecomputedas proof of idle generation.Caused by
New requested functionality replacing an unused scaffold; no earlier PR caused a bug. Compatibility and diagnostic defects found in local review are fixed within this PR.
Models
Size: L
Implementer: Codex, gpt-6.1-sol, xhigh effort
Design change
The maintainer decision selects evidence-backed estimates with provenance and sensitivity; unsupported cases stay unknown. Attempts are never excluded or assigned observed zero usage. The policy was recorded in
quest/m0/replay.mdin 7afed1a. The maintainer subsequently chose B1: rethink cache tracking as explicit known, estimated and unknown states with predecessor provenance. This checkpoint is resolved and the narrow flag-only approach was rejected.The estimator uses immediately adjacent measured requests in the same conversation/file with matching attribution/client/pricing. It copies identical geometry, or conditionally estimates a growing-context retry within the minimum documented five-minute cache lifetime using the next measured reads and the preceding warm prefix, excluding the next request's own new writes from the attempt's total input. Recorded Claude compaction boundaries interrupt estimation, entrypoints must match, and historical settings visibility remains incomplete. It does not infer output tokens or actual execution. Raw usage, missing-usage coverage and historical spend stay unknown. Cache charges form the TTL comparison; ordinary input/output/server fees and identical summary proxies cancel under fixed behavior even when unknown. Missing scenario-dependent cache inputs/rates still block the affected comparison. Conditional no-billed/cache-affecting-attempt and cold-cache sensitivities can reverse the winner; they are not guaranteed bounds or an all-scenarios veto.
docs/semantics.mdrecords the exact supported evidence and limits.Review
Independent reviewer: Claude Opus 5.5, high effort, size L.
B1 session round 3: 9/10, clean, no unresolved findings. Reviewed head
c49d3f96485c6a7882e939bfe87d839975625b7bagainstorigin/mainat666ccb11f4e36fde6e06f593c2adcc396586a3fe. Fresh T3 delegate sessione42b2808-350f-4dea-89b9-c368230aadc1verified all round-2 and earlier fixes, the explicit state/compression model, scan billing bound, quest cleanup, and settled estimate/net-winner policies. The remaining pure-read TTL attribution limit and maintenance complexity are documented limits, with no requested code change.The reviewer ran 62 Bun tests and TypeScript, plus 1,900 synthetic full-versus-filtered histories and 1,500 subagent-isolation cases with zero mismatches/skips. The author's full checks also cover the built JavaScript under Node 24 and Node 22 and all 13 remaining quest documents. No implementation or background work remains.
Review history: Cross-provider reviews use the required T3 delegate route. Every recheck is a new delegate. B1 starts a fresh review session against main. Round 1 (Claude Opus 5.5, high, 7/10 at 80788ff, delegate
2b03747b-bc33-46f0-abee-d49175ea4697) verified the state/compression model and every earlier fix. It found: (1) explicit--sincediscarded pre-window evidence, (2) unknown-kind parts were filtered before informing other buckets, (3) estimated-prefix labels lost their evidence type with incomplete cohorts. All are fixed with separate pre-window cache context, all-part replay plus agent cohorts and unknown selected-bucket comparisons, and independentprefixEvidence/cohortKnownprovenance. Reader and built CLI tests cover these paths, invalid unselected usage and straddling request links. Round 2 (Claude Opus 5.5, high, 7/10 at b70a7de, fresh delegate611ba044-8148-4119-8e37-fafad42e0422) verified those fixes and found (1) missing pre-window summary claim order, (2) copied splits being treated as observed TTL evidence for estimated attempts, and (3) unknown kinds in a known subagent unnecessarily blocking main. All three are fixed and tested: process every boundary before selecting emitted charges, apply observed-TTL guards only to measured parts, and preserve the known outside-main role of subagents. Checks also inherit the scan billing bound and reject expanding a filtered scan. Round 3 verified all three fixed, with no unresolved findings.Earlier sessions verified the fixes for fixed-fee cancellation, source-cohort TTL transitions, copied boundary origins and pricing, managed ownership, API-key billing ambiguity, non-directory symlink discovery, empty usage iterations,
not_availablegeography, missing-usage diagnostics and sanitized internal errors. They also verified bracket estimation, matching client provenance, growing-retry writes excluding the following request's new writes, pre-window estimated-seed sensitivity and strict audit limitations.The last pre-B1 round (Claude Opus 5.5, high, 8/10 at c6c8252, fresh delegate session
ebd67db8-97c1-4ff8-b205-0a3f0aa947cd) found pre-window unsupported cache evidence disappearing or clearing the chain depending on missing speed. It triggered the mandatory design checkpoint, now settled by B1. The author reproduced opposite priced winners and replaced the implicit map with the explicit state model. Both metadata variants now leave the affected comparison unpriced. Explicit reader windows retain predecessor requests and compaction boundaries outside selected billing/coverage, with invalid unselected usage kept unknown. Unknown kind/agent attribution spans possible buckets; selected unresolved bucket charges stay unpriced in both conditional comparisons. Missing metadata spans compatible cohorts; known touches recover their own cohort; invariant geometry can remain priced with unknown attribution. Dense-history tests compare compressed witnesses against all possible last touches at lifetime boundaries. All earlier nonpolicy fixes must also be verified in the new review.(Written by GPT-6.1 Sol with xhigh)