You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
refactor: calibrate compaction summaries once for window, idle and T3 replays - #180
Window (compaction), idle and T3 idle replays now size a compaction summary from one calibration, so a fix to how summaries are grouped or measured reaches all three. Finishes quest/m1/summary-calibration.md.
Changes
src/compaction.rs: CompactionSamples::new builds one Calibration per group: the samples in order, uniformity of the summary pricing cohort, and median and maximum Geometry (output, prefix, read, overhead). It groups once by SampleCohort (window replays, cohort()) and once by SampleKey, which is SampleCohort plus version and client (idle and T3, idle()).
src/idle.rs: ReplayIndex reads the shared calibration instead of regrouping per replay. The tuple SampleKey, sample_key, and idle's own Geometry/Calibration are gone. t3_replay looks up the same calibration instead of scanning every sample for each gap and checking uniformity field by field.
Repeated event ids: CompactionSample::event is now the last event recorded with the boundary's id, not just the id. Every replay reads that event's pre_tokens for summary overhead, and window replays use it as the template for a modeled boundary. Before, ReplayIndex used the last event, while T3 and the window template (through WindowEvents::by_id) used the first. Following the quest's Plan, the last record now wins everywhere.
WindowEvents::by_id existed only for that template lookup, so it is now a plain event list that suppress_idle filters.
docs/semantics.md says the three replays share one calibration, that idle and T3 also match client and version, and that a repeated boundary calibrates from its last record.
The quest is deleted. quest/m1/test-only-code.md now notes that Calibration::events only feeds PeriodCalibration, so it goes when the detail trace does. The trace itself is otherwise unchanged.
Net: -64 lines in src/ before the new test.
Choices made without asking
Idle and T3 now also reject an Agent::Unknown request, because SampleKey contains SampleCohort, which already did. The output can't change: idle and T3 replay only Claude profiles, and Claude requests are always Main or Subagent (Unknown occurs only for Codex).
The window template now follows the last record too. The quest named only idle and T3, but its Goal is one calibration for all three replays.
Left unchanged:compaction_samples still links the first CompactionCharge per event id. That decides which charge applies, not how a summary is sized, and charges come one per scanned event.
Validation
just check passes: quest check, fmt, clippy -D warnings, 587 lib tests plus the integration tests.
New test a_repeated_event_calibrates_every_replay_from_its_last_record: two records of one boundary with different pre_tokens. It checks that both the window and idle lookups give overhead 500 from the last record, and that idle also needs a client version while window replays don't. Switching last to first-wins makes it fail (tried).
Differential (scratch test, not committed): 300 randomized sets of 1 to 12 samples across provider, version, client, agent, model, tier, speed, summary model/provider and pre_tokens, all with unique ids. For every probe it compared the old T3 linear scan plus field-by-field uniformity and medians, and the old window cohort grouping, against idle() and cohort(). Median and max output, prefix, read and overhead, uniformity, template sample and event order were all identical. The only difference was Agent::Unknown, as intended.
Ran the release binary from main and from this branch, doctor --details --plain --offline --since 2026-10-20, on this host's live transcripts. Live sessions kept writing, so two main runs already differed from each other, and a byte-for-byte comparison wasn't possible. The calibration-derived figures matched exactly, e.g. added compactions "output 4168 to 5728 tokens; resumed prefix 25413 to 26120 tokens". The other idle, window and T3 lines differed only by the same small drift seen between the two main runs.
Public API
None. All the touched modules are private. No wire or report-format change.
Caused by
kixel-ai/token-max#36 - the T3 replay looked up a sample's event with find, so it kept the first record of a repeated id while idle's ReplayIndex kept the last (found by person: m1 code audit)
Follow-ups
None new. The edit to quest/m1/test-only-code.md adds one line to that quest's existing work.
Models
Size: M
Implementer: Claude Code, claude-opus-5-5, unknown effort (arm B)
Score: 9/10, No actionable regressions were found: shared calibration preserves replay behavior while consistently using the last event record, though builds and tests were not run as instructed.
Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0
Findings: none
Reviewer: Codex, gpt-6.1-sol, medium effort, shadow
Score: 9/10, The shared calibration preserves existing cohort matching and replay calculations while consistently using the last event record; code inspection found no actionable regressions, but builds and tests were not run as instructed.
Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0
Findings: none
Summary by CodeRabbit
Improvements
Compaction window, idle, and T3 replays now use consistent summary and resumed-prefix calibration.
Idle and T3 calibration also matches the client and version, helping keep replay comparisons specific to the corresponding environment.
When a boundary is recorded more than once, calibration uses its latest record.
Documentation
Clarified the calibration behavior for compaction windows and idle and T3 replays.
Compaction, idle, and T3 replay now use shared calibration data. Idle and T3 calibration additionally match client and version. Duplicate event records use the last record for calibration.
Changes: Compaction replay calibration
Layer / File(s)
Summary
Shared calibration and event sampling src/compaction.rs, docs/semantics.md, quest/m1/summary-calibration.md, quest/m1/test-only-code.md
Calibration groups samples and derives geometry from structural events. Duplicate records use the last record. The semantics documentation and quest plans reflect these calibration details. Tests cover duplicate records and idle client/version matching.
Window replay calibration src/compaction.rs
Added-pass simulation uses cohort calibration geometry. Added-compaction evidence uses event and resumed-request IDs from calibration samples.
Idle and T3 calibration src/idle.rs
Native idle and T3 replay use the shared idle calibration lookup. T3 uses its median geometry and calibration events instead of local calculations.
kixel-ai/token-max#174: This PR lists summary calibration as a follow-up item that PR #180 implements.
Suggested reviewers:pullfrog
Merge Risk: Minimal · up to 19958
This change shares summary calibration across the window, idle and T3 replays, and repeated boundary records now use the last record. No concrete regression was established in review. The one open item is the failing CI gate job, which should be rerun and confirmed green before merge.
Security architecture risk: Minimal · up to 19958
The inspected change remains within local replay calculations. It does not introduce a new external interface or privileged action, and missing or incompatible calibration remains explicitly unpriced.
Retained concerns
No architecture-level concerns identified.
Security review details
Security Blast Radius
inferred — The demonstrated exposure is to transcript-derived replay estimates and modeled events, including existing keepalive calculations. The changed paths do not introduce a new external caller or privileged sink. Modeled window events explicitly receive the target request's session and origins rather than inheriting the calibration event's ownership.
Trust Boundaries and Controls
observed — Cohort, client, and version values are calibration-matching attributes, not authorization identities. Sample construction still requires a measured positive resumed prefix and linked positive billed summary output. Missing matching attributes or ambiguous summary pricing lead to unavailable calibration or unknown cost, not additional authority.
Resilience and Maintainability Implications
inferred — Calibration is built before consumption and borrowed through shared references; mutable simulation state remains local to a replay and its sessions. The inspected calibration transition adds no persistent partial-commit state requiring cancellation cleanup, migration, or rollback coordination.
Pre-merge checks: 4
Check name
Status
Explanation
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Title check
The title names the main change: compaction summary calibration is computed once and shared by window, idle, and T3 replays. The PR objectives and the src/compaction.rs and src/idle.rs summaries match…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the main change: compaction summary calibration is computed once and shared by window, idle, and T3 replays. The PR objectives and the src/compaction.rs and src/idle.rs summaries match this. The title is a single concise sentence using the conventional refactor prefix.
Finishing Touches
Generate unit tests (beta)
Commit to this branch
Create a new PR
Fix failing CI checks
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
… replays
One Calibration per cohort, built in CompactionSamples, now gives window,
idle and T3 replays their summary geometry. Idle and T3 key it by the
window cohort plus client and version. A boundary recorded more than once
calibrates from its last record in every replay; T3 took the first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…for test-only-code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
changed the title
Calibrate compaction summaries once
refactor: calibrate compaction summaries once for window, idle and T3 replays
Nov 21, 2026
The reason will be displayed to describe this comment to others. Learn more.
✅ No new issues found.
Approach: sound. The PR collapses three copies of the summary grouping and measuring code into one Calibration built per SampleCohort and per SampleKey. Repeated boundary ids now resolve to the last record in a single place (compaction_samples), not in three places that disagreed.
Reviewed changes
I reviewed the full diff across src/compaction.rs, src/idle.rs, docs/semantics.md and the two quest files.
Shared Calibration: CompactionSamples::new groups the samples twice through the generic group(). The window lookup cohort() and the idle/T3 lookup idle() both index into one calibrations vector. Median and max output, prefix, read and overhead match the old idle formulas.
Last record wins: CompactionSample::event now holds the last StructuralEvent recorded for the id. Idle's ReplayIndex events come from all_events, a stable sort of time-sorted events plus modeled: ids, so its last record is the same as before. Only T3 and the window template change from the first record to the last, which is the intended change.
T3 replay: samples.idle(..).filter(|c| c.uniform) replaces the linear scan and the field-by-field uniformity check. I checked the overhead (including input == None) and the read median against the old code, and they match.
WindowEvents: by_id is gone, so suppress_idle now filters every event instead of the first record per id. In practice nothing changes. The reader's add_event already dedups ids inside scan.events, and every pre-window cache_context.boundaries entry passes the e.time < since filter anyway.
Docs and test: docs/semantics.md describes the shared calibration and the client/version match. The new test covers a repeated id and the version requirement that applies to idle only.
The reason will be displayed to describe this comment to others. Learn more.
Additional comments: 2
src/compaction.rs: 1
235-240: Trivial · Performance & Scalability · Heavy lift
Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.
Skip idle-only calibration work for window-only callers.
compaction_checks and keepalive only use window geometry. They now build idle cohorts too, and each window cohort sorts cache-read and overhead values and allocates an event-ID list it never uses. Build the idle index only for callers that use it, and keep read, overhead, and trace data out of window cohorts. Native idle and T3 can still share one indexed calibration across their replay runs. (github.com)
docs/semantics.md-370-371: 1
370-371: Minor · Functional Correctness · Quick win
Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.
Specify which fields use the last boundary record.
The transcript reader merges copies with the same ID and retains the first copy’s token metadata. If repeated records reach replay, the last supplied record sets summary overhead and supplies the window event template. Resume matching and summary linkage still run per record. State that distinction so readers do not expect a later transcript copy to replace the earlier sample. (raw.githubusercontent.com)
Review info
Run configuration
Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 0712d450-fecf-4e6a-a983-883499dec3db
Commits
Reviewing files that changed from the base of the PR and between 99d2cc0 and 1995854.
Files selected for processing: 5
docs/semantics.md
quest/m1/summary-calibration.md
quest/m1/test-only-code.md
src/compaction.rs
src/idle.rs
Files with no reviewable changes: 1
quest/m1/summary-calibration.md
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 3 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Window (compaction), idle and T3 idle replays now size a compaction summary from one calibration, so a fix to how summaries are grouped or measured reaches all three. Finishes
quest/m1/summary-calibration.md.Changes
src/compaction.rs:CompactionSamples::newbuilds oneCalibrationper group: the samples in order, uniformity of the summary pricing cohort, and median and maximumGeometry(output, prefix, read, overhead). It groups once bySampleCohort(window replays,cohort()) and once bySampleKey, which isSampleCohortplus version and client (idle and T3,idle()).src/idle.rs:ReplayIndexreads the shared calibration instead of regrouping per replay. The tupleSampleKey,sample_key, and idle's ownGeometry/Calibrationare gone.t3_replaylooks up the same calibration instead of scanning every sample for each gap and checking uniformity field by field.CompactionSample::eventis now the last event recorded with the boundary's id, not just the id. Every replay reads that event'spre_tokensfor summary overhead, and window replays use it as the template for a modeled boundary. Before,ReplayIndexused the last event, while T3 and the window template (throughWindowEvents::by_id) used the first. Following the quest's Plan, the last record now wins everywhere.WindowEvents::by_idexisted only for that template lookup, so it is now a plain event list thatsuppress_idlefilters.docs/semantics.mdsays the three replays share one calibration, that idle and T3 also match client and version, and that a repeated boundary calibrates from its last record.quest/m1/test-only-code.mdnow notes thatCalibration::eventsonly feedsPeriodCalibration, so it goes when the detail trace does. The trace itself is otherwise unchanged.Net: -64 lines in
src/before the new test.Choices made without asking
Agent::Unknownrequest, becauseSampleKeycontainsSampleCohort, which already did. The output can't change: idle and T3 replay only Claude profiles, and Claude requests are alwaysMainorSubagent(Unknownoccurs only for Codex).compaction_samplesstill links the firstCompactionChargeper event id. That decides which charge applies, not how a summary is sized, and charges come one per scanned event.Validation
just checkpasses: quest check, fmt, clippy-D warnings, 587 lib tests plus the integration tests.a_repeated_event_calibrates_every_replay_from_its_last_record: two records of one boundary with differentpre_tokens. It checks that both the window and idle lookups give overhead 500 from the last record, and that idle also needs a client version while window replays don't. Switchinglastto first-wins makes it fail (tried).pre_tokens, all with unique ids. For every probe it compared the old T3 linear scan plus field-by-field uniformity and medians, and the old window cohort grouping, againstidle()andcohort(). Median and max output, prefix, read and overhead, uniformity, template sample and event order were all identical. The only difference wasAgent::Unknown, as intended.mainand from this branch,doctor --details --plain --offline --since 2026-10-20, on this host's live transcripts. Live sessions kept writing, so twomainruns already differed from each other, and a byte-for-byte comparison wasn't possible. The calibration-derived figures matched exactly, e.g. added compactions "output 4168 to 5728 tokens; resumed prefix 25413 to 26120 tokens". The other idle, window and T3 lines differed only by the same small drift seen between the twomainruns.Public API
None. All the touched modules are private. No wire or report-format change.
Caused by
find, so it kept the first record of a repeated id while idle'sReplayIndexkept the last (found by person: m1 code audit)Follow-ups
None new. The edit to
quest/m1/test-only-code.mdadds one line to that quest's existing work.Models
Size: M
Implementer: Claude Code, claude-opus-5-5, unknown effort (arm B)
🤖 Generated with Claude Code
Review
Rewritten by
quest reviewafter each round.Head: 1995854
Base: origin/main at 99d2cc0
Risk: medium,
the base has no RISK.md to name what is always high or customer-facing, so every change is mediumClassified: medium by default
Reviewer: Codex, gpt-6.1-sol, high effort
Session:
01a1fbd9-11b4-7420-b6e3-c96a7799f590, round 1Score: 9/10,
No actionable regressions were found: shared calibration preserves replay behavior while consistently using the last event record, though builds and tests were not run as instructed.Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0
Findings: none
Reviewer: Codex, gpt-6.1-sol, medium effort, shadow
Session:
01a1fbd9-11b3-74b0-b4fd-c013da862dde, round 1Score: 9/10,
The shared calibration preserves existing cohort matching and replay calculations while consistently using the last event record; code inspection found no actionable regressions, but builds and tests were not run as instructed.Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0
Findings: none
Summary by CodeRabbit