Skip to content

Compare native idle compaction on/off net costs - #8

Merged
kixelated merged 5 commits into
mainfrom
quest/m0/idle-compaction
Nov 7, 2026
Merged

kixelated merged 5 commits into
mainfrom
quest/m0/idle-compaction

Conversation

@kixelated

@kixelated kixelated commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Complete the idle-compaction quest. Compare native Claude idleCompaction on/off at current controls and after the TTL/window recommendations, and recommend the lower estimated aggregate net cost. Charge final unresumed summaries, preserve full system/project/resumed prefixes, carry context changes through later requests and summaries until convergence, and retain actual returned cache evidence. Sensitivity explains the estimate instead of vetoing it.

The shared terminal renderer shows both baselines, window/monthly on-minus-off costs, opportunities and final/censored counts, lifetime/summary/timing sensitivities and break-evens. Details/JSON retain period eligibility, attribution, activation basis, calibration and context-size costs. Current owner/version/prerequisite guards control actionable changes. Settings and transcripts remain read-only.

Design choices

  • Reuse landed cache replay, prices, linked/proxy compaction charges and the window simulation. Export the existing summary/full-prefix calibration and selected-window replay for this downstream consumer. Do not duplicate the analyzer or renderer.
  • Resolve native user/project/local/managed idle controls and active-profile environment owners. Preserve scoped TTL/window owners rather than applying a profile recommendation over a project override. Unknown execution, flags, ownership or required geometry stays advisory/unpriced.
  • Verify native informational idle-notice shapes and entrypoints from Anthropic v2.1.316, and explicit parentUuid lineage from v2.1.319. Only an explicit path through compact-summary/attachment records to an automatic boundary attributes idle application; orphan notices stay unattributed. precomputed, boundary time and duration do not establish idle generation. Native transcripts do not invent historical settings or flag-fetch records. When comparable known-on metadata is absent, use the accepted open-and-awake documented-trigger fallback, including its charge on final periods.
  • The documented trigger is near one hour. Use the inspected v2.1.316 90%-lifetime timer as an explicit timing estimate, with one-hour sensitivity; it is not a documented timer control or a 55-minute setting. Actual returned reads/writes take precedence over gap-only inference.
  • Weight activation branches and merge equal context states. Price full prefixes and later summary inputs until observed convergence; avoid fractional-token pricing across tiers. The closed/open break-even replays a common uncertain activation fraction, including sequential effects.
  • Merge current main, including kixelated/token-max#7, and integrate evidence into src/report.ts with the existing plain/details/JSON/cancellation flow.
  • Remove the completed quest and references. Keepalive remains downstream work. No threshold tuning, T3/SSH adapter, settings edits, transcript text, publication or version bump is included.

Public API and wire impact

JSON schema version 1 gains the idle-compaction member of the discriminated CheckResult union and its period/scenario evidence. Profiles gain optional current idle controls and project-directory owners. Structural events recognize verified native informational idle notices while discarding their text. TypeScript consumers must narrow by check. CLI flags, atomic exit 0/1/2 and the read-only/privacy contract are preserved. Human reports use the common renderer; booleans display as true/false.

Validation

  • nix develop --command quest gate just check passes: formatting, TypeScript, built JavaScript, 121 synthetic tests under Bun and Node 24, and quest checks.
  • Idle tests cover both directions, a zero tie, aggregate winners contrary to most periods, a sensitivity crossing zero, known and unknown final lifetime, recorded and inferred activation, cohort isolation, unsupported clients/versions/flags/controls, current versus recommended prerequisites, full prefix calibration, actual warm/cold resumed evidence, near-expiry returns, later summary repricing/convergence, current project owners (including pre-window cwd evidence), comparison estimates without explicit windows, cleared abandoned/id-less tool waits, explicit notice lineage/orphans, strict malformed notices, selected windows and privacy.
  • Provider guidance checked 2026-11-07: native settings, native cache behavior, API cache lifetime. Native record-shape verification inspected provider code only; no real transcript content was printed or committed.
  • No complete local-host dollar audit was attempted. Multiple corrupt complete histories remain a known blocker to that separate audit; old host totals are not validated by these tests.
  • Independent exact-head review passed in round 2 at 8/10, with zero open findings; all eight fixes and both requirement-based refutations were accepted. The record is below. Parent handles landing; this worker does not merge or wait for GitHub CI/reviewers.

Caused by

  • kixelated/token-max#6 treated a lower-scope autoCompactEnabled: false as disabling compaction even when a higher scope enables it. The reused resolver now respects the effective highest scope. The remaining work is the new idle quest.

Models

Size: M
Implementer: Codex, gpt-6.1-sol, xhigh effort

Review

Rewritten by quest review after each round.

  • Reviewer: Claude Code, claude-opus-5-5, medium effort
  • Session: 582b3f08-facc-4995-b375-2e98e0e1126d, round 2
  • Head: 5948df3
  • Base: origin/main at 1b8d74b
  • Score: 8/10, Every earlier finding is either fixed at the head or refuted by the documented contract, and spot checks of the changed code turned up nothing new, though this round's look at new code was limited.
  • Findings: none
  • The last round's findings, at af79046:
    • Fixed: high, src/idle.ts:1057: Idle scenario drops usage estimates when windows replay nothing; src/idle.ts:1106 now passes `options.estimates` when replayWindows returns the group unchanged (`part.requests === group`), so estimates are no longer dropped.
    • Fixed: medium, src/reader.ts:950: Idle owners are discovered only for post-since requests; src/reader.ts:1001 calls discoverIdleOwners with `allRequests`, which includes the pre-since cacheContext requests, not just `sortedRequests`.
    • Fixed: medium, src/idle.ts:255: Pending tool set never clears for id-less or abandoned tool calls; src/idle.ts:290 clears `pending` on every turn-start or turn-end, so abandoned and id-less tool calls no longer persist past their turn.
    • Fixed: medium, src/reader.ts:934: Idle notice attributed to any earlier compaction with no time bound; src/reader.ts:972-995 follows explicit parentUuid lineage to an auto, same-session boundary that is not yet attributed, and no longer picks the latest earlier compaction by time.
    • Refuted: medium, src/reader.ts:613: One idle notice without a timestamp aborts the whole scan; README.md:52-54 and docs/semantics.md:464 record the contract that a complete malformed selected record fails the whole scan and that there is no skip-malformed mode, so throwing on a recognized notice with no timestamp is intended.
    • Fixed: medium, src/idle.ts:939: Math.min spread can overflow the stack on large histories; src/idle.ts:989 now takes the minimum with a reduce instead of spreading every request into Math.min.
    • Refuted: medium, src/idle.ts:903: A single unknown period nullifies the whole profile's idle recommendation; quest/m0/README.md:95-96 states that missing inputs which cannot be estimated prevent a priced recommendation, so a null aggregate when a period is unknown is the specified behavior.
    • Fixed: low, src/idle.ts:1144: Break-even bisection has no tolerance; each step is a full replay; src/idle.ts:1214 stops the bisection once `high - low <= Number.EPSILON`, which bounds it to about 52 iterations.
    • Fixed: low, src/idle.ts:1075: Quadratic replayed-part lookup, and replay repeated per owner; src/idle.ts:1120-1130 builds a `replayedParts` Map keyed by `${id}:${part}` in place of the per-part `find`, and the author reports the replay is now hoisted out of the per-owner loop.
    • Fixed: low, src/idle.ts:1186: Activation-cohort aggregation duplicated between periods() and idleChecks; periods() returns `activationCohorts` (src/idle.ts:441), and idleChecks destructures and reports that same value (src/idle.ts:1146, 1290) instead of a duplicate cohortMap.

(Written by GPT-6.1 Sol with xhigh)

kixelated and others added 4 commits November 7, 2026 10:54
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
@kixelated kixelated changed the title Idle compaction check Compare native idle compaction on/off net costs Nov 7, 2026
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: Native idle notices now require an explicit parentUuid path through compact-summary or attachment records to an automatic boundary. An orphan notice no longer relabels an older compaction by timestamp. Shared TTL/window replay is also computed once per baseline and reused across idle owners.

Two review suggestions conflict with the accepted task requirements, so the review response preserves those requirements: complete malformed selected idle notices fail the atomic scan, and required inputs without an accepted estimate leave the aggregate net unpriced. The latter still reports conditional costs; estimated activation, client lifetime, and sensitivity continue to use the settled aggregate-winner policy.

The full local gate passes 121 tests under Bun and Node 24. The next independent round is reviewing committed head 5948df33257f706f4029af84bf3c3a4c9db3a572 against current main.

(Written by GPT-6.1 Sol with xhigh)

@kixelated
kixelated marked this pull request as ready for review November 7, 2026 19:32
@kixelated
kixelated merged commit 7692106 into main Nov 7, 2026
2 checks passed
@kixelated
kixelated deleted the quest/m0/idle-compaction branch November 7, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant