Skip to content

refactor: remove production code that exists only for tests - #185

Merged
kixelated merged 6 commits into
mainfrom
quest/m1/test-only-code
Nov 21, 2026
Merged

kixelated merged 6 commits into
mainfrom
quest/m1/test-only-code

Conversation

@kixelated

@kixelated kixelated commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m1/test-only-code.md: production no longer carries anything only tests read, and the quest file is deleted.

Summary

Idle's detail trace (src/idle.rs, src/compaction.rs)

  • Production code no longer builds a trace that only tests read. Deleted: IdlePeriod, PeriodCalibration, Period::detail, the detailed_checks / detailed_t3_checks wrappers, the [Vec<IdlePeriod>; 2] return tuples, the Profiled::detail flag, and every detail.get_mut(i).filter(|_| point) block in Runner::run and t3_replay.
  • Also deleted:
    • Calibration::events, which only that trace read.
    • scale, which became dead.
    • Period::cohort, which activate now keeps as a local.
  • idle_checks, t3_idle_checks, Profiled::scenario and t3_comparison now return their checks directly.
  • About 20 test sites now assert on what the report uses:
    • PeriodCounts kinds, through a small periods(counts, |kind| ...) helper.
    • by_context.
    • activation_cohorts.
    • Exact deltas, derived by hand in comments.
  • No cfg(test) trace was needed. The per-period summary costs and calibration geometry are pinned through those exact deltas. Two examples:
    • In the same-cohort test, the delta is applied + 0.05 + 0.1: one measured session, a final period at the cohort's 0.5 fraction, and another version falling back to full activation.
    • The T3 test asserts the proactive cost as a $0.10 summary plus b rewriting 40k instead of 320k at $5/MTok. It also checks the cost rises by exactly 10k writes when the sample resumes 10k larger.

Serialize on the Scan tree (src/model.rs and the privacy tests)

  • Removed:
    • 73 Serialize derives (72 in model.rs, plus Unreadable).
    • 121 #[serde] attributes.
    • The six hand-written impls: Id, Few, Settings, ServerTools, Ids, Exposure.
  • Serde stays only on types that wire, sessions or doctor carry. Usage also keeps a bare Serialize, since its JSON is the reader's Codex dedup key and feeds request ids; it drops renames that changed nothing.
  • model.rs's header now says this, instead of claiming the SSH stream carries these types.
  • Privacy tests now check format!("{x:?}") instead of serde_json::to_string. That is stricter, since Debug includes fields serde skipped.
    • One test needed narrowing: t3::tests::native_history_is_shared_and_exact_run_evidence_never_fills_billed_tiers. Debug shows Scan.sessions, which holds native session IDs on purpose for the sessions export, so the check empties that field first and says why.
    • Three serialization-shape assertions are gone, each already covered by a structural assertion: the exposure.rs references count, the Id JSON in safe.rs, and Ids JSON equality in model.rs.

Compile time

Median of 3, after touch src/lib.rs, measured before merging main. The machine was loaded, so the CPU time is the steadier figure.

wall before wall after CPU before CPU after
cargo build 12.8s 11.8s 18.1s 13.3s
cargo test --no-run 24.5s 14.9s 45.0s 38.4s

That is about 25% less CPU for the build and 15% less for the test build.

Validation

  • just check passes after merging main: 589 lib tests, plus the cli, run_script and ssh suites.
  • Real surface: I built main and this branch at release and ran both on this host's real transcripts at the same moment: doctor --details --plain --offline --since 2026-10-01 (216,794 requests, native and T3 idle lines included) and sessions --since 2026-10-01. With the report's now timestamp normalized, both doctor reports are byte-identical, and the sessions exports are identical too. Both doctor runs exit 1, which is the code for its red recommendations.
  • Second, independent check (mutations): I broke production in four ways to confirm the rewritten tests still catch what the trace caught. Each mutation fails tests:
    • Ignoring the cohort fraction fails the same-cohort test.
    • Shifting the native resumed offset by 1k fails 7 tests.
    • Adding 100 tokens to the native summary output fails 12 tests.
    • Shifting the T3 calibrated prefix by 1k fails the T3 five-minute test. The relative check alone missed this mutation, which is why that test now also asserts the absolute cost.

Public API

  • No wire change. wire::Outcome and sessions::Session keep their serde.
  • The library re-export token_max::BackgroundFeature no longer implements Serialize. The crate's re-exports serve its own binary and integration tests, and nothing serialized this type. Choice for review: I left the title without !, since the crate publishes a CLI rather than a library API.

Choices for review

  • I used aggregates for every test site instead of a cfg(test) trace. The quest allowed either.
  • In the narrowed T3 privacy test, the check empties Scan.sessions first, as described above.

Follow-ups

None.

Models

Size: M
Implementer: Claude Code, claude-opus-5-5, unknown effort (arm B)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Review

Rewritten by quest review after each round.

  • Head: b12e04f

  • Base: origin/main at 12de0b6

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, high effort

  • Session: 01a1fda8-5638-7fb1-bade-3d54fa73b78d, round 1

  • Score: 9/10, The refactor preserves production behavior and wire/export serialization with no actionable regressions found in static review, but builds and tests were not run as instructed.

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

  • Reviewer: Codex, gpt-6.1-sol, medium effort, shadow

  • Session: 01a1fda8-5638-7602-baee-7e7fa20ae1ae, round 1

  • Score: 9/10, The refactor removes test-only tracing and unused serialization without apparent production behavior changes or actionable defects, but builds and tests were not run as instructed.

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

Summary by CodeRabbit

  • Refactor
    • Idle-check results now focus on aggregate period counts and context-size totals; per-period detail is no longer included.
    • Serialization support has been narrowed for internal data, while serialization for wire, session, and other designated data remains.
  • Tests
    • Updated checks to verify sensitive fixture text is absent from debug output rather than serialized output.

Quest-Claim: b928b367c07ed32b
@coderabbitai

coderabbitai Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The changes remove per-period detail collection from idle checks and replay paths. They also remove Serde serialization from many model types and update related tests. Calibration no longer stores sample event IDs.

Changes: Idle check aggregation

Layer / File(s) Summary
Remove per-period replay details
src/compaction.rs, src/idle.rs
Calibration no longer stores sample event IDs. Idle replay and scenario paths stop collecting per-period details and return aggregate results.
Return aggregate idle checks
src/idle.rs
Native and T3 check paths return aggregate checks and comparisons without period-detail vectors.
Use aggregate idle assertions
src/idle.rs
Idle test helpers and assertions use aggregate counts, groups, and scenario totals instead of individual period records.

Model serialization changes

Layer / File(s) Summary
Remove serialization from model types
src/model.rs, src/unreadable.rs
Many model types lose Serde serialization derives, custom serializers, and serialization attributes. Usage retains Serialize with fewer naming rules.
Update serialization-related tests
quest/m1/test-only-code.md, src/compaction.rs, src/discovery.rs, src/exposure.rs, src/model.rs, src/reader.rs, src/safe.rs, src/subagents.rs, src/t3.rs
Serialization-specific assertions are removed or replaced with checks of Rust debug output. The proposed-change document is deleted.

Estimated code review effort: 4 (Complex) | ~40 minutes

Change: Refactor

Suggested reviewers: pullfrog

Merge Risk: Minimal · up to b12e0

This change removes test-only per-period idle detail and Serialize derives from model types that do not need them. The replacement tests use aggregate and Debug-based checks. No user-facing or production behavior change was identified, and the remaining serialized types (wire, sessions, doctor, Usage) are unchanged. The change looks safe to merge once CI passes.

Security architecture risk: Minimal · up to b12e0

The refactor removes test-only detail and narrows serialization capabilities. The inspected command-line and remote output paths retain their existing boundaries; no introduced or worsened security or failure-containment risk was established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The inspected change affects transcript-derived analysis and Rust serialization capabilities. It does not establish expanded remote access to the scan tree: the SSH implementation continues to construct explicit report or session outcomes from a destination-local scan.

Trust Boundaries and Controls

  • observed — Removing Serialize also removes the corresponding serde skip annotations; it does not replace them with serialization of previously hidden fields. Unreadable retains its existing in-memory hash sets, and session export retains a separate serialized representation.

Resilience and Maintainability Implications

  • observed — The changed idle transitions retain the same cohort key and full-activation fallback. Scenario execution still rebuilds session periods for independent passes and finalizes failed sessions. The T3 replay deletion removes detail writes while retaining cost accumulation and cache/context updates; no persistent partial transition is introduced by these deletions.
Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "refactor: remove production code that exists only for tests" matches the main change. The PR removes Idle's test-only per-period trace and related types, and removes Serialize derives, serd…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "refactor: remove production code that exists only for tests" matches the main change. The PR removes Idle's test-only per-period trace and related types, and removes Serialize derives, serde attributes, and manual implementations from the Scan tree that existed only for privacy tests. The title is specific and short. It does not name every part of the change, such as the Debug-based privacy tests, but that is expected.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
Fix failing CI checks

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7a1e1097-5459-4cb4-8d1c-0b5d5015fc63
Commits

Reviewing files that changed from the base of the PR and between 12de0b6 and b12e04f.

Files selected for processing: 11

  • quest/m1/test-only-code.md
  • src/compaction.rs
  • src/discovery.rs
  • src/exposure.rs
  • src/idle.rs
  • src/model.rs
  • src/reader.rs
  • src/safe.rs
  • src/subagents.rs
  • src/t3.rs
  • src/unreadable.rs
Files with no reviewable changes: 3

  • quest/m1/test-only-code.md
  • src/exposure.rs
  • src/safe.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.

kixelated and others added 5 commits November 21, 2026 02:21
Nothing in production serializes the scan or the model types under it:
the SSH stream carries wire::Outcome and the session export carries
sessions::Session. The derives, serde attributes and six hand-written
impls (Id, Few, Settings, ServerTools, Ids, Exposure) served only
privacy tests, which now check Debug output instead. Debug includes the
fields serde skipped, so those tests are stricter. Usage keeps a plain
Serialize for the reader's Codex total key and request identity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Production built an `IdlePeriod` per period and threaded a `detail` slice
through every native and T3 run only for tests to read. Tests now assert
on the period counts, context groups, cohorts and exact deltas the report
uses, and `Calibration::events`, which only that trace read, goes too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Return idle scenarios directly, name every remaining serde user in the
model header, and delete the finished quest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Remove production code that exists only for tests refactor: remove production code that exists only for tests Nov 21, 2026
@kixelated
kixelated marked this pull request as ready for review November 21, 2026 10:32

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Approach: sound. It deletes the idle detail trace and the Scan tree's Serialize surface outright and moves the tests onto aggregates the report already exposes, so no cfg(test) scaffolding takes their place.

Reviewed changes

I read the full diff at b12e04f, checked that serialized output is unchanged, and ran clippy and the lib tests locally.

  • Idle trace removal: Runner::run and t3_replay lose only the detail.get_mut(i).filter(|_| point) side-recording blocks. Every ledger.charge / add path is untouched. activate now resolves the cohort through IndexMap::get into a local, with the same fraction fallback to 1.0.
  • Test rewrites in src/idle.rs: per-period lookups became PeriodCounts / by_context / activation_cohorts assertions plus hand-derived exact deltas. A few assertions are now aggregate rather than per-request, for example "no eligible periods" instead of "a is absent". The cost assertions beside them still pin the behavior.
  • Serialize removal in src/model.rs:
    • Usage dropped rename_all = "camelCase" and its explicit write5m/write1h/… renames. None of its fields contains an underscore, so the JSON keys are unchanged, and so are the reader's Codex dedup key and request ids.
    • The remaining serde types (Harness, Location, Usage, T3IdleMode, Margin, ContextKind, Usd) match the new module header.
  • Privacy tests now use Debug: the crate's only hand-written Debug impls are Id (prints the full text) and Ids (debug_list). Neither redacts anything, so the format!("{x:?}") checks are real and stricter than the serde ones they replace. The T3 test that empties Scan.sessions first is narrowed only for the native session ids the sessions export keeps on purpose.
  • Validation: cargo clippy --all-targets --locked -- -D warnings is clean and cargo test --lib passes all 589 tests. No docs or learnings still mention the removed symbols.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision, in the merge pick round for this PR:

  • Pick: ✅ /quest-merge (rejected: /quest-iterate, skip)
  • Title: ✅ leave ! off for dropping Serialize from BackgroundFeature (rejected: add ! and a migration note). The library surface is treated as internal to the CLI.

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

The maintainer decision needs no change: head b12e04f already has the title without !.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Nov 21, 2026
Merged via the queue into main with commit c325c20 Nov 21, 2026
10 of 11 checks passed
@kixelated
kixelated deleted the quest/m1/test-only-code branch November 21, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant