Skip to content

test: restore cost fixture for large sessions - #35960

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/35875-restore-ci-fixture
Nov 12, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/35875-restore-ci-fixture

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Refs #35875

Problem / Motivation

Goal: Pin the cost of restoring a large session with count-based CI tests, so the bounded-restore work is measured against a fixed fixture.

Nothing in CI guards how many rows a restore parses for a 16 MiB transcript, or how many times a slot's crew-log unit is walked when its folds are primed cold. The plan for bounded restore is in draft PR #35916.

Why it matters

Without a fixture, a performance change to restore has nothing to be measured against, and a regression in either count would reach main unnoticed.

Not a goal

  • No product code change and no restore behaviour change.
  • No wall-clock assertion.
  • No tail window, focus-first order or single-walk fold; those are separate changes that flip the target tests here.

What changed (motivation → approach → change)

Restore cost has to be pinned before it is reduced. The new test, test/test_restore_large_session_fixture.py, builds a synthetic home at test time (made-up ids and text, real row and record shapes): one 16 MiB transcript plus 40 small tabs, and one crew-log slot unit of about 2 MiB written through CrewLog. The unit is about 1/8 of the 16 MiB size in the plan; walks per fold do not depend on unit size.

  • Eager tab at the threshold: rows parsed. Ceiling is one full read (5451 rows). Strict xfail target is at most 500 rows.
  • Lazy tab: parses at most the 100-row tail (passes).
  • Cold prime of the 12 slot folds: walks of the unit. Ceiling is one per fold (12). Strict xfail target is 1 walk.

Backwards compatibility: Compatible: test-only change, nothing is rejected, renamed or removed.
Removes nothing: only a test file is added.

Tests

The new file runs 5 passing tests and 2 strict xfail tests (the targets). With --runxfail the targets fail on the counts above (5451 vs 500 rows, 12 vs 1 walks), so they fail for the intended reason.

Proof: not provable -- test-only change

Manual verification

N/A — unit coverage sufficient; the change is a test.

Related Issues

Refs #35875

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 12, 2026
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 800bebf11b9bf099d106c33d5e3bf202e539af53 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 800bebf11b9bf099d106c33d5e3bf202e539af53 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 800bebf11b9bf099d106c33d5e3bf202e539af53 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the change only adds tests, and its count-based ceilings plus strict-xfail targets give the bounded-restore work a fixed fixture to be measured against.

Notes

  • The eager-tab ceiling parsed_rows["full"] <= rows has no lower bound, so it still passes if a change silently stops reading the eager tab. Add > 0, as the lazy-tab and crew-log tests already do, so the ceiling cannot be passed by doing nothing.

[DESIGN-REVIEWED] 800bebf

Description read

[DESCRIPTION-READ] b46e3da6be47331283a66e2f158079024a71dbd4f1e7bdb07afec06347256d91

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 800bebf11b9bf099d106c33d5e3bf202e539af53 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 800bebf

Verdict parsed from the review's SHA-scoped output markers for commit 800bebf11b9bf099d106c33d5e3bf202e539af53.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 800bebf11b9bf099d106c33d5e3bf202e539af53 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 800bebf11b9bf099d106c33d5e3bf202e539af53 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 800bebf

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 800bebf11b9bf099d106c33d5e3bf202e539af53 false-positive: <reason, 20 to 500 characters>

@chenmingwei23
chenmingwei23 force-pushed the fix/35875-restore-ci-fixture branch from bd9a889 to 800bebf Compare November 12, 2026 18:13
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 12, 2026
@chenmingwei23
chenmingwei23 marked this pull request as ready for review November 12, 2026 20:08
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 12, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the full diff, a test-only change for #35875 that touches no product code. It builds a synthetic home at test time with made-up ids and text in the real row shapes. It then pins restore cost by counting work (rows parsed at the reader, CrewLog._walk_from calls), never by timing, so it cannot flake on a slow runner. Each target over today's bar is a ceiling that fails on regression, plus a strict xfail stating the target, so reaching it cannot go unnoticed. Regexes, if any, would be test-only; there are none. The 6 reds are cancelled or earlier failed runs that newer passing runs on this head superseded. LGTM.

@KiroCrewCaptain
KiroCrewCaptain merged commit c12eb7b into main Nov 12, 2026
158 of 174 checks passed

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: adds one test file that pins restore cost by counting parsed rows and crew-log walks; no source change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants