Skip to content

Split page.html, test_meter.py, and app.py so agents read less context - #140

Merged
bhavsarpratik merged 10 commits into
mainfrom
refactor/session-cost-split
Nov 6, 2026
Merged

Split page.html, test_meter.py, and app.py so agents read less context#140
bhavsarpratik merged 10 commits into
mainfrom
refactor/session-cost-split

Conversation

@bhavsarpratik

@bhavsarpratik bhavsarpratik commented Nov 6, 2026

Copy link
Copy Markdown
Member

Why

A survey of October's 100 most expensive agent sessions found that almost all of the cost is context, not output. In this repo's 68 sessions in that set, about 46% of the tool output that stayed in context came from grep and sed -n probes into three oversized files: page.html (1.26 MB), tests/test_meter.py (1.43 MB), and token_meter/app.py (631 KB). page.html alone is bigger than the context window, so it can never be read whole. The analysis and plan are in specs/plans/2026-11-06-session-cost-refactor.md, which is historical.

This PR breaks those files into parts small enough to read whole, and adds guardrails so they don't grow back.

What changed

  • Size budget (tests/contracts/test_source_size_budget.py): new source files must stay under 80 KB. Existing large files get a ceiling that can only be lowered or removed.
  • Dashboard (page.html → dashboard/): 42 parts, each under 80 KB, grouped as styles/, markup/, and js/ by view.
    • dashboard/page.manifest sets the serving order, and token_meter/dashboard_page.py concatenates the parts. There is no build step and no new dependency.
    • The served page is byte-identical to the old page.html.
    • /health reports ready only when every part listed in the manifest exists.
    • Updated to match: the runtime manifest, start script, Windows runtime check, CI (scripts/check-page), and docs.
  • Tests (tests/test_meter.py → 29 modules, tests/test_*.py): each module covers one surface. The same 1,034 test methods are kept with unchanged bodies, and no test is collected twice. Tests that read the page now use one assembled copy (tests/dashboard_page.py).
  • app.py (631 KB → 485 KB): 413 top-level helpers moved verbatim into token_meter/app_helpers/, and app.py re-exports them, so meter.NAME is unchanged.
    • Anything tests patch through meter, plus anything that depends on it, stayed in app.py.
    • Guard tests stop helpers from importing app.py and stop tests from patching moved names through meter.
  • Agent helpers:
    • scripts/show [FILE] NAME prints one definition with its line numbers.
    • scripts/check-page assembles the dashboard and parses its JavaScript.
    • specs/AGENTS.md delivery-flow bullets are condensed to pointers into the skills that already spell them out. The authority limits are kept inline.
  • Work evidence: dashboard/ scripts now count as frontend files.

Open branches that edit page.html or tests/test_meter.py

These branches will hit a modify/delete conflict. specs/CONTRIBUTING.md → "Porting a page.html or test_meter.py change" has a three-way port using scripts/split-dashboard; the recipe was rehearsed on a recent branch. Branches older than the splitter's anchors have to be ported by hand.

Validation

  • Independent tester on the final head 5785aca:
    • git diff --check is clean, and the assembled page matches 777a6d1:page.html (cmp).
    • scripts/check-page prints js ok, and py_compile and bash -n pass.
    • python3 scripts/run-tests runs 2540 tests: OK, 21 skipped.
    • The earlier head was also checked with an in-process server: GET, HEAD, assets, and /health match base, and a missing part returns 503 on both / and /health.
    • Staged-runtime parity passes, and fails when a part is removed.
  • Two project reviewers, one on the installer/runtime and shared-helper lens and one on the coverage, privacy, and docs lens. All findings were fixed and both approved 5785aca.

Not verified

  • A live ./scripts/install on macOS. The installed runtime was left on main.
  • Linux systemd and Windows install or upgrade on a real host.
  • Browser rendering. The page bytes are identical to before.

Follow-ups

  • Split services/work_insights.py and domain/work.py.
  • Move more code out of app.py once tests patch the modules that own it.
  • Trim specs/AGENTS.md further. That needs a decision on which rules can live only in skills.

🤖 Generated with Claude Code

bhavsarpratik and others added 10 commits November 6, 2026 08:31
@bhavsarpratik @claude
test(contracts): add a source-file size budget that can only shrink
9e614e4 
Large single files make coding-agent sessions expensive: agents cannot read
them whole, so they search them repeatedly and every result stays in context.
New source files stay under 80 KB; existing large files carry a ceiling about
5% above their current size that may only be lowered or removed.

Adds the October session-cost survey and refactor plan.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik @claude
refactor(dashboard): split page.html into per-view source parts
bf72e62 
page.html was 1.26 MB (about 315k tokens), larger than an agent's context
window, so every dashboard change began with dozens of grep and sed probes
whose results stayed in context. The October survey attributes about 46% of
carried tool output in this repo's sessions to that navigation.

The dashboard source now lives in dashboard/ as 42 parts (all under 80 KB):
styles/, markup/, and js/ files grouped by view. dashboard/page.manifest lists
their serving order and the server concatenates them byte for byte, so the
served page is identical to the old page.html. No build step or dependency.

- token_meter/dashboard_page.py: validated manifest, cached assembly
- server: serves the assembled page; assets resolve beside dashboard/
- runtime manifest, start script, Windows runtime check, CI updated
- tests read one assembled copy through tests/dashboard_page.py
- scripts/check-page replaces the inline node parse command
- scripts/split-dashboard re-splits a branch's own page.html by anchors,
  so open branches can port their dashboard edits

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik @claude
fix(work): count dashboard/ scripts as frontend files
fa507a3 
The dashboard now lives in dashboard/js/*.js. Work evidence classified
.js outside a recognized frontend folder as backend, so dashboard edits
would have read as backend work. Add dashboard(s)/ to the frontend folders.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik @claude
test: split test_meter.py into one module per surface
63eb35d 
tests/test_meter.py was 1.43 MB with 97 classes, so agents grepped it
instead of opening the tests for the surface they changed. Its classes now
live in 29 modules under 80 KB each (test_dashboard_*.py, test_pricing.py,
test_menubar_tray.py, ...). The 230-method DashboardLayoutTests and the
OpenCode tests are split by topic onto shared bases that hold their
original setup and helpers (dashboard_layout_support.py,
opencode_test_support.py); module-level fixtures move to meter_support.py.

Coverage is unchanged: the same 1,034 test methods, byte-identical bodies,
and the full suite passes serially and in parallel.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik @claude
refactor(app): move leaf helpers out of app.py into app_helpers/
1638eb3 
app.py was 631 KB. Move 421 top-level functions and constants (about
150 KB) into 12 modules under token_meter/app_helpers/ (trace parsing for
Claude, Codex, and Cursor, normalization, the menu-bar payload, config-file
parsing, and shared leaf helpers). app.py re-exports every moved name, so
meter.NAME and token_meter.app.NAME are unchanged.

Only code that is safe to move moved. A dependency analysis kept in app.py
anything that tests patch through meter (including patched imported modules
such as meter.threading), anything declared global or rebound, anything that
reads __file__ or sys.modules, and anything that depends on those. Helper
modules may import only app_helpers.common, never app.py; a guard test
enforces this. The composition guards now cover the helpers too, and the
app.py size ceiling drops to 510 KB.

Function bodies are moved verbatim; the full suite passes unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik @claude
chore(agents): add bounded lookup helpers and condense delivery-flow …
316d689 
…rules

- scripts/show [FILE] NAME prints one Python or dashboard JS definition
  with line numbers, replacing wide grep and sed -n probes (about 46% of
  carried tool output in October's sessions).
- specs/AGENTS.md is loaded on every turn of every session and subagent.
  The reporter, PR-status, and contributor-acknowledgment procedures,
  which the intake, github-ops, and communication skills already spell out,
  shrink to a pointer that keeps their authority limits.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik @claude
fix: address review findings on the session-cost split
bf60853 
- Dashboard parts no longer end in a blank line (git diff --check was red):
  the splitter now moves leading blank lines with each anchor. Re-split
  from the old page.html; the assembled page is still byte-identical.
- /health reports page_ready and ok only when every manifest part exists,
  so a partial runtime no longer reads as ready while / returns 503.
- specs/CONTRIBUTING.md documents a rehearsed three-way port for branches
  that still edit page.html or tests/test_meter.py; split-dashboard refuses
  a folder that already holds parts.
- A contract test rejects patching names that moved into app_helpers
  through meter.
- dashboard/ counts as frontend only as a folder (dashboards?/), with a
  negative test case.
- specs/AGENTS.md delivery bullets restate their gating and scope limits.
- Stale page.html references updated; plan header notes implementation.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik @claude
docs(agents): keep the reporter flow's commit before its gates
5785aca 
The condensed bullet placed the commit after the gates, contradicting the
intake and github-ops skills: gates bind to a commit, so the scoped commit
comes first and the push, PR, issue handoff, and Slack reply follow once
every gate passes. Also correct the health_state docstring.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik @claude
Merge main into refactor/session-cost-split
ad18cb8 
Port main's edits to files this branch split or moved:
- page.html changes (#138, #139) applied to dashboard/ parts with the
  CONTRIBUTING three-way recipe; the assembled page equals main's page.html.
- tests/test_meter.py hunks moved to test_dashboard_models.py and
  test_menubar_tray.py; new page reads use the assembled page.
- The debug skill's code map now points at dashboard/ (new release hash
  registered); its path test also searches app_helpers.
- Size ceilings re-measured after the merge for test_git_delivery.py,
  services/work_insights.py, and services/git_delivery.py, which grew on
  main before the budget landed.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik @claude
fix(git): let simultaneous first opens of the delivery ledger share o…
302b764 
…ne salt

CI on this branch failed with "UNIQUE constraint failed: delivery_metadata.key":
two test workers created the same fresh ledger at once, and the salt was
read, then inserted, in separate steps. The server and the MCP can race the
same way on a new machine. Insert with OR IGNORE first and read the stored
salt back, so the first writer wins and nobody fails. A test injects the
competing write between the two steps; it fails on the old code with the
CI error and passes now.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@bhavsarpratik
bhavsarpratik merged commit 696243d into main Nov 6, 2026
10 checks passed
@bhavsarpratik
bhavsarpratik deleted the refactor/session-cost-split branch November 6, 2026 06:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@bhavsarpratik