Repository navigation
efficiency: what slows the t27b and network lanes, and the fix for each (measured 2026-10-10) #8637
Copy link
Copy link
Open
Labels
Description
Owner, 2026-10-10 (translated): "implement everything, and think how to do it more efficiently". Part of epics #8326 and #8613.
Measured bottlenecks and the fix for each
- Every spec PR edits one ledger file, and the lanes fight over it. docs/reports/t27b_expectations.json takes a row per new spec, so parallel lanes conflict and re-merge. Master's native ratchet also turns red whenever a spec lands without its row (t27b ledger: rows for 9 specs merged without one (Closes #8287) #8289, t27b ledger: master's native run at c9d36f314, row by row (Closes #8322) #8548).
- Fix: spec PRs stop touching the ledger. A steward step adds rows from master's own run and opens one PR with auto-merge. The rules already live in .t27 (specs/tri/t27b/steward.t27); only the runner is missing (a Queen job card, jobs.t27). See t27b ratchet: UNLISTED blames the PR that adds the spec, not every later PR #8321, option 3.
- The lab queue is capped at 4 requests, and request runs come after master runs. With 4 agents, a fifth request is dropped as QUEUE_FULL.
- Fix A: REQUEST_QUEUE_MAX 4 -> 8 in specs/verified/corpus_receipt.t27. t27b-lab: take up to two lane requests per master run (Closes #8586) #8588 already runs 2 requests per master run, and requests expire after 6 h, so 8 still fits.
- Fix B: a requester writes to whichever lab has the shorter queue. t27b-lab-2 (receipts: register the second lab signer, t27b-lab-2, key fed03daa6459a7fa #8624) runs the same image, so the receipts are interchangeable.
- A quorum takes two full runs of one commit.
- Fix: when lab B signs, it reuses lab A's published inputs (the input_root leaves). It re-runs only the verdicts and then compares, so a second signature costs a run, not a run plus a rebuild. The reference cache (lab.py read_ref_cache) already covers part of this.
- Each agent costs about 5% of the weekly budget per hour, measured 2026-10-09/10. Most of that goes on re-reading files and rebuilding context.
- Fix: agent tasks are one PR each, with the files to read named in the prompt.
- Fix: lab worktrees and the target dir are shared (already done).
- Fix: receipts are compared after the merge, never by waiting inside the agent.
- Fix: run at most 4 agents, and as few as possible while the 5-hour window is above 50%.
- A spec rewrite by another session looks like a t27b regression (UNEXPECTED FAILURE on c757043).
- Fix: the ratchet names the commit that last changed each failing spec, so a reader sees at once whether the spec or t27b moved.
Order
5, 2A and 1 are .t27-only and unblock the most; then 2B, 3 and 4.
User Scenarios
- Given a spec PR lands without its row in the ledger, when the ratchet runs, then it stays green because the steward step adds the row from master's own run in a single auto-merge PR.
- Given the lab queue is at 4 requests, when a fifth request arrives, then it is accepted because REQUEST_QUEUE_MAX is 8.
- Given a spec was rewritten in another session, when t27b reports UNEXPECTED FAILURE, then the ratchet names the commit that last changed that spec so the reader sees whether the spec or t27b moved.
Requirements
- FR-001: The steward spec (specs/tri/t27b/steward.t27) MUST define a step that reads master's run output and opens one PR adding all missing rows to docs/reports/t27b_expectations.json with auto-merge.
- FR-002: specs/verified/corpus_receipt.t27 MUST change REQUEST_QUEUE_MAX from 4 to 8.
- FR-003: The lanes spec (specs/tri/lanes/lanes.t27) MUST make the ratchet emit the commit that last changed each failing spec alongside the failure.
- FR-004: Spec PRs MUST stop editing docs/reports/t27b_expectations.json directly; the steward step is the sole writer.
Success Criteria
t27c check specs/tri/t27b/steward.t27exits 0 and the compiled steward step adds rows to the ledger in a single PR when invoked.grep -F REQUEST_QUEUE_MAX specs/verified/corpus_receipt.t27shows the value 8.t27c check specs/tri/lanes/lanes.t27exits 0 and a failing spec report includes the last-changed commit hash for that spec.- No spec PR in the last 20 merges touches
docs/reports/t27b_expectations.json(verified bygit log --oneline -20 -- docs/reports/t27b_expectations.jsonshowing only steward PRs).
Boundary
- specs/tri/t27b/steward.t27
- specs/verified/corpus_receipt.t27
- specs/tri/lanes/lanes.t27
- docs/reports/t27b_expectations.json
Reactions are currently unavailable
Activity
corpus_receipt: REQUEST_QUEUE_MAX 4 -> 8, so a fifth lane's request i…
t27b ledger: rows for 9 more specs that landed without one (Closes #8645