Skip to content

signoff: the merge gate cannot go green on a loaded laptop — 6077/6078 with a different 5s-timeout test each run #576

Description

@drewstone

Symptom

pnpm signoff --source head — the repo's stated merge gate — could not return a green run on a developer laptop across six attempts, while 6077 of 6078 tests passed every time. Each failure was a different single test, always a 5-second default timeout or a wall-clock budget, never an assertion about behaviour.

Measurements (2026-09-11, same commit family, one machine)

attempt outcome failing test class
1 FAIL unit step killed at 1800 s step timeout under load
2 FAIL vitest worker EPIPE host fork exhaustion (~5,151 processes)
3 FAIL pnpm install exit 221 (generated projects) host fork exhaustion
4 FAIL src/signoff/workspace.test.ts ×3 5 s default timeout, shells out to git
5 FAIL src/web-react/use-file-mentions.test.tsx "well under 50ms" one-sample wall-clock budget
6 FAIL src/signoff/proof-record.test.ts "leaves the real index untouched" 5 s default timeout, shells out to git

Attempts 5 and 6 both reported 6077 passed, 1 failed. Every other step (install, typecheck, incident-class gates, build, generated projects, knip) was green in attempts 4–6.

Cause

Two test families encode machine speed rather than behaviour:

  1. src/signoff/*.test.ts create temp git repositories and run several git processes per test against vitest's 5 s default. On a contended box a single git init + commits exceeds it; the same tests pass 27/27 standalone on the same commit seconds later.
  2. Wall-clock performance budgets measured from one sample (expect(elapsed).toBeLessThan(50)).

docs/SIGNOFF.md already names this exact class and prescribes the mitigation:

A per-test timeout that encodes machine speed is the live example of this class … The honest mitigation is not a gate feature: raise or remove per-test timeouts that encode machine speed.

Proposal

  • Give src/signoff/*.test.ts an explicit per-file timeout sized for a loaded machine (30 s), since every test there pays process-spawn cost that has nothing to do with the assertion.
  • Convert remaining single-sample wall-clock budgets to best-of-N. feat(chat-routes): stream lifecycle progress live through model failover #575 already did this for rankFileMentions: an algorithmic regression still blows the budget on every sample, so the guard is unchanged.

Until then the gate is not reliably reachable on a laptop, which pushes contributors toward merging without it — the outcome docs/SIGNOFF.md exists to prevent. (That is what happened on #575, recorded in its merge comment.)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions