chore(scripts): guard the file set each coverage run measures (#2709)

[cliffhall]

Closes #2709

Problem

The per-file ≥90 gate only judges files that reach coverage-summary.json, and nothing checked that the report holds the files it should. During #2557, vitest 5 shrank web's report from 418 files to 11, and local:gate stayed green.

What changed

• scripts/verify-coverage-scope.mjs (new, npm run verify:coverage-scope). This is the issue's preferred shape: it compares each report against an independent expansion of the client's own declaration.
  • For each of the five gated clients, it resolves the coverage include/exclude through Vitest, so it checks the patterns the run actually used, including Vitest's hard-coded default excludes.
  • It expands those patterns with Node's own fs.globSync / path.matchesGlob, never Vitest's matcher.
  • It fails if any declared file is missing from the report, naming the client and every missing file. It also fails when the include expands to nothing and when a summary is missing.
  • There is no count to maintain.
  • A client with a test:coverage script that the guard doesn't list fails (deny-by-default, like verify:test-timeouts).
  • Intentional differences go in KNOWN_OMISSIONS, which must state a reason. It is empty today.
  • The script's header comment states how a pattern is read. A relative pattern resolves against the client directory, unless it starts with the client's own repo path (web's clients/web/server/**). Excludes match leniently, so they can only shrink what the guard expects.
• Wiring: it is the last step of the root coverage script, so it runs in local:gate and in CI's coverage job. It reads coverage output, so it is not part of validate.
• clients/web/vite.config.ts: src/utils/pendingReauth.ts is added to web's existing pure-type coverage excludes. It contains only type declarations, so v8 reports no row for it, and the guard flagged it on unmodified v2/main. That was the only difference on the current branch.
• Tests: scripts/verify-coverage-scope.test.mjs has 14 cases in test:scripts. They cover the pattern-reading rules, the diff, the roster, and run() over a fixture repo, including the vitest-5 collapse to clients/web/server/**.
• Docs: a row in docs/quality-gate.md, a diagnosis entry in the pre-push-gate skill, one sentence in AGENTS.md next to the existing coverage-whitelist rule, and a comment on the CI step.

Acceptance

• v2/main as-is passes. npm run local:gate is green. The guard reports web 419, cli 20, mcpdo 31, tui 41, launcher 1, matching the counts in the issue.
• Re-applying the vitest 5 bump fails. In a throwaway worktree I changed the four exact pins (root vitest and @vitest/coverage-v8, web's @vitest/browser-playwright and its vitest override) to 5.0.4, as chore(deps): monthly dependency refresh #2557 did, and ran web coverage. The report held 11 files, and the guard failed with web: 408 of 419 file(s) … missing, listing each one.
  • That install needed legacy-peer-deps, because @storybook/addon-vitest@10.5.5 peers browser-playwright ^4.
  • The vitest 5 suite also hit unhandled errors, which are part of the migration chore(deps): monthly dependency refresh #2557 describes.
  • So the trial shows the report collapse and the red guard. It is not a clean vitest 5 run otherwise.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

The omission escape hatch accepts missing reasons and fails to detect exemptions that become stale.

1 open finding

• Validate omission reasons before honoring exemptions · New

What changed in this PR

Adds a durable guard ensuring coverage reports contain every file declared by each client’s coverage configuration.

Changes:

• Adds and tests verify:coverage-scope.
• Wires the guard into coverage execution and CI.
• Documents the guard and excludes a type-only web module.

File	Description
scripts/​verify-coverage-scope.mjs	Implements coverage-scope validation.
scripts/​verify-coverage-scope.test.mjs	Tests expansion, comparison, roster, and failure cases.
package.json	Adds and wires the verification command.
clients/​web/​vite.config.ts	Excludes a type-only module.
docs/​quality-gate.md	Documents the new gate.
AGENTS.md	Adds the coverage-scope rule.
.github/​workflows/​main.yml	Documents CI coverage enforcement.
.claude/​skills/​pre-push-gate/​SKILL.md	Adds troubleshooting guidance.

🧠 Review effort: Balanced

[aron-intframe]

verify:coverage-scope fails when a client's include expands to nothing at all, but not when one pattern inside it goes dead. That leaves #1301's half of this class, a path change rather than a tooling upgrade, where it was.

I ran this PR's own expandCoverageScope / diffCoverageScope at 2b1db6b, feeding them the include/exclude arrays as written in each client's config rather than resolving them through Vitest. The expansion reproduces the counts in the PR body exactly, which is what makes the second run worth reading:

web       patterns 15  expected 419  matching nothing: 0
cli       patterns  2  expected  20  matching nothing: 0
mcpdo     patterns  1  expected  31  matching nothing: 0
tui       patterns  1  expected  41  matching nothing: 0
launcher  patterns  1  expected   1  matching nothing: 0

Then I renamed one gated directory on disk and left include untouched, so src/theme/**/*.{ts,tsx} is still declared and simply matches nothing:

v2/main + PR #2711 : expected 419 file(s); patterns matching nothing: 0/15
after `git mv src/theme src/themes` (include left untouched):
  expected 395 file(s)  — 24 files left the gate
  patterns matching nothing: 1/15  -> src/theme/**/*.{ts,tsx}
  diffCoverageScope -> 0 failure(s)
  verify:coverage-scope prints "web: 395 declared file(s), all measured" and exits 0

diffCoverageScope got a report equal to the shrunken expansion there, which is what the run produces: the coverage run reads the same declaration, so those 24 files leave coverage-summary.json too. Expected and reported agree at 395 and the gate is green with 24 files outside the per-file gate. An IDE-driven directory rename updates every import and leaves the whitelist in vite.config.ts alone, so nothing else goes red either. expected.length === 0 fires only when every pattern for a client dies at once, which is the vitest 5 shape, not the path-change shape.

The fix fits the loop that already exists: expand each include pattern on its own and fail when one yields zero files, naming the client and the pattern. The first block above is that check run over all five clients on this branch, and no pattern is dead today, so "v2/main as-is passes" still holds. An intentionally empty pattern would then want the KNOWN_OMISSIONS treatment (state a reason) instead of passing silently.

We keep a catalog of this shape; the entry is "expected to find this, found nothing is a failure, not a pass", and the check we run for it is guard-selftest.py (ours): https://github.com/aron-intframe/agent-guards

[Copilot]

🟢 Approval recommended

The guard is correctly wired, comprehensively tested, and addresses the reported silent coverage regression.

0 open findings

1 resolved since last review

• Validate omission reasons before honoring exemptions

🧠 Review effort: Balanced

[cliffhall]

Review summary:

• Round 1 (1 finding): KNOWN_OMISSIONS honored entries with no reason. Fixed in a305dca: an omission is honored only with a non-empty string reason, and an entry without one is reported as a failure on its own. Also added: an omission for a file the report already measures is flagged as stale. Tests cover both. Answered in the thread.
• Round 2: clean (0 open findings, 1 resolved). Review loop complete.

[cliffhall]

✅ Verdict: MERGEABLE

The guard passed on unmodified real coverage reports, both locally (macOS) and in CI (Linux), with identical counts. It failed correctly in every negative case tried. CI is green, DCO passes, and the branch is 0 commits behind v2/main. Nothing blocking. Two non-blocking notes are at the bottom.

---

Smoke test of PR head a305dcaf

The PR branch was checked out in a fresh worktree, then npm install, npm run build and npm run coverage were run.

Positive: real reports

Where	Result
Local, the full npm run coverage chain	✓ web 419 · cli 20 · mcpdo 31 · tui 41 · launcher 1, exit 0
CI coverage job (run 38341760321)	✓ same five counts, success

The counts match the ones given in the PR body and in #2709.

Negative: the guard must go red

All four cases were run against real reports and restored afterwards.

#	Mutation	Result
1	Web summary cut down to the 11 clients/web/server/** rows, which is the vitest 5 / #2557 shape	✗ web: 408 of 419 file(s) … missing (the report holds 11), with 244 clients/web/src/** files among those listed, exit 1
2	New type-only file clients/web/src/utils/smokeTypeOnly.ts	✗ web: 1 of 420 … missing, naming the file, exit 1
3	Row for core/mcp/inspectorClient.ts dropped from web's summary	✗ web: 1 of 419 … missing, naming core/mcp/inspectorClient.ts
4	clients/launcher/coverage/coverage-summary.json removed	✗ launcher: no …coverage-summary.json — run its coverage first. A missing summary fails; it is not skipped.

After restoring, the guard exits 0 again and git status is clean.

Other checks

• node --test scripts/verify-coverage-scope.test.mjs: 16/16 pass. The PR body says 14; the follow-up commit added the two omission cases.
• prettier --check and eslint --max-warnings 0 on both new scripts pass.
• verify:skills passes. The listing is at 3907/4000; the new entry is skill body text, so the listing cost is unchanged.
• The pre-push-gate diagnosis entry covers both failure causes and the missing-summary case.

Non-blocking notes

1. The guard repeats a Vite warning four times. Resolving web's config through createVitest re-emits Vite's configLoader: 'native' deprecation warning (about __dirname and extensionless imports in vite.config.ts) four times. That adds about 28 lines between the guard's banner and its verdict. The warning comes from the existing config, not this PR, so this is only noise. Setting VITE_CONFIG_NATIVE_IGNORE_WARNING=true in the guard's environment, or fixing the config later, would quiet it.
2. New type-only files now need an exclude entry. Negative 2 shows the trade-off this guard makes. A new type-only module under a gated web directory now fails coverage until someone adds it to the coverage exclude, because v8 reports no row for it. That is the intended behavior, and the skill entry points to the fix. It is called out here so the first person to hit it isn't surprised.

🤖 Generated with Claude Code