test(agent-ux): define a fast inner verification loop and retain the full completion gate

[Jamie-BitFlight]

Parent: #280
Related performance investigation: #148

Outcome

Give agents and contributors a documented, mechanically reliable fast inner verification loop for iterative changes, while keeping the existing full repository gate authoritative before completion.

Why

#148 measured that the current test suite spends a large portion of wall time in repeated subprocess/import startup and that tests marked slow are still included in ordinary pytest runs.

For an agent making small iterative edits, a slow feedback loop increases context switching and encourages under-verification. The answer should not be to weaken final verification; it should be to distinguish fast affected-boundary checks from the full completion gate.

Current-main reconciliation — 2026-09-29

PR #290 completed the fast/full verification split:

• fast iteration: uv run python scripts/run_fast_tests.py [pytest args];
• the fast profile applies -m "not slow" and disables repository-wide coverage for narrowed iteration;
• AGENTS explicitly states that this is affected-boundary evidence, not completion proof;
• CI runs the same Fast Tests profile while preserving the full Python 3.11/3.12 pytest jobs.

Remaining work is the measured #148 program: intentional ownership of process-isolated tests and reducing subprocess/import startup only where the boundary does not justify its lifecycle cost.

Required distinction

Inner loop

A fast command or small set of commands intended to answer:

Did the affected seam regress?

It may use:

• targeted pytest files;
• a defined not slow profile;
• focused static checks;
• in-process CLI tests where process isolation is not part of the behavior being tested.

Completion gate

The existing repository-level required checks remain authoritative, currently including the full pre-commit/prek gate and full pytest expectations defined by repository policy/CI.

Passing the inner loop must never be described as proving repository-wide completion.

Plan

1. Resolve perf(tests): suite spends ~two thirds of its wall clock on subprocess startup #148's measurement program sufficiently to identify safe fast-path changes.
2. Decide what the normal local pytest contract should mean:
   • fast suite by default plus explicit full/slow suite, or
   • preserve full pytest and provide a separate named fast command/profile.
3. Inventory subprocess tests and distinguish:
   • tests that genuinely require process/installation isolation;
   • tests that can use the established in-process Typer/CliRunner path;
   • packaging/E2E tests that belong in the slow/full tier.
4. Provide a stable command/interface for agents rather than prose recipes that must be rediscovered.
5. Update AGENTS.md so agents know:
   • what to run during iteration;
   • which affected-boundary tests are expected;
   • what must be run before declaring completion.
6. Keep CI coverage of the full behavior.

Constraints

• Do not classify a test as slow merely because it currently performs poorly; the tier should reflect the boundary it proves.
• Do not replace process-boundary tests when the process boundary is the behavior under test.
• Do not reduce final coverage to make agents faster.
• Measure material changes rather than assuming xdist/coverage/subprocess effects.

Acceptance criteria

• A documented fast verification command/profile exists.
• Its intended proof boundary is explicit.
• The full completion gate remains explicit and authoritative.
• Slow/E2E/process-isolation tests have intentional ownership.
• perf(tests): suite spends ~two thirds of its wall clock on subprocess startup #148's slow-marker mismatch is resolved or deliberately superseded by the chosen model.
• Repeated subprocess startup is reduced where process isolation is not required.
• CI still runs the full required verification.
• Agent instructions no longer force the entire suite for every tiny intermediate edit while still requiring full completion verification.