feat(swe-auto): cost-aware router skill (#123)

[aarora79]

Summary

Implements /swe-auto (issue #123), the first concrete slice of docs/vision.md: a router skill where the developer does not pick the model. Given a repo, a ref, and a problem (a description or a GitHub issue link), it triages the task into a tier (budget / workhorse / frontier), consults the measured cost/quality Pareto frontier, selects the cheapest non-dominated model that clears that tier's band, runs /swe3 through the in-repo headless runner with that model, optionally judges it, and escalates one tier on shortfall (bounded by max_escalations), writing a routing.json.

Follows the sequence diagram in vision.md end to end.

Design decisions

• Monorepo executor (Option B). The skill runs the in-repo run-swe-headless.py in place; no packaged/pinned installable and no vendored copy (a copy would drift, which is wrong for a benchmark repo). Standalone/external use is a follow-up.
• Ephemeral one-task dataset. The dataset-driven runner is unchanged; /swe-auto writes a synthetic single-task dataset and invokes it with --dataset/--tasks/--model/--agent/--skill swe3.
• Frontier is the selectable universe. model_execution is a thin "how to launch" layer, not a model catalog: Bedrock recipes are built in (self-hosted needs an endpoint), so the router never picks a model it cannot run.
• Split of concerns. swe_auto_router.py is the pure, deterministic decision core (frontier parse, tier->band, cheapest-clearing select, reliability gating, escalation); swe_auto_run.py orchestrates (ephemeral dataset, executor, judge, escalation loop, preflight, routing.json). The one model-driven step (triage) lives in the skill.
• Default harness: pi.

Components

• benchmarks/scripts/swe_auto_router.py - decision core.
• benchmarks/scripts/swe_auto_run.py - executor + escalation loop + preflight, with --dry-run and --preflight.
• .claude/skills/swe-auto/ - SKILL.md, triage-examples.md, swe-auto.example.yaml.
• docs/swe-auto.md - user guide; linked from docs/vision.md.

Testing

• 43 new unit tests (unittest); full suite 283 pass; ruff / mypy / bandit clean.
• Security-check gate: APPROVED (two low-severity, non-blocking notes).
• Smoke-tested end to end on the hello-world task (budget -> haiku, 6/6 artifacts, routing.json written).

Scope / follow-ups (out of v1)

• Per-phase routing and in-flight model switching.
• Packaging the runner as a pinned installable for out-of-repo use.
• Dispatching triage to a distinct router_model (v1 triages with the active session model; both are recorded).

Closes #123.

[aarora79]

Closing in favour of #161, which reworks this into a model router: an advisory skill that recommends a model instead of routing and running the task itself.

The idea here was right — use the measured Pareto frontier to pick a model rather than guessing — but two things about this design did not survive contact with the newer data and a clearer product framing.

It only works inside this repository. /swe-auto needs the headless runner, a dataset, and endpoint config for every model it might select. The people who want model advice are not in this repo; they are in their own codebase, in whatever assistant they already pay for. The measurements are the asset, not the harness that produced them, so the new design vends the data and gives advice, and stops there.

It routes on the wrong signal. This design triages a task into a tier and maps that tier to a quality band. On the v2 dataset, complexity explains 6% of the variance in the per-task gap between claude-opus-5 and claude-sonnet-5 — the other 94% sits between tasks inside the same tier, and five tasks all labelled trivial split from +0.4 to +13.8. What survived the same data was the quality-bar framing: ask what happens if the change is wrong, then take the cheapest available model clearing that floor.

Nothing here is wasted. The frontier parsing, the band-and-select logic and the tests are the starting point for #161, and the branch stays put if we want to pull from it.

Follow the work in #161.