[feature] supervised-run usability: watch, gating clarity, planner selection, per-issue delivery, verify and attempt records #50

Summary

Two live runs on 0.3.0 found gaps in how an origin agent supervises a run, beyond the bugs filed as #46, #47 and #48. Run 1 was smota/meshloop #44 and #45: 2 nodes, Claude Sonnet workers, delivered as PR #49. Run 2 was a sprint for smota/holoself #31–#34: 12 nodes, Claude and Codex workers, 3 graph versions. Each gap below cost a human round-trip, hand-written tooling or a wrong assumption. This issue tracks them together. Split out any item that grows.

Not covered here: prompt delivery and wrapper scripts (#27), spend-aware routing (#17), and the status, liveness and output bugs (#46, #47, #48).

Version / OS

meshloop 0.3.0, origin/main @ 2769f08. Windows 11 native, Claude Code 2.1.292, codex-cli 0.160.1.

Findings and proposals

1. No way to wait for a run except polling inspect (high)

In both runs the origin agent wrote its own polling loop. It ran inspect --json every N seconds, diffed node states with jq, and treated changes as events. Every orchestrator rewrites this loop, and it depends on status and inspect being correct, which #46 and #47 show they aren't yet.

Proposal: meshloop watch --graph <id> [--config] [--json] prints one NDJSON line per node or plan state change. It exits 0 when the run needs a human (AwaitingReview, awaiting_acceptance) or reaches a terminal state, and exits non-zero on failure. The exit reason goes in the last line. The skills (meshloop-run, meshloop-status) should point to it.

2. Acceptance gating isn't clear from tier or status (high)

• Run 2's operator expected only Tier3 nodes to stop for accept. In fact every node stops at AwaitingReview, and the tier only selects the harness. The operator found this out by reading the code after node 1 (Tier2) stopped.
• The plan used "HUMAN GATE: wait for PR merge" in node descriptions. Meshloop doesn't enforce prose, so a dependent node could start from main without the merged change.

Proposal:

• Document the rule in docs/start.md and the meshloop-review-plan skill: every node waits for acceptance; tier routes harnesses only.
• status and inspect show a waiting_for reason on each blocked node (acceptance, dependency <id>, retry budget).
• Optional, needs a decision: a node-level gate (human (default) | verify), so read-only or low-risk nodes can advance on a passing verify_command. This would be a change to ADR 0009, so it belongs in a proposal.

3. The planner needs a harness labelled top (medium)

Run 1 wanted a low-cost Sonnet-only setup. plan failed with MidTier does not fit task Tier3 until Sonnet was given model_tier = "top". As a result, the plan record says "top" for a mid-tier model, which makes the provenance and any later cost analysis wrong.

Proposal: a config key planner_harness = "<id>" (or planner_tier) that picks the planner directly. Without it, fall back to the current rule. When no harness fits, the error names the rule and the key that overrides it.

4. Delivering a multi-issue graph as one PR per issue is manual (medium)

integrate merges every accepted node into one branch. Run 2 had four issues, and AGENTS-style policy wants one PR per issue. The plan was to cherry-pick each issue's commits from the integrate branch onto fresh branches by hand.

Proposal: an optional deliverable tag on each node (e.g. "issue-32"), plus integrate --deliverable <tag> --into <branch>. That merges only that group's nodes, plus their accepted dependencies, onto the target branch. status groups nodes by deliverable.

5. An empty verify_command passes every node without a word (medium)

Run 1's config had verify_command = []. In node 2, the worker added tests that didn't compile: a non-Sync test double was passed into a Sync harness interface. The node passed "verification" anyway and reached AwaitingReview. Nothing in doctor, run, status or inspect said no verification had run. The origin agent found the problem only after integrating, when it ran cargo test itself.

Proposal:

• doctor --config and run warn when verify_command is empty.
• An empty command records a verify: skipped evidence row instead of nothing, so the attempt doesn't look as if it passed a check.
• inspect shows the verify command and its exit status next to the diff, so the reviewer sees what was and wasn't checked before running accept.

6. Attempts don't record duration or model (medium)

Working out what run 1 cost (about $4.50 estimated, about $5.70 if everything had run on Opus) meant parsing each worker's harness transcript. Meshloop records HarnessExited but not wall time or the resolved model_ref in a form inspect shows.

Proposal: store model_ref, started_at, ended_at and the exit status on each attempt, and show them in inspect --task. Add a run summary to status: attempts per node, total worker wall time, retries used. Token usage can stay out of scope, or be read from harness JSON output when present, which ties in with #17.

7. Supervised operation needs the user to run the start commands (low, docs)

Under Claude Code auto mode, the permission classifier blocks the origin agent from running meshloop plan and run, because they start unattended writing agents. That is reasonable. But the skills don't mention it, so the agent tried, failed, then handed the user commands to paste. One of those had a <scratchpad> placeholder that PowerShell parsed as a redirect.

Proposal:

• The meshloop-plan and meshloop-run skills say up front that launching workers needs either an explicit permission rule the user adds for meshloop plan and meshloop run, or the user running the command. When the user runs it, the skill gives a ready-to-paste command with absolute, quoted paths and no placeholders.
• Read-only commands (status, inspect, watch, doctor) stay agent-runnable.
• doctor --json could report whether an origin session was detected, since run 2 had to pass --origin-harness and --origin-session by hand.

8. Unknown flags are dropped silently (low)

args.rs builds Command::Status from --graph only, so status --config x --db y parses fine and the two flags are ignored. That is the parser half of #46. It's a general problem: a mistyped or unsupported flag should be an error.

Proposal: each subcommand rejects flags it doesn't define, with an error that names the flag and the subcommand.

Suggested order

1 and 2 first: they decide whether an agent can supervise a run without guessing. Then 3, 5 and 8, which are small. Then 4 and 6.

Acceptance criteria

• Each item above is either done, split into its own issue, or explicitly declined with a reason recorded here.
• watch: fixture test where a run with one node reaching AwaitingReview makes watch exit 0 with that reason as its last line.
• Unknown flag test: status --bogus exits non-zero and names the flag.
• docs/start.md states the acceptance-gating rule and how to choose the planner harness.

🤖 Generated with Claude Code