Bootstrap an isolated daemon for real-model memory evals and always finalize cleanup #4205

Priority: P1 · Type: bug

Problem

The advertised real-model memory runner cannot initialize its own fresh profile with the current CLI. It creates isolated storage, immediately calls tracedecay init, and never starts the daemon that initialization now requires.

create_eval_environment() allocates a new home and TRACEDECAY_DATA_DIR. build_fixture() creates a separate fixture directory and issues init before host/model setup. The main loop contains no daemon bootstrap. The current CLI initialization path returns code_index_scheduler_unavailable when no daemon is connectable for that profile.

Audit basis

This is a static control-flow finding at 30c23baa1967d4bb04aee3f20deb7a19f302ba61, not a claimed live-model reproduction. A normal fresh isolated profile has no separately launched daemon, so initialization fails before a model turn. No live agent or operator profile was used in the audit.

There are two connected failure-handling defects. run() calls sys.exit on failed initialization, bypassing the final report. The caller's fixture assignment occurs only after build_fixture() returns; if initialization exits, cleanup sees fixture=None and leaves the separately allocated fixture directory behind.

Proposed change

Reuse a maintained isolated-daemon lifecycle helper. Allocate the profile and fixture under one owned lifecycle, launch the selected binary's daemon with explicit isolated inputs, wait for readiness, then initialize and seed the project. Stop the owned daemon and remove all temporary state on every exit path. Convert operational failures into explicit scenario/error reports before returning a failing status. Pinning the evaluated binary to the shipping build is tracked separately.

Behavioral acceptance

• A fresh environment can initialize, seed memory, and execute a selected scenario with the current CLI.
• Initialization failure, daemon startup failure, agent failure, and cancellation each write an error report and clean all owned temporary directories/processes.
• Setup failure never leaves an orphan fixture because an assignment did not complete.
• The runner cannot fall back to the operator's daemon or storage.
• Existing model-cost opt-in remains effective; lifecycle verification must not require spending model tokens.

Audit provenance

Source evidence is pinned to 30c23baa. Relevant implementation was rechecked against master at 5daa37d1; the finding remains applicable. Verification performed and its limits are stated above.