fix(serve): the watcher read three things out of the recipe it parsed at startup

[marevol]

What this fixes

_RecipeWatchState.recipe was the body parsed when the process started. Nothing
refreshed it — the only two assignments to it sat inside YAML-error-recovery
branches — so for a recipe that always parses cleanly it stayed the startup body
for the life of the process.

#417 fixed one reader of that field (the recipe-drift warning) by adding a
second field, latest_recipe, refreshed on every rescan. Three more readers
were left on the startup body. This PR fixes all three by making the one field
mean what its name says: the recipe as it is on disk now.

1. item_metadata — a retrain silently produced a mixed response

_build_entry joins the response against the table the recipe names. Repoint
item_metadata.path and retrain, and the new artifact hot-swaps in while the
join still reads the table the startup body named. Nothing warns, because the
artifact and the recipe on disk now legitimately agree — the drift warning is
silent by design in exactly this case.

Which body it reads now, and why that one. The watcher cannot reconstruct
the body an artifact was trained from; the header carries only that body's
recipe_hash, not the body. So "the trained-from body" is not an option that
exists — the startup body was only ever a proxy for it, and it stops being one
at the first hot-swap: after an edit and a retrain, the startup body is neither
the body on disk nor the body the new artifact was trained from. The body on
disk is the only one the watcher can name, and it is already the one app.py
uses on the startup load path, so a hot-swap and a restart now build the same
entry from the same bytes. That parity is the same reason the recipe-hash gate
was mirrored onto both load paths in #270.

When the two bodies do differ — an edit with no retrain — the drift warning is
already firing, and this PR corrects that warning's docstring, which claimed the
response is "of a piece" while it fires. It is not: what comes out of the
artifact is the older recipe, while item_metadata is read from the recipe on
disk at load time. The docstring now says which half is which.

2. output.path — an edit was ignored until restart

state.artifact_path was taken from recipe.output.path once, at discovery. An
edited output path was ignored for the life of the process, and silently: the
drift warning is emitted from the load path, and no load ever happens at the new
location, so nothing anywhere says the watcher is polling a file the recipe no
longer names.

The rescan now follows it. A re-point drops everything the watcher remembers
about the previous file — change marker, payload digest, .sha256 sidecar
contents, load backoff, repeat-suppression signatures — because none of it says
anything about the new path. Dropping last_sha256 matters in particular: a new
path holding byte-identical content would otherwise take the "unchanged bytes"
short-circuit and never rebuild the entry, leaving the served entry naming the
old path. One INFO line, recipe_output_path_changed, carries previous_path
and path.

A path that does not exist yet — the ordinary case, an operator who edited
the recipe and has not retrained — degrades rather than evicting. The model in
the registry keeps serving, the next poll records "artifact missing or
unreadable" against the entry and /v1/health/details goes degraded. That is
the same M-2 contract every other missing artifact already gets, and there is a
test for it.

3. The .sha256 sidecar suppression latch could never be released

Three consecutive unreadable sidecar reads set sidecar_unsupported so a broken
sidecar cannot drive a full reload on every tick (m7). The documented escape —
clear it when the recipe changes (C4) — was reached through:

getattr(state.recipe, "_yaml_path", None) or getattr(state.recipe, "yaml_path", None)

Recipe is a pydantic model with extra="forbid" and no private attributes, so
it has never carried either name. That chain has answered None for every
recipe that has ever existed, the guard it feeds (yaml_mtime is None or ...)
always declined, and the recovery has never run once in a live server. The latch
was permanent, and the sidecar is a real backstop: it is what catches a change
the (mtime, size) marker cannot discriminate.

Chosen fix: make the recovery work, rather than delete it. Deleting the dead
chain would have been a pure no-op — the latch is already permanent in practice
— and would have made "no recovery at all" the design. Instead the clearing
moves to _scan_recipes_dir, where the watcher already decides that the YAML
changed (_yaml_mtime_cache miss). That is a signal the watcher genuinely has,
it removes a second, independently-drifting mtime comparison, and it is now
load-bearing for a reason the old placement did not have: with fix 2 above, a
recipe edit can move output.path, and the sidecar path is
artifact_path + ".sha256", so an edit really can move the sidecar out from
under a latch set on the old one.

The three dead getattr chains and the sidecar_unsupported_at_mtime field
they compared against are removed.

The test for this recovery passed, and proved nothing.
test_sidecar_unsupported_clears_on_yaml_mtime_change built the state around a
MagicMock with recipe._yaml_path = yaml_path — the only kind of object in
existence for which the code worked. It is replaced by
test_sidecar_unsupported_is_never_cleared_by_the_sidecar_check_itself, which
uses a real Recipe and pins the other half of the split, so a reintroduced
mtime probe in _check_sidecar_changed fails there.

Pre-existing since 2.0.0 — evidence

None of these is a regression from #415, #417, #418 or #362.

defect	evidence
item_metadata from the startup body	_build_entry(name, state.recipe, ...) and _RecipeWatchState come from 8395a811 (2026-05-14, the 2.0 rewrite); git merge-base --is-ancestor 8395a811 v2.0.0 succeeds
frozen output.path	same commit introduced _RecipeWatchState.artifact_path
dead sidecar_unsupported recovery	git show v2.0.0:src/recotem/serving/watcher.py | grep -c '_yaml_path", None' → 3, the same count as on main; the chains are at lines 1245 / 1277 / 1338 of the 2.0.0 file, and git show v2.0.0:src/recotem/recipe/models.py has no yaml_path

A note on the dating, because the obvious command misleads: git log -S '_yaml_path' -- src/recotem/serving/watcher.py returns f15ebbd6 (2026-09-03, #167), which is not in v2.0.0 and reads like "new in 2.1.0". -S reports commits that changed the count of the string, and #167's only _yaml_path line is a docstring mention of the unrelated _yaml_path_to_name map. The getattr chains themselves are untouched since 2.0.0.

Tests

Each behaviour has a test that fails on unmodified main on its assertion,
not on an import or a signature error. New file
tests/unit/test_watcher_live_recipe_body.py.

On main (696d55c), before the fix — 4 failed, 1 passed:

E  AssertionError: the artifact that just swapped in was trained FROM the edited
   recipe, yet the metadata join still uses the table the STARTUP body named.
   The response is the new model on the old table. Titles: ['OLD_TITLE']
E  assert ['OLD_TITLE'] == ['NEW_TITLE']

E  AssertionError: output.path was repointed under a running watcher and the
   artifact at the new path never loaded: the watcher is still polling
   '.../old.recotem', the path captured at discovery.

E  AssertionError: output.path now names a file that does not exist and nothing
   was recorded against the entry: the watcher is still polling the old path
   ('.../old.recotem') and reporting success for it.

E  AssertionError: the recipe YAML was re-parsed and the suppression was not
   cleared: the documented recovery reads Recipe._yaml_path, which does not
   exist, so the latch is permanent and the sidecar backstop is off until
   restart.
E  assert True is False

The one that passes on main is the control,
test_item_metadata_is_not_reloaded_without_a_hot_swap: an edit with no
retrain must leave the old model and the old table in place. Without it,
"the join followed the edit" could as easily mean "the join re-reads every
tick", which would be a different behaviour.

After the fix: 5 passed.

Each fix is load-bearing — single-call-site mutations

Each mutation rewrites one expression and leaves the statement in place (never a
deletion, so a presence check cannot score as a behaviour check):

mutation	change	result
M1	existing_state.recipe = recipe → assign the field back to itself	test_item_metadata_follows_the_edited_recipe_after_a_retrain and both #417 drift tests fail — one site now carries both fixes
M2	_repoint_artifact(..., recipe.output.path) → pass existing_state.artifact_path, so the call still runs and is a no-op	both output.path tests fail
M3	existing_state.sidecar_unsupported = False → assign the field back to itself	the sidecar test fails

M1 failing both #417 tests is the check that removing latest_recipe did not
quietly drop #417's fix.

Suite

main baseline: 3157 passed, 1 skipped, 4 deselected.
This branch: 3162 passed, 1 skipped, 4 deselected (+5 = the new file).
ruff check src tests and ruff format --check src tests clean.
uv run bash tests/e2e/run.sh passes.

Concurrent watcher work

fix/watcher-pointer-content-marker (the pointer-content change marker) is in
flight against the same file. The two touch disjoint hunks, so this opens
against main rather than stacking. Merged together locally the tree is clean
and green: 3165 passed, 1 skipped, 4 deselected, ruff clean — no conflict
and no cross-failure. That branch narrows, but does not remove, the reason the
sidecar backstop matters: it folds a digest into the marker only for
pointer-sized files, so versioning: always_overwrite (a real artifact at the
watched path) still relies on (mtime, size), and the sidecar is still its only
backstop.

Deliberately not changed

• An item_metadata edit alone does not trigger a reload. Metadata is read
  where the model is built, so an edit with no retrain leaves the old model and
  the old table — which is coherent, and is what the drift warning describes.
  Making a metadata edit its own hot-swap trigger is a new trigger, not this fix.
• ModelEntry.artifact_path is not updated eagerly on a re-point. That field
  documents "the path last successfully loaded", which stays true; the
  recipe_output_path_changed log line ties the old and new paths together.
• _stat_marker is untouched — that is the other branch's lane.

Docs follow-up (separate repo)

https://recotem.org/2.1/docs/operations.html needs two edits that are not in
this repo:

• the "Watcher and registry semantics" bullet list should say that an edited
  output.path is now followed at the next rescan, and that item_metadata is
  re-read from the recipe on disk at the next hot-swap;
• the watcher event table should gain recipe_output_path_changed (INFO).

Also pre-existing and unrelated to this PR, but worth correcting while that page
is open: it states that during a drift warning "/v1/recipes/{name} reports the
current recipe's algorithms, metric and cutoff". It reports the
artifact's — routes.py never reads a Recipe object; every field on that
endpoint comes from entry.header.