fix(mcp): refresh floating registry defaults at startup

changelog.d/595.fixed.md

@@ -0,0 +1,3 @@
+**MCP registry defaults**: server startup now checks for a newer default-branch
+commit even with a warm cache, while retaining the recorded SHA as a fallback
+when ref resolution fails.

docs/mcp-server.md

@@ -112,6 +112,14 @@ narrows which *registries* are even considered before the ladder runs.
 `--workflow-dir <path>` (repeatable) adds workflows from a local directory
 alongside, or instead of, any registry.
 
+At every server start, configured GitHub registries check the current default
+branch and pin the catalogue to its commit SHA. Content already cached at that
+SHA is reused; if default-ref resolution fails, the server falls back to the
+last recorded SHA (subject to cache source validation). A running catalogue
+stays frozen at its startup SHA and tool list: restart the server to adopt
+registry changes. `conductor doctor mcp` uses an offline, cache-only snapshot,
+not the online default-ref check performed by server startup.
+
 The startup summary the server prints to stderr (see
 [Startup Summary](#startup-summary)) names every workflow it *does*
 expose, its source registry, and its pinned identity — a workflow

docs/projects/mcp-server/conductor-mcp.design.md

@@ -322,8 +322,9 @@ independent of building a server — see DD0 for the empirical result.
    a registry is large.
 8. **G8** — Every exposed workflow is pinned to an immutable identity at
    session start, and drift is reported rather than silently applied.
-9. **G9** — Server startup does not depend on the network when the registry
-   cache is warm.
+9. **G9** — A warm registry cache supports an explicitly cache-only build
+   without network access and a fallback for failed online ref resolution;
+   normal server startup checks floating defaults online (issue #595).
 10. **G10** — Every governance control a workflow author declared still applies
     when the caller is a model: gates gate, budgets cap, schemas validate. The
     server adds a caller, not an exemption.
@@ -377,7 +378,7 @@ independent of building a server — see DD0 for the empirical result.
 
 | ID | Requirement |
 |---|---|
-| NFR1 | Cold-start to first `tools/list` response ≤ 2s with a warm registry cache, with **zero network I/O**. |
+| NFR1 | Cold-start to first `tools/list` response ≤ 2s with a warm registry cache in an explicitly cache-only build, with **zero network I/O**. **Issue #595 amendment:** ordinary online server startup re-resolves each GitHub registry's floating default ref even with a warm cache; this network resolution is outside the schema-resolution deadline. On resolution failure, a recorded SHA provides a cached fallback. |
 | NFR2 | A workflow whose schema cannot be resolved is exposed with a permissive schema and a description saying so — never silently dropped. |
 | NFR3 | No tool **accepts** a filesystem path, URL, or registry source as a parameter. (Returning a path *outward*, as DD12's `resource_link`s do, is the opposite direction and is not constrained by this rule.) |
 | NFR4 | Any YAML-authored text reaching a tool `description` is sanitized and length-capped. |
@@ -479,7 +480,10 @@ achievable:
 2. **SHA-keyed parse cache** — a parsed, normalized tool definition stored
    beside the existing mirror under
    `$CONDUCTOR_HOME/cache/registries/<registry>/_meta/<sha[:12]>/`. A SHA-keyed
-   entry is immutable, so a warm cache makes startup a no-network operation.
+   entry is immutable, so a warm cache avoids re-parsing or fetching at the
+   selected SHA. Online startup still checks the default ref first; explicitly
+   cache-only catalogue builds use the recorded pointer with no network I/O
+   (issue #595).
    This reuses `registry/cache.py`'s existing layout, sentinel convention, and
    `CACHE_LAYOUT_VERSION` invalidation rather than inventing a second cache.
 3. **Fetch and parse**, under a startup deadline. On failure — including the
@@ -1449,8 +1453,11 @@ terminal record alongside its event log, DD13), `registry/index.py` and
   them.
 
 **Performance.** Startup is the sensitive path (hosts respawn stdio servers
-aggressively). Warm cache: local reads only, no network (NFR1). Cold cache: one
-index fetch plus one fetch per unresolved workflow, under a deadline, with
+aggressively). A warm cache permits zero-network explicitly cache-only builds,
+and supplies a fallback when online default-ref resolution fails; ordinary
+online startup still resolves the default branch before reusing any SHA-keyed
+index or schema cache (issue #595). Cold cache: one index fetch plus one fetch
+per unresolved workflow, with a deadline for schema resolution and
 degraded-schema fallback. Per-invocation cost is one process fork plus the
 existing three-stage health gate — the same cost `conductor run --web-bg`
 already pays. `conductor_run_status` on a live run is a bounded tail read of

docs/projects/mcp-server/conductor-mcp.plan.md

@@ -73,7 +73,7 @@ and test churn are accepted. Consequences propagated:
 **R2 — A floating registry ref resolves offline through a
 `_meta/_refs/<ref-slug>.json` pointer in `registry/cache.py`.** *(affects E5)*
 
-NFR1 ("zero network I/O" to first `tools/list` on a warm cache) and DD6 (every
+NFR1 ("zero network I/O" in explicitly cache-only builds on a warm cache) and DD6 (every
 exposed workflow pinned to an immutable identity) were **unsatisfiable together**
 against the current code. Verified: `registry/cache.py` keys everything by SHA
 (`_meta/<sha[:12]>/`), and the only way to learn what a floating ref (`latest`,
@@ -88,8 +88,11 @@ fallback has a checkout to choose (`AGENTS.md`, *Git-backed plugin sources*:
 "without a record of what a floating ref last meant, the offline fallback has no
 checkout to choose"). The registry cache gains the same shape, written on every
 successful online resolution and read when a caller declares the network
-off-limits. Startup is then genuinely zero-network on a warm cache, and DD6's
-interval drift re-check is what refreshes the pointer.
+off-limits. **Issue #595 amendment:** explicitly cache-only builds are
+zero-network on a warm cache; ordinary online startup re-resolves the default
+branch and refreshes the pointer on success. Resolution failure falls back to
+the recorded SHA. Interval drift checks only report drift; they do not refresh
+the pointer or mutate an existing catalogue.
 
 ---
 
@@ -190,7 +193,8 @@ about run state: an offline-answerable registry, and a typed `mcp:` block.
 **Exit criteria**
 - [ ] A warm registry cache answers "what is this workflow's `input:` and
       `mcp:` block, at what SHA" with **zero** network calls, proven by a test
-      that patches `registry/github.py` to raise on any call (NFR1, G9, R2).
+      that patches `registry/github.py` to raise on any call in an explicitly
+      cache-only build (NFR1, G9, R2); online startup checks the ref first.
 - [ ] A floating ref resolves offline through the `_meta/_refs/` pointer, and
       a cold cache still resolves online exactly as today (R2).
 - [ ] `workflow.mcp:` parses, validates, and is reported by `conductor
@@ -205,7 +209,8 @@ No protocol, no process launching — pure functions over the registry.
 
 **Exit criteria**
 - [ ] `build_catalogue(...)` returns an immutable catalogue from a fixture
-      registry in under 2s with the network patched to raise (NFR1).
+      registry in under 2s with the network patched to raise and
+      `allow_network=False` (NFR1); default online startup re-resolves refs.
 - [ ] The four-rung exposure ladder (`--deny` > `--allow` > `mcp.expose` >
       default-on) is exercised in every ordering that distinguishes the rungs
       (FR2, DD4).
@@ -511,9 +516,9 @@ single-forward-pass constraint is untouched.
 
 ### E5 — Registry index fields, offline ref pointer, and parse cache (P4, NFR1, G9, R2) — DONE (completed 2026-08-22)
 
-**Goal.** Make the catalogue answerable from a warm cache with zero network
-I/O — the three-tier schema ladder's first two tiers (*Key Components → 1*) —
-and close the NFR1/DD6 conflict R2 resolves.
+**Goal.** Make an explicitly cache-only catalogue answerable from a warm cache
+with zero network I/O — the three-tier schema ladder's first two tiers
+(*Key Components → 1*) — and close the NFR1/DD6 conflict R2 resolves.
 
 **Prerequisites.** E6 (for `McpConfig`, which E5-T1 imports); otherwise
 independent of E1–E4.
@@ -536,7 +541,7 @@ API, so a floating ref cannot be resolved offline — the gap R2 closes.
 | E5-T5 | TEST | Index round-trip with and without the new fields; an old index loads unchanged; a new index loads on a build that ignores the fields. Ref pointer write/read, slug safety, and atomicity. Parse-cache hit avoids re-parse; `CACHE_LAYOUT_VERSION` bump invalidates. **The load-bearing test:** with every function in `registry/github.py` patched to raise, a warm cache still resolves a GitHub registry's workflows to schemas and SHAs (NFR1, G9, R2). | `tests/test_registry/test_index.py`, `tests/test_registry/test_cache.py` | DONE |
 
 **Acceptance criteria**
-- [x] A warm cache answers "input schema + `mcp:` block + pinned SHA" for every workflow with the network patched to raise, including for a floating ref.
+- [x] An explicitly cache-only build answers "input schema + `mcp:` block + pinned SHA" from a warm cache with the network patched to raise, including for a floating ref. Ordinary online startup re-resolves that ref first (issue #595).
 - [x] Existing registries and indexes work unchanged.
 - [x] A cold cache still resolves online exactly as today, and writes the pointer as a side effect.
 
@@ -603,11 +608,12 @@ process launching.
 | E7-T6 | IMPL | `build_catalogue(...)`: enumerate → filter by the four-rung ladder (`--deny` > `--allow` > `mcp.expose` > default-on, with `--registry` selecting the candidate set one level above it) → resolve schemas through the three-tier ladder under a startup deadline → pin → sanitize → qualify collisions → decide direct-tools vs discovery. Return an immutable `Catalogue`. Reject a workflow whose `input:` collides with `_wait_seconds`, logging the reason per FR10. On any parse failure — including the `${VAR}`-missing and parent-directory `!file` cases from P4 — expose with `{"type": "object"}` and an explanatory description (NFR2). | `src/conductor/mcp/serve/catalogue.py` | DONE |
 | E7-T7 | TEST | Naming and sanitizing: slug charset and length, prefixing, both-sides collision qualification, control-character stripping, length cap. | `tests/test_mcp/test_serve_naming.py` | DONE |
 | E7-T8 | TEST | Tool generation: all five input types; `required`/`default`/`description` survive; `_wait_seconds` present and documented; no `outputSchema`; a workflow declaring `_wait_seconds` is rejected. | `tests/test_mcp/test_serve_toolgen.py` | DONE |
-| E7-T9 | TEST | Catalogue: every ladder ordering that distinguishes a rung (`--deny` beats `--allow`; `--allow` overrides `mcp.expose: false`; `--registry` excludes non-candidates entirely); the schema ladder's three tiers; NFR1 (network patched to raise, warm cache, under 2s); NFR2 for both parse-failure modes; the discovery threshold decision. | `tests/test_mcp/test_serve_catalogue.py` | DONE |
+| E7-T9 | TEST | Catalogue: every ladder ordering that distinguishes a rung (`--deny` beats `--allow`; `--allow` overrides `mcp.expose: false`; `--registry` excludes non-candidates entirely); the schema ladder's three tiers; NFR1 (network patched to raise, warm cache, `allow_network=False`, under 2s); NFR2 for both parse-failure modes; the discovery threshold decision. Online builds re-resolve floating defaults (issue #595). | `tests/test_mcp/test_serve_catalogue.py` | DONE |
 | E7-T10 | TEST | Pinning: SHA for GitHub, content hash for path, drift detected and reported without the catalogue changing. | `tests/test_mcp/test_serve_pinning.py` | DONE |
 
 **Acceptance criteria**
-- [x] A frozen catalogue is built from a fixture registry with zero network I/O.
+- [x] A frozen catalogue is built from a fixture registry with zero network I/O
+      in cache-only mode; ordinary online startup re-resolves floating refs.
 - [x] The exposure ladder behaves exactly as FR2 specifies in every distinguishing case.
 - [x] No workflow is ever silently dropped for an environmental reason.
 - [x] Two registries publishing one slug yield two qualified names.

src/conductor/mcp/serve/catalogue.py

@@ -11,9 +11,9 @@
 1. Enumerate the registries selected by ``--registry`` (one level above
    the exposure ladder — a registry outside this set is never a
    candidate, full stop).
-2. For each registry, load its index — for a GitHub registry, through the
-   E5 offline-capable ref-pointer + SHA-keyed cache path, so a warm cache
-   never touches the network (NFR1).
+2. For each registry, load its index — online GitHub builds resolve the
+   current default ref before consulting the SHA-keyed cache; explicitly
+   cache-only builds use the recorded ref pointer without network I/O.
 3. Filter each workflow through the four-rung exposure ladder (``--deny``
    > ``--allow`` > ``mcp.expose`` > default-on, DD4).
 4. Resolve each surviving workflow's schema through the three-tier ladder
@@ -72,9 +72,9 @@
 
 logger = logging.getLogger(__name__)
 
-# NFR1: cold-start to first `tools/list` response <= 2s with a warm cache.
 # This bounds tier-3 (fetch-and-parse) attempts specifically -- a tier-1 or
-# tier-2 hit never reaches this check. A workflow whose tier-3 resolution
+# tier-2 hit never reaches this check. Online default-ref resolution is not
+# covered by this deadline. A workflow whose tier-3 resolution
 # would blow the deadline degrades (NFR2) rather than stalling startup.
 DEFAULT_SCHEMA_RESOLUTION_DEADLINE_SECONDS = 2.0
 
@@ -276,11 +276,12 @@ def build_catalogue(
             ``registry.config.load_config()`` (``~/.conductor/registries.toml``);
             overridable so tests and embedders can supply a fixture set
             without touching disk-backed configuration.
-        allow_network: When ``False``, every registry resolution is
-            answered entirely from the local cache — no GitHub API calls
-            — matching NFR1. A cache miss under this constraint degrades
-            that workflow (or, for a whole unreachable registry, skips
-            it) rather than raising.
+        allow_network: When ``True`` (default), GitHub registries resolve
+            their current default branch on every build, then reuse the
+            SHA-keyed index cache when valid. A resolution failure falls
+            back to the recorded pointer. When ``False``, resolution is
+            cache-only with no GitHub API calls; a missing index skips the
+            registry rather than aborting the build.
         schema_resolution_deadline: Wall-clock seconds, from this call's
             start, budgeted for tier-3 (fetch-and-parse) schema
             resolution. Exceeding it degrades the remaining unresolved
@@ -550,8 +551,8 @@ def _validated_cached_index(
 def _resolve_registry_index(
     registry_name: str, entry: RegistryEntry, *, allow_network: bool
 ) -> tuple[RegistryIndex, str | None]:
-    """Load a registry's index, honoring the E5 offline/warm-cache path
-    for GitHub registries. Returns ``(index, sha)`` — ``sha`` is ``None``
+    """Load a registry's index, resolving online or using the E5 cache-only
+    path for GitHub registries. Returns ``(index, sha)`` — ``sha`` is ``None``
     for path registries, which have no ref/SHA concept.
 
     Reuses ``registry/cache.py``'s E5 primitives directly (its own module
@@ -561,25 +562,26 @@ def _resolve_registry_index(
     if entry.type == RegistryType.path:
         return load_index(entry), None
 
-    # NFR1: consult the warm cache first -- network is used only on a cold
-    # miss (no recorded ref pointer yet, or the index isn't cached for the
-    # pointer's SHA), even when allow_network=True. Resolving the ref
-    # online unconditionally would touch the network on every build, warm
-    # cache or not.
+    # An online build checks the floating default on every start. Only a
+    # resolution error falls back to the last recorded SHA; an index fetch
+    # failure for a newly resolved SHA must not publish an older catalogue.
     if allow_network:
+        using_fallback = False
         try:
-            offline_sha = registry_cache._resolve_sha_offline(registry_name, None)
+            resolved_ref = resolve_ref(entry, None)
+            sha = materialize_to_sha(entry, resolved_ref)
         except RegistryError:
-            offline_sha = None
-        if offline_sha is not None:
-            meta = registry_cache._meta_dir(registry_name, offline_sha)
-            cached_index = _validated_cached_index(meta, registry_name, entry, offline_sha)
-            if cached_index is not None:
-                return cached_index, offline_sha
-
-        resolved_ref = resolve_ref(entry, None)
-        sha = materialize_to_sha(entry, resolved_ref)
-        registry_cache._write_ref_pointer(registry_name, None, sha)
+            sha = registry_cache._read_ref_pointer(registry_name, None)
+            if sha is None:
+                raise
+            using_fallback = True
+            logger.warning(
+                "Could not resolve default ref for registry %r; falling back to recorded SHA %s",
+                registry_name,
+                sha,
+            )
+        else:
+            registry_cache._write_ref_pointer(registry_name, None, sha)
     else:
         sha = registry_cache._resolve_sha_offline(registry_name, None)
 
@@ -588,6 +590,14 @@ def _resolve_registry_index(
     if cached_index is not None:
         return cached_index, sha
 
+    if allow_network and using_fallback:
+        metadata = registry_cache._read_source_metadata(meta)
+        if metadata is not None and not registry_cache._metadata_matches(metadata, entry, sha):
+            raise RegistryError(
+                f"Recorded SHA {sha} for registry {registry_name!r} has cached metadata "
+                "from a different source; refusing to use that index."
+            )
+
     if not allow_network:
         raise RegistryError(
             f"Registry {registry_name!r} index at {sha[:12]} is not available in the "