One failed plugin must not take down the whole plugin system (host/src/services/plugins.ts) #672

Problem

The host's plugin loader already isolates the common failure mode well: a PluginBootstrapError from loadRemoteWithRetry is caught per plugin (plugins.ts:679) — that plugin is logged, recorded in status.failures, and given a failedPluginsClientEntry so dependents fail loudly instead of crashing.

But any other error class inside the load loop escapes the catchTag("PluginBootstrapError") and hits the outer Effect.catch (plugins.ts:759) → unavailableResult(...) — which discards all plugins, including ones that already loaded successfully (loadedPlugins is dropped; plugins: {} is returned). Auth and API go down, every /api/* route 503s, and tenant bindings cannot be resolved.

Error classes that currently escape per-plugin isolation:

1. Secrets resolution — resolveSecretsWithDatabaseFallback (plugins.ts:447, inside loadPluginEntryEffect but before the pipe that maps errors to PluginBootstrapError). A ConfigError on any plugin's secret key nukes every plugin.
2. Auth base variables — buildAuthBaseVariables throws (plugins.ts:371, "No reachable origin" in production); it is called in the loop at plugins.ts:666 outside any per-plugin catch.
3. Runtime creation — the Effect.tryPromise block (plugins.ts:578–621) surfaces as PluginError in the outer catch: a single unresolvable shared dep in registerAppSharedDeps (plugins.ts:236–244 throws) drops the entire plugin system.
4. DAG / misc — any other throw in the loop body (DAG lookups, logging epilogue) has the same blast radius.

The host process does technically still boot, but as a husk: degraded health, 503 API, broken tenant SSR — and a red deploy smoke can roll the deploy back.

Scope

• Wrap each loop iteration's full body (secrets resolution, base variables, load, singleton bookkeeping) in per-plugin failure isolation: any error → PluginFailureInfo in status.failures + failedPluginsClientEntry for dependents; already-loaded plugins are kept.
• Decide what legitimately stays degraded-all: creation of the plugin runtime itself is arguably global; a boot-wide misconfiguration (e.g. genuinely unreachable reachable-origin) may still warrant a loud failure — make that decision explicit and documented.
• Keep the existing PluginBootstrapError path (classification, retryable hints, DB-url diagnostics) working for the new catch as well.

Acceptance criteria

• One plugin failing for any error reason leaves the host serving with every other plugin live (auth, api, and sibling plugins).
• /health reports the per-plugin failure in status.failures and keeps the rest ready.
• Host boot and the deploy smoke are unaffected by a single plugin's failure.
• A regression test pins at least one of the currently-escaping error classes (secrets ConfigError; auth-origin throw) as remaining per-plugin rather than degraded-all.