[JaredP94]

Description

Engine status no longer holds the lifecycle lock during slow readiness/identity probes. Delayed observations cannot undo a start, stop, or port change, and a model inventory sweep that exceeds its budget returns HTTP 503 so peers retain their last-good lists.

Install verification, opaque commands, and oMLX disk-image copying now report stages without invented percentages. Real download progress remains visible; desktop (local and remote) and TUI render unmeasured stages sensibly.

Local non-loopback bind overrides require the persisted, live-read engine_allow_lan_bind opt-in (default false). Remote starts remain loopback-only. Shutdown callers join one concurrent engine sweep, and cancellation cannot skip daemon cleanup just because the start CLI had not returned yet.

References: NVIDIA#84, NVIDIA#180, NVIDIA#13, NVIDIA#81, NVIDIA/Personal-AI-Router@27b342a, and NVIDIA/Personal-AI-Router@72d5605.

Scope

The original five focused implementation commits cover:

1. Shared bounded passive status probes, lifecycle revisions, stale-result rejection, and deadline-bound HTTP model inventory.
2. Optional unmeasured install percentages, desktop projections, TUI rendering, progress tests and documentation.
3. Live LAN-bind permission in node-settings with a narrow read-only worker/broker relay. This fork does not have upstream's engine-settings subsystem, so only this permission read is relayed. Internal notifications are documented and excluded from renderer delivery.
4. One joined shutdown sweep, preserving concurrent engine stops and retries of failed actionable process/command stops.
5. Deterministic detached-daemon cancellation regression, correct cleanup after an executed CLI exits unsuccessfully, readiness-phase synchronization, and crash-report synchronization.

Three signed-off review follow-up commits:

• 2f78b8c — blocker 1: Observe lifecycle entry (opDone) while the passive readiness request is still blocked. Hold final intent persistence so even fast startup remains observable. Release the probe and await real startup separately, with unconditional fixture cleanup. The short entry assertion no longer times spawning, readiness, and persistence together.
• b5cee58 — blocker 2: Register executor StopAll cleanup immediately after construction. Replace the single 300ms daemon TCP check with the existing bounded polling helper. A fatal setup/readiness assertion can no longer leave the hour-long control CLI running.
• 8fd4080 — non-blocking overlap: Drop this branch's competing inline broker cancellation change. Merge PR Isolate cross-process tests, fix the failing broker tests, and add CI #42 first: its 6b58e5d owns the canonical helper and cancellation tests. A temporary, uncommitted merge of PR Isolate cross-process tests, fix the failing broker tests, and add CI #42 was conflict-free; broker supervisor.go exactly matched PR Isolate cross-process tests, fix the failing broker tests, and add CI #42. No PR Isolate cross-process tests, fix the failing broker tests, and add CI #42 implementation or tests are copied into this branch.

Version decision relative to v0.8.0: node-settings 3.1.0 → 3.2.0 for the additive setting; reuse existing unreleased engine-manager 1.4.0, broker 2.5.0 and TUI 1.2.1 bumps. The follow-up engine changes are test-only. No changelog/release-asset changes.

Validation

Fresh worktree from origin/fix/engine-manager-batch; its own npm ci --no-audit --no-fund, no dependency symlink. macOS arm64, isolated temporary fixtures only; no real engine installation/uninstallation or user engine/configuration paths modified.

Current follow-up verification:

• Engine-manager status/cancellation group: go test -race -count=100 -run '^(TestStatus|TestStartDoesNotWaitForPassiveProbe|TestModelsResult_CanceledCaller)' ./... — passed.
• Detached-daemon cases: go test -race -count=20 -run '^TestStopAllStopsDetachedCommandDaemon' ./... — passed, including a fresh run after all temporary fault injections were removed.
• Engine-manager full go test -race -count=1 ./... — passed (126s). Full GOMAXPROCS=2 go test -count=1 ./... — passed (125s). The unconstrained normal run failed; see the complete list below. These passes do not prove those failures pre-existing.
• Node-settings and TUI: full go test -count=1 ./... — passed. Broker standalone full go test -vet=off -count=1 ./... — passed.
• Engine-manager, node-settings and TUI: gofmt -l . clean; native go vet ./..., GOOS=linux go vet ./..., and GOOS=windows go vet ./... — passed.
• With the temporary PR Isolate cross-process tests, fix the failing broker tests, and add CI #42 merge: broker formatting, native/Linux/Windows vet, full go test -count=1 ./..., and permission-relay go test -race -count=20 -run '^TestEngineSettingReadRelay$' ./... — passed. The merge was aborted afterward; no integration commit is pushed.
• Expected standalone prerequisite: after dropping the competing cancellation edit, this branch alone has the original broker vet warning at supervisor.go:538/:523 about cancellation ownership. PR Isolate cross-process tests, fix the failing broker tests, and add CI #42 resolves it. This is not reported as a passing standalone vet gate.
• Desktop: typecheck, lint (zero warnings), unit tests (78 files / 769 tests), dead-code check, and service-contracts check — passed.
• SPDX: 1,225 checked, zero missing; git diff --check clean.

Regression/failure-path evidence for the follow-ups:

• Temporarily reacquiring the lifecycle lock in the passive probe made the revised Start test fail specifically at “Start waited for a passive readiness probe.” Restoring production code passed the 100-repeat race group.
• Temporarily disabling executed-CLI cleanup made the before-CLI-exit regression fail because the stop marker was absent. Restoring it passed the detached-daemon race stress.
• Injecting a fatal assertion after daemon startup but before explicit StopAll failed as intended; process inspection found zero surviving fixture control-CLI processes. The injection was removed before final checks.

Observed unconstrained normal-suite failures (not asserted to be pre-existing)

The failed run included loopback connection/readiness timeouts and the following test failures. No production probe was weakened and no test was removed. A subsequent full constrained normal run and a full race run passed; neither establishes baseline causality.

• TestStopReclaimsOurOrphanOnManagedPort
• TestStopDoesNotReclaimSameExternalImage
• TestStopDeclinesForeignListenerWithActionableError
• TestStopRejectsAdoptedProcessEngine
• TestStopReconcilesAdoptedProcessAfterExternalStop
• TestStopKeepsAdoptedProcessRunningWhileEndpointIsUnhealthy
• TestCommandStopFailureSucceedsWhenEndpointIsConfirmedDown
• TestCommandStopFailureRestartsHealthMonitoring
• TestCommandStopReportsStoppedOnlyAfterEndpointCloses
• TestRunOpInstallAutostart
• TestLMStudioDeleteModelRemovePath
• TestDeleteModelRestartAfterBouncesRunningEngine
• TestDeleteModelWithoutRestartAfterDoesNotBounce
• TestDeleteModelRestartFailureFailsTheAction
• TestModelLifecycle
• TestModelSurface_PinGateIsUnconditional
• TestModels
• TestModelsResult
• TestModelsResultLoaded
• TestModelsResultNoLoadedActionOmitsKey
• TestSweepLoadedSeedsThenEmitsOnChange
• TestInstallFlowReportsOnlyMeasurablePercents
• TestPullEngineErrorBecomesRPCError (local and peer-endpoint)

The review's other non-blocking observations required no production changes: revision fencing/shared-probe cancellation/shutdown joining, model deadlines, optional progress propagation, live-read fail-closed permission, and version decisions remain as reviewed. Timing failures are recorded rather than labeled pre-existing without evidence.

Limits: Linux/Windows vet is static/compile coverage, not runtime certification. Live-engine, multi-node, and the separate services/tests suite were not run. A fresh optional second-model review attempt timed out; it is not counted as approval.

Risk

• Merge ordering: PR Isolate cross-process tests, fix the failing broker tests, and add CI #42 must land first to provide broker cancellation ownership and its tests.
• Passive UI status during a lifecycle operation is its current snapshot, not a wait for completion; explicit port checks and model sweeps wait within their budgets.
• LAN permission applies to per-call local overrides, not operator-authored manifest commands. Enabling it exposes a third-party engine API without adding authentication. Revocation affects subsequent starts, not existing listeners. No desktop toggle is added; documented node-settings JSON-RPC methods are the operator surface.
• An executed control CLI can detach a daemon before exiting unsuccessfully; cleanup uses its official bounded stop command. Failed spawns do not authorize cleanup.
• Unmeasured install percent is absent; matched consumers render the stage instead of retaining the preceding download percentage.

Checklist

• I have read the Contributing Guidelines.
• Every commit is signed off, with upstream co-authorship where ported.
• Both blocking review findings have focused fixes and regression/failure-path evidence.
• The broker overlap is removed; PR Isolate cross-process tests, fix the failing broker tests, and add CI #42 integration is verified and its merge-first prerequisite is explicit.
• Relevant documentation is updated; generated service contracts check passes.
• Changed files and commit messages checked for private data and unintended generated artifacts.
• Validation commands, limitations, observed failures, and the standalone broker prerequisite are recorded accurately.
• Affected component version decisions and user-visible outcomes are documented.

[JaredP94]

Addressed both blocking findings in 2f78b8c and b5cee58:

1. The status/locking test now observes lifecycle entry while the passive probe remains blocked, then waits for real startup separately. The entire requested status/cancellation group passed -race -count=100; reintroducing the passive lifecycle lock fails the entry assertion.
2. Detached fixtures register unconditional executor cleanup immediately after construction and poll readiness within the existing bounded helper. Both daemon cases passed -race -count=20; an injected fatal before explicit shutdown left zero fixture CLI processes.

For the non-blocking overlap, 8fd4080 removes this branch's competing broker cancellation edit. PR #42 must merge first. A temporary merge was conflict-free, retained #42's helper/tests, and passed broker native/Linux/Windows vet, full tests and relay race stress. No #42 implementation was duplicated in this branch.

The PR description now records all observed unconstrained normal-suite failures by name, without claiming they are pre-existing. Full engine-manager race and GOMAXPROCS=2 normal suites passed, as did desktop gates, node-settings/TUI tests, formatting and headers. Standalone broker vet still requires #42; this prerequisite is explicit rather than reported as green. Other non-blocking production/contract/version observations needed no additional runtime changes.