Repository navigation
Isolate cross-process tests, fix the failing broker tests, and add CI - #42
Merged
Merged
Conversation
Port NVIDIA/Personal-AI-Router@937cf6c to this fork’s standalone proxies. Route every runtime command through a per-test app-data environment, retain explicit fixture roots for restarts and remote engine manifests, and guard against bypassing the command factory or replacing its environment. Keep Go build caches outside disposable application state. Own the manual bridge tests’ node-info fixture instead of inheriting a running PAIR instance’s cluster identity. Check wildcard and IPv6 listeners before tests require fixed worker ports, including the workload and cluster restart cases, so occupied developer ports are reported as skips rather than misleading product failures. Co-authored-by: Som <somtri@iastate.edu> Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Go vet cannot prove the lifetime of the cancellation function assigned inside the watch loop. Extract probe launch into a helper that returns its context, cancellation function, and buffered result together; watch retains the same completion, deadline, and exit cleanup. Cover cancellation on stop, requested restart, and worker exit. Keep the existing unreleased broker 2.5.0 bump relative to v0.8.0, which already covers this internal refactor. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Adapt NVIDIA/Personal-AI-Router@b4a0cc5 for this fork without release automation. Run component formatting, vet, and tests on macOS arm64 and Ubuntu; run the isolated cross-process suite on macOS only. Check the desktop contracts, attribution, types, lint, unit tests, dead code, and repository SPDX headers. Pin action tags and toolchains, cache npm and Go modules, use read-only checkout permissions, and normalize three existing test files so the formatting gate starts clean. Co-authored-by: Noah Tervalon <ntervalon@nvidia.com> Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Bind install claims to device, inode, and a generation timestamp: birth time on Darwin and ctime on Linux. A replaced Linux bundle must not inherit a claim when its inode is reused. Keep unchanged identities stable and preserve Darwin staging renames. Add deterministic reused-inode regressions including timestamp nanoseconds, and document conservative invalidation of prior claims. The existing unreleased engine-manager 1.4.0 bump over v0.8.0 already covers this fix. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Install the pinned Go toolchain in the desktop job and download declared modules, including Windows-only dependencies, before the offline license check. Verify the preparation does not change service manifests. The first run never acquired either macos-latest runner: both jobs had zero executed steps and runner_id 0. Pin the supported macos-15 arm64 image instead of raising test timeouts for an infrastructure queue failure. Emit verbose Go test progress for future slow-test diagnosis. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
The first acquired arm64 runner completed all tests in eight minutes, but real mDNS sends failed with no route to host on en0. Add an explicit on-link IPv4 multicast route on the disposable runner before running the complete isolated cross-process suite. Keep every discovery and workload assertion enabled. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
8 tasks done
Stream verbose Go JSON events to both the job log and an always-uploaded artifact. Keep the Go deadline below the job deadline so a stuck suite reports its active test and goroutines rather than losing all evidence to job cancellation. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
…ulticast gap The macOS cross-process job failed 12 discovery tests on every run, and not because of a hang: with the diagnostics in place the suite finished in eight minutes. Hosted macOS runners cannot send IPv4 multicast at all; every mDNS send to 224.0.0.251 fails with "no route to host", even with an explicit 224.0.0.0/4 route, so no node is ever discovered. Run the cross-process suite on ubuntu-latest as well, where multicast works, so discovery is exercised in CI. Tests that need real mDNS now probe whether this host can multicast and skip only when it cannot, naming the environment; on any machine that can, including a developer Mac, they run as before. Drop the route step, which did not help. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Review follow-up. The multicast probe tried only the default route, but discovery sends through every eligible interface, so a multicast-capable LAN without a default route would have skipped the discovery tests. The probe now sends out of each up, multicast-capable IPv4 interface, runs the tests if any succeeds, skips only when every one fails for lack of a multicast route, and fails the test on any other socket error. The decision has its own table test. Also corrects a stale workflow comment. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
Review follow-up. The probe stopped at the first interface that sent, so an unexpected error on another went unseen. Every result is now inspected: unexpected errors are logged even when another interface sent (the test still runs, as discovery tolerates per-interface failures), and fail the test when no interface could send. Signed-off-by: Jared Ping <13906915+JaredP94@users.noreply.github.com>
JaredP94
force-pushed
the
test/isolation-and-ci
branch
from
November 3, 2026 05:41
81925f8 to
cb6ce63
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Port the app-data isolation in NVIDIA/Personal-AI-Router@937cf6c to this fork's separate proxies, diagnose the five reported cross-process failures, and add fork CI adapted from NVIDIA/Personal-AI-Router@b4a0cc5.
Every runtime launch in
services/testsnow goes through an isolated command factory.HOME,USERPROFILE,XDG_CONFIG_HOME,APPDATA, andLOCALAPPDATApoint into disposable test fixtures and are inherited by supervised workers. Explicit fixture roots preserve restart state and remote-engine manifests; Go builds retain their normal caches. An AST guard rejects bypass constructors and environment replacement, a unit test checks all five roots, and a real LM Studio child verifies port persistence under its private root.Root causes of the five reported failures
All five were reproduced after isolating the process environment. They are test/environment interference, not demonstrated product defects:
TestBrokerBridgesManualNodeIntoProxy: the fixture mocked Ollama but not node-info. A running local PAIR supplied a real cluster principal/UUID; the manual-node worker correctly avoided anonymous engine probes for that clustered identity, so the expected synthetic candidate never appeared. Supply a controlled, unclustered node-info fixture as well. The existing learned-UUID test still covers the UUID-keyed path.TestBrokerBridgesManualNodeIntoLMStudioProxy: the same uncontrolled node-info dependency affected the LM Studio fixture. Supply its own unclustered node-info fixture too.TestBrokerAcceptsClientErrorsReport: the IPv4-loopback-only port preflight missed an existing IPv6/wildcard listener. The supervised errors worker then exited on its wildcard bind, leaving no datastore to emiterrors:update. Probe wildcard and both loopback families, ignoring unsupported IPv6, and skip when the required port is occupied.TestBrokerAcceptsClientErrorsReportRequest: the same errors-worker bind failure. The broker's immediate acknowledgement explainsack=true, update=false; it is receipt, not confirmation of datastore mutation. The corrected shared preflight prevents the misleading timeout.TestBrokerCrashSurfacingAndRestart: the same unavailable errors datastore prevented the proxy-crash report from surfacing, independently of proxy supervision. Use the corrected preflight. No product error-pipeline change is warranted by this reproduction.The full isolated run also exposed missing occupied-port checks in workload and cluster-restart cases. Add the same explicit skips there rather than connecting test peers to an unrelated running service. A skip remains untested coverage, not a successful integration assertion.
Desktop intermittent failure investigation
Ran
npm run -s test:unit30 consecutive times, with separate captured logs and exit statuses: 30/30 passed, each with 78 files / 767 tests. The reported intermittent failure was not reproduced; no speculative desktop change was made.Scope
main: Go formatting/vet/tests for all service modules on macOS arm64 and Ubuntu; isolated cross-process tests on macOS only; all requested desktop gates and SPDX checks. Exact action-version tags, npm/Go caches, hosted runners, read-only permissions, and no release automation.2.5.0bump versus2.4.1atv0.8.0; it already covers the internal refactor under the fork's unreleased-bump policy. Other binary implementations are unchanged.Validation
Local environment: macOS arm64, Go 1.27.3, Node 26.7.0. CI pins Go 1.26.7 and Node 25.5.0.
go vet ./...andgo test -count=1 -timeout=15m -v ./...in all 17 services modules: passed on the final source (untouched modules from the first run; all touched modules re-run).services/testssuite (go test -count=1 -timeout=10m -v ./...): passed in 286.4 seconds, 18 occupied-port-dependent tests skipped. In particular, the two manual bridge tests ran and passed; the three reported error-pipeline cases skipped because the developer's running PAIR owns port 14319. Their end-to-end success still needs the clean hosted runner; no running developer service was stopped.GOOS=linux go vet ./...andGOOS=windows go vet ./...inservices/tests,nvpair-node-info,nvpair-ui-broker, andshared: passed. Native Linux/Windows tests were not run locally.-count=3, passed. Re-ran test-module vet for native/Linux/Windows after the final guard edits.USERPROFILEfailed the environment regression; the IPv6-listener regression failed against the original preflight; both manual bridge tests failed before their fixture fixes and passed after; removing exit cancellation failed all three cancellation regression cases, then passed when restored. The original broker vet failure is also recorded.typecheck,lint(zero warnings),test:unit,dead-code:check,service-contracts:check, andlicenses:check: passed; plus the 30-run unit-suite investigation above.gofmt -l services: empty.node scripts/spdx-headers.mjs: 0 missing/review/unclassified.git diff --check: clean.actionlintv1.7.7: passed. Verified referenced action tags exist. Hosted workflow execution remains to be verified by this PR's checks.Risk
Mainly test harness and CI behavior. Runtime children no longer share developer app-data; tests requiring an occupied fixed port now skip explicitly. Network discovery is still exercised, so this is app-data isolation, not a network sandbox. The broker refactor preserves the existing cancellation contract and has focused regression coverage. CI may expose runner-specific failures not reproduced locally. The desktop intermittent failure remains unidentified after the requested 30 attempts.
Checklist
git commit -s), certifying the Developer Certificate of Origin.