These are the Go services behind PAIR. They discover other PAIR nodes on the
local network: each node advertises itself over mDNS as one consolidated
_nvpair-node._tcp record, and peers learn from that record which services a
node offers and where to reach them.
What a discovered node can actually serve is a separate question, answered after discovery. A node may run Ollama, LM Studio, oMLX, any supported combination, or none. Model inventory is fetched from engine-manager rather than crammed into mDNS TXT records. The bundled oMLX engine manifest supports macOS arm64 only and has no model pull/delete actions.
Locally, each node exposes compatibility proxies — Ollama-compatible and OpenAI-compatible — so an unmodified client on that machine can reach any capable node in the cluster. Routing is automatic: the proxy picks a node that is online, has a compatible engine running, and holds the requested model.
Standalone, the Ollama proxy defaults to http://localhost:11435; under the
broker it serves the Ollama-compatible http://localhost:11434 when that port is
free. If an inherited OLLAMA_HOST names a different local HTTP port, the broker
can serve that loopback-only address from the same router when it is free —
localhost is claimed on IPv4 and IPv6 together, and remote or HTTPS targets are
never intercepted.
The oMLX proxy defaults to 1234 standalone. Under the broker, when LM Studio
occupies or reserves 1234, oMLX is assigned a free fallback starting at 1236,
excluding reserved ports. Use reported proxy status rather than assuming its
endpoint. Model inventory is served on 14322; node-info on 14318 serves
hardware, utilization, and identity telemetry, not the model list.
PAIR ships a graphical UI alongside these services. The UI launches
nvpair-ui-broker from the same directory; the broker orchestrates the
workers and exposes a newline-delimited JSON-RPC 2.0 API over stdio (or a Unix
socket / Windows named pipe via --ipc). For supported platforms and engines,
see the root README.
This tree builds 14 service executables: nvpair-ui-broker, 12 supervised
workers, and nvpair-tui. The broker attempts to start available workers at
startup. Standalone, only the scanner is required; missing default auxiliary
binaries and initial auxiliary spawn failures are non-fatal. Invalid explicit
worker paths and scanner initial spawn failures are fatal. Optional initial
spawn failures retry under the same budget as crashes and eligible hang restarts:
1 s → 16 s backoff, five replacement attempts, with reset requiring 60 s uptime
and a successful probe since that spawn. Missing defaults stay missing without
automatic retry until workers:restart finds the binary. Desktop also prevalidates
broker, scanner, Ollama proxy, node-info, and node-settings binaries.
Each worker answers worker/health with {"ok":true}. The broker probes every
30 s (--worker-health-interval, minimum 1 s) with a 5 s deadline capped at the
interval. Two consecutive failures report degraded; a successful probe restores
running and clears the failure count. Three failures restart scanner, errors,
settings, node-info, workload-manager, manual-nodes, cluster-manager, and scheduler.
Engine-manager and the three proxies never auto-restart on a hang: restart them
explicitly if needed. This checks control responsiveness, not inference readiness.
workers:get-status and full-snapshot workers:update expose all twelve workers;
workers:restart requests a restart or retry even after budget exhaustion.
Desktop renders the registry under Settings → Service → Workers (including
"Not responding — restart manually" for ineligible hung workers). TUI Overview
shows all twelve and offers R with confirmation. Worker probing is separate
from the broker's 15 s heartbeat and clients' 45 s inbound-silence handling.
Cross-process hang tests are Unix-only; Windows hang behavior remains unproven.
nvpair-tui launches and supervises its own broker instead of being supervised.
Processes use newline-delimited JSON-RPC 2.0 over stdio or optional Unix socket /
Windows named pipe, with relay protocol v2 notification envelopes. See
Control Flow.
| Binary | Role |
|---|---|
nvpair-ui-broker |
Parent service and JSON-RPC API surface used by the bundled UI and other clients. Supervises workers, relays consolidated discovery, and coordinates routing and scheduling. |
ollama-proxy |
Ollama-compatible HTTP reverse proxy. Routes only to advertised model owners, with owner failover and scheduler priorities. |
lmstudio-proxy |
LM Studio counterpart to ollama-proxy, forwarding OpenAI-compatible inference routes with equivalent owner-only routing and failover behavior. |
omlx-proxy |
OpenAI-compatible routing proxy for oMLX, with owner-only routing, failover, and cache-affinity hints. |
nvpair-node-info |
Local HTTP service on :14318 exposing GPU, CPU, and memory inventory at /v1/node-info. |
nvpair-node-scanner |
Consolidated discovery daemon. Advertises and browses _nvpair-node._tcp, maintains the node directory, and enriches peers with hardware and model information over HTTP. |
nvpair-manual-nodes |
Manages user-added nodes that don't appear via mDNS; probes them every 10 s, delivering compact scheduling telemetry every ~2 s. |
nvpair-workload-manager |
Cluster-wide workload relay. |
nvpair-errors |
Service-errors datastore + cross-node sync. |
nvpair-engine-manager |
Config-driven control plane for local inference engines, model inventory, and trusted remote engine operations. |
nvpair-node-settings |
Typed key-value store for per-node preferences. |
nvpair-cluster-manager |
Node identity, PIN pairing, and the trusted-node store. |
nvpair-job-scheduler |
Responsive scheduler combining total node queue depth across engines with smoothed GPU pressure. |
nvpair-tui |
Terminal interface for headless and SSH operation; launches and supervises its own broker. |
Shared code lives in the local shared/ Go module (imported as nvpair-shared/…, replaced via replace nvpair-shared => ../shared). It provides logging, wire types, JSON-RPC and IPC, discovery records, mDNS, network monitoring, stable node identity, application data paths, and cluster trust helpers.
The mDNS responder is our own rather than the host's, because Windows ships
none. Its receive and per-interface send sockets bind UDP 5353 as RFC 6762
requires. Socket reuse lets them coexist with sibling PAIR processes and with a
system responder — avahi-daemon on Linux or Bonjour where present — without
configuration.
The broker feeds every accepted local or peer workload transition plus compact
GPU telemetry to the scheduler. Queued and running work is counted by destination
node across Ollama, LM Studio, and oMLX together. Fresh maximum-GPU utilization is
smoothed into pressure 0–3; missing or stale telemetry is neutral. Rankings use
pending + gpuPressure, and each proxy adds local reservations before choosing,
so bursts spread without waiting for workload feedback.
nvpair-ui-broker/ Parent service / JSON-RPC API surface
ollama-proxy/ Ollama-compatible routing proxy
lmstudio-proxy/ OpenAI-compatible routing proxy for LM Studio
omlx-proxy/ OpenAI-compatible routing proxy for oMLX
nvpair-node-info/ Local GPU-inventory HTTP service
nvpair-node-scanner/ Consolidated _nvpair-node._tcp discovery daemon
nvpair-manual-nodes/ Manual-node manager
nvpair-workload-manager/ Cluster workload relay
nvpair-errors/ Service-errors datastore + sync
nvpair-engine-manager/ Local and remote inference-engine control plane
nvpair-node-settings/ Per-node preferences store
nvpair-cluster-manager/ Node pairing / trust service
nvpair-job-scheduler/ Cluster job scheduler
nvpair-tui/ Terminal interface for headless / SSH operation
shared/ Shared Go module (nvpair-shared/…)
eap-noob/ EAP-NOOB implementation used by cluster pairing
tests/ Cross-process integration tests (separate go.mod)
versions.json Single source of truth for every component version
build.bat Builds all fourteen service executables (Windows)
build.sh Builds all fourteen service executables (Linux/macOS)
VERSIONING.md SemVer rules and version-bump workflow
Every directory above is its own Go module and holds its own tests beside the source it covers. See Testing.
- Go 1.25 or newer.
jqonPATH— the build scripts use it to parseversions.json. Install viawinget install jqlang.jq/choco install jq/scoop install jqon Windows,sudo apt install jqon Debian/Ubuntu,sudo dnf install jqon Fedora/RHEL, orbrew install jqon macOS.- No C toolchain, GUI, or webkit dependencies. Every module is pure Go (
cgois not used).
Always use the platform-appropriate build script, run from this services/
directory.
On Windows:
.\build.batOn Linux and macOS:
./build.shBoth scripts read versions.json, build all fourteen service executables with
-X main.Version=… ldflags, and stage them together in services/build/bin/.
Do not build individual components by hand without also copying their binaries into build/bin/: the broker will silently keep using the older binary there.
The staged binaries in build/bin/ are what you run locally. Installable builds
come from the
releases page.
Normal installations launch the bundled UI, which starts the broker from the same installation directory.
To drive a services-only build interactively, run nvpair-tui — it launches and
supervises its own broker, so nothing else needs to be running. See
Using the PAIR terminal interface.
For service development or direct API access, launch the broker with
build/bin/ as its working directory. Default worker paths resolve from the
current working directory, not from the broker executable's location:
Set-Location build\bin
.\nvpair-ui-broker.exe # Windowscd build/bin
./nvpair-ui-broker # Linux/macOSThe bundled UI connects to the broker over its JSON-RPC API (stdio or --ipc).
Other clients can use the same contract. See nvpair-ui-broker/README.md for
the API surface.
Every binary respects two inputs for its starting log level, resolved before any other work happens. Precedence: --log-level flag > NVPAIR_LOG_LEVEL env var > default (info). The broker passes its own resolved level to every subprocess it spawns (--log-level <level>), so setting either before launch turns on debug output from the very first line of all processes.
From services/build/bin/, use either the environment variable or the flag:
# Windows / PowerShell
$env:NVPAIR_LOG_LEVEL = "debug"
& ".\nvpair-ui-broker.exe"
# Alternatively:
& ".\nvpair-ui-broker.exe" --log-level debug# Linux/macOS
NVPAIR_LOG_LEVEL=debug ./nvpair-ui-broker
# Alternatively:
./nvpair-ui-broker --log-level debugAccepted values: debug, info, warn, error. The level can also be changed live over the broker's log/set-level JSON-RPC method (not persisted across restarts), so for launch-time issues use the env var or the flag.
Those are standalone defaults. Desktop persists modularLogLevel, defaults to
warn, and passes an explicit level to the broker. Current unreleased desktop
logging preserves structured service severity and writes verbose protocol/debug
entries to its log file only at debug; stderr alone does not mean warning.
Proxy OpenTelemetry tracing is configured dynamically by nvpair-ui-broker via
telemetry/configure. Proxies start with tracing disabled and do not inspect
ambient environment variables (PAIR_TELEMETRY_*, OTEL_*). Settings are stored
in nvpair-node-settings under schema 2 with monotonic revision tracking, and
payload capture requires explicit user consent (payload_consent_version: 1).
Export is strictly loopback-only (127.0.0.1:<collectorPort>, default 4317), not
cluster mTLS. Arize Phoenix is managed by the broker from an embedded Compose
template, publishing only 127.0.0.1:6006 and 127.0.0.1:<collectorPort>. Serving
nodes record no spans for peer requests, trace context headers are stripped across
nodes, request headers are allowlisted to Content-Type and Content-Length, and
captured payloads are truncated to 16 KB UTF-8. Trace volumes (nvpair-phoenix-data)
can be purged via telemetry/purge-storage ({confirm: "delete-traces"}). See
Observability
and SECURITY.md for details.
There are two layers, and while you are editing a component you want the first one.
Every component keeps its tests beside its source, and each is its own Go module, so run them from that component's directory:
cd nvpair-engine-manager
go test ./...These are fast and are what you should run on each edit. Substitute any component
directory — shared/ and eap-noob/ have their own tests too:
cd shared
go test ./...Every one of the fourteen service executables has tests, as do shared/ and
eap-noob/. Depth varies with how much behaviour a component carries:
nvpair-engine-manager and nvpair-cluster-manager have the largest suites,
while a component with one test file may still hold twenty test functions in it.
File count is a poor proxy for coverage, so read the suite rather than counting
files.
tests/ has its own go.mod and exercises real binaries talking to each other.
Its TestMain builds the workers it needs into a temp directory first, so you do
not have to build beforehand:
cd tests
go test ./...Every runtime launch uses a per-test temporary app-data root for HOME,
USERPROFILE, XDG_CONFIG_HOME, APPDATA, and LOCALAPPDATA, inherited by
supervised workers. Restart tests explicitly reuse their own temporary root.
Runtime commands inherit the remaining environment with those five roots
overridden; build commands keep the original home/config roots so Go's caches
are not redirected into disposable test state. The
TestProcessHelpersRequireIsolation source guard rejects launch helpers that
bypass the isolated command factory or replace its environment.
Expect this to take a few minutes. Run it before opening a merge request, and after any change to a JSON-RPC method, payload, or notification. CI runs the cross-process suite on macOS arm64; component vet/tests run on both macOS and Linux.
Some tests need something the machine may not have, and skip rather than fail:
- Tests that bind a well-known port (
11434,14319,14321) skip when it is already in use — so quit PAIR before running them, or expect gaps. - Tests needing a real mDNS responder skip when one is unavailable.
- Live engine tests only run when you opt in:
From nvpair-engine-manager/, both the live build tag and environment opt-in
are required; the ordinary go test ./... excludes these files:
NVPAIR_LIVE_OLLAMA=1 go test -tags live -run '^TestLiveOllamaCleanRoom$' -v -timeout 900s
NVPAIR_LIVE_LMSTUDIO=1 go test -tags live -run '^TestLiveLMStudioCleanRoom$' -v -timeout 1200sThese perform real engine lifecycle operations with platform-specific guards;
read the selected test before running it. Separate live_pound_test.go coverage
uses an existing Ollama install. Run deliberately on systems you control.
From omlx-proxy/, the smoke test uses an existing loopback engine without
restarting it (no live build tag):
PAIR_OMLX_LIVE_URL=http://127.0.0.1:1235 PAIR_OMLX_LIVE_MODEL='your-model-id' \
go test -run '^TestLiveOMLXCacheAffinityConversation$' -v -timeout 10mReplace your-model-id with a real advertised model ID. This single-node test does not
prove cross-node routing, KV-cache hits, or comparative performance.
A skip is not a pass. If you are relying on a test, check it actually ran.
services/versions.json is the single source of truth for every component version and the umbrella product version. The build scripts read it and stamp each binary via -ldflags "-X main.Version=...". Bump the components your change affects in the same pull request, and describe any user-facing change in the pull-request description so it reaches the release notes. VERSIONING.md has the bump rules.
You can verify a built binary's stamped version at any time:
.\build\bin\ollama-proxy.exe --version # Windows./build/bin/ollama-proxy --version # LinuxVERSIONING.md— SemVer bump rules and how a release is declared.bom.md— bill of materials for a given release.- Per-component
README.mdfiles in each subdirectory. - Architecture — how these services fit together at runtime, including node-to-node transport and discovery.