performance-release-summary.mdx

{/* SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */}

Performance and Energy: Upcoming Release

Release-Candidate Profile (2026-10-20)

The unsigned macOS arm64 0.7.0-rc.2 candidate was profiled against the published 0.7.0-rc.1 with the release profiling harness. This is the 0.7.0-rc.2 comparison; no profile has been recorded for the current candidate. It is a partial gate result, not a certified benchmark: the inference scenario and tracing-enabled runs were not measured (see What Was Not Covered).

Result: in this setup the candidate uses about the same CPU and memory as rc.1. Idle GPU sampling now relaxes to its 10-second rate, so the candidate's ioreg collector costs the same as rc.1's. Package idle wakeups are higher in the candidate, and the cause has not been established.

Setup

Item	Value
Candidate	main @ 86214bd, NVPAIR-Setup-0.7.0-rc.2-arm64.dmg SHA-256 0c02e16f348f2e4710ca412cfdbe215d9d542de02959586e82a9555a4d1f18ee; app bundle content-manifest SHA-256 32f6db3b882ddd408cba57dfd14e7b1aba43b8811d1bcab2feb2912c03b474f6
Baseline	Published 0.7.0-rc.1 with only Contents/MacOS/nvpair-helper-ctl removed, so it skips its first-launch privileged-helper prompt; bundle content-manifest SHA-256 deb6e122702aef1adc24833476ff1c901d69bedac6bdf5424e831170fefe79bc
Machine	Apple M5 Max, macOS 27.0.1 (26A434), AC power, display on; the Mac was in interactive use, so Overview windows could be partly covered
Topology	Each run launched a fresh sandboxed home: an unclustered node with no engines and tracing off (default). One LAN peer ran the candidate with its app hidden
Protocol	60 s warm-up, 120 s steady sampling, sampling every 10 s (tray) or 5 s (Overview), baseline and candidate alternated
Scenarios	Tray: Overview closed through its own Close button. Overview: Overview left open

Four runs were excluded: three where the harness's window check saw no window, and one where someone interacted with the window. Statistics use the remaining runs.

Tray Only (rc.1 n=6, candidate n=5)

Medians, with min–max in brackets. CPU is percent of one core.

Metric	rc.1	rc.2 candidate
Total CPU, owned processes (excl. ioreg)	0.97 (0.95–3.99)	0.94 (0.90–3.66)
nvpair-node-info ioreg child CPU	0.33 (0.32–1.14)	0.34 (0.31–1.44)
nvpair-node-info self CPU	0.14 (0.14–0.35)	0.14 (0.13–0.44)
nvpair-ui-broker CPU	0.051 (0.040–0.052)	0.061 (0.053–0.065)
Package idle wakeups per second	0.44 (0.33–0.66)	2.28 (1.52–3.12)
Interrupt wakeups per second, all processes	302 (298–373)	304 (297–376)
Physical footprint (mean)	258 MB (257–523 MB)	258 MB (255–578 MB)
RSS (mean)	725 MB (719–841 MB)	723 MB (720–851 MB)
Logical writes	1.5 KB/s (1.4–2.5)	1.4 KB/s (1.2–1.8)

Overview Open (rc.1 n=5, candidate n=6)

Metric	rc.1	rc.2 candidate
Total CPU, owned processes (excl. ioreg)	3.66 (3.11–4.15)	3.32 (3.11–8.07)
nvpair-node-info ioreg child CPU	1.60 (1.55–1.65)	1.61 (1.59–1.63)
Renderer (PAIR Helper (Renderer)) CPU	0.79 (0.59–0.98)	0.66 (0.60–2.40)
Package idle wakeups per second	0.88 (0.62–2.52)	2.70 (1.67–3.72)
Interrupt wakeups per second, all processes	369 (358–397)	367 (355–477)
Physical footprint (mean)	563 MB (519–574 MB)	549 MB (532–576 MB)
RSS (mean)	876 MB (873–879 MB)	871 MB (869–877 MB)
Logical writes	1.5 KB/s (1.3–1.6)	1.5 KB/s (1.1–2.7)

With Overview open, both versions sample the GPU every 2 seconds, so the ioreg cost is the same by design. The total CPU ranges overlap, so the lower candidate median is not evidence of an improvement.

What the Numbers Show

• Idle sampling works as designed. Tray only, with no workloads, the candidate's sampling falls back to 10 seconds, the same cost as rc.1. Before the fix in #13, a tray-only candidate sampled every 2 seconds (ioreg 1.66% against rc.1's 0.35% in a diagnostic run).
• No measured saving against rc.1 in this setup. The earlier development rounds below were measured on a real clustered installation with live peers, where trust refresh, discovery and alias handling do much more work. A fresh unclustered node gives most of those changes nothing to act on, so this profile neither confirms nor refutes them.
• Package idle wakeups rose from about 0.4 to 2.3 per second. They are spread across several workers (job scheduler, the three proxies, cluster manager). The new 30-second worker health checks are a plausible contributor, but the cause is not established. The increase is small next to roughly 300 interrupt wakeups per second.
• Older peers can hold a node at the fast rate. An rc.1 peer polls /v1/node-info without X-NVPAIR-Poll, and node-info treats a missing header as foreground. While any rc.1 node on the LAN polls a candidate node, that node samples its GPU every 2 seconds. Diagnostic runs measured ioreg at 1.66% in this situation. Idle backoff applies once every peer is upgraded.

What Was Not Covered

• Controlled inference and tracing-enabled runs (gate step 2).
• Clustered nodes, engines, and routing eligibility, freshness or alias failover checks during the runs (gate step 5).
• GPU utilization counters, file growth, and physical power. The kernel billed-energy counter is recorded by the harness but is not reported here, because its units and attribution are unvalidated.
• Windows and Linux.

Publishing a percentage improvement for this release still requires those scenarios.

Historical Development Rounds

These are development observations from three profiling rounds run during the release work on the published 0.7.0-rc.1 app (Round 1) and two patched development builds (Rounds 2 and 3). They are not fresh measurements of the corrected 0.7.0-rc.2 candidate and must not be presented as a certified benchmark: raw samples, power state, and complete build identities were not retained.

Unattended idle backoff is restored behind scheduling leases; the release-candidate profile confirms it reaches the idle rate. An earlier Round 3 attempt let scheduling polls read cached GPU snapshots without coordination, which risked exceeding the scheduler's 10-second freshness budget and was reverted. That unattended idle backoff has now been architecturally restored: nvpair-node-info uses explicit X-NVPAIR-Poll leases (foreground, scheduling, background) and broker demand signaling (stats/set-demand), relaxing sampling to 10 seconds when idle while keeping scheduler decision-point freshness proven <= 8.5 seconds. Reaching that idle rate depends on nothing renewing a lease it does not need, which is why the broker now also owns the scheduling lease (telemetry/set-scheduling-demand) instead of both pollers renewing it unconditionally, and why a tray-only desktop session reads node-info as background. Neither change alters node-info's policy constants or the scheduler's freshness budget; both were profiled only as part of this work, so the quantitative rows below remain historical. In addition, nvpair-manual-nodes delivers scheduling telemetry every ~2 seconds separately from 10-second topology probes. Because this coordinated implementation was completed after the historical profiling rounds, rows marked † (dominated by the earlier prototype) cannot be taken as measurements of the current code. The candidate must be re-profiled on final release artifacts before publishing quantitative performance claims; the release-candidate profile above is that profile for the tray-only and Overview scenarios. Unmarked Round 3 improvements come from changes the candidate keeps (network-event monitoring, trust-pin caching, warning deduplication, TLS-aware probing).

Three-Round Development Comparison

Setup as reported: macOS arm64, all PAIR-owned processes (17 tray-only, 18 with Overview), proc_pid_rusage deltas. CPU is percent of one core; the ioreg child is listed separately.

Idle, Tray-Only (10-second samples)

Metric	Round 1 (published rc.1)	Round 2	Round 3	Round 1 → 3
Total CPU, owned processes (excl. ioreg) †	1.94%	1.60%	1.13%	−41.8%
nvpair-node-info self CPU †	0.36%	0.36%	0.13%	−63.9%
nvpair-node-info ioreg child CPU †	~1.35%	~1.35%	0.35%	−74.1%
PAIR main-process CPU	0.42%	0.35%	0.12%	−71.4%
GPU utilization (AGXAccelerator)	0.00%	0.00%	0.00%	—
Physical footprint	320.2 MB	289.8 MB	276.2 MB	−44.0 MB (−13.7%)
Resident memory (RSS)	784.5 MB	746.9 MB	728.7 MB	−55.8 MB (−7.1%)
Package idle wakeups	14.49/s	0.20/s	0.10/s	−99.3%
Interrupt wakeups, all processes †	741.65/s	519.84/s	362.53/s	−51.1%
nvpair-node-info interrupt wakeups †	94.7/s	97.3/s	32.0/s	−66.2%
nvpair-node-scanner interrupt wakeups	135.5/s	113.1/s	65.3/s	−51.8%
Three proxies' interrupt wakeups	179.9/s	180.7/s	134.1/s	−25.5%
Kernel billed energy, 10-second delta †	not captured	2,343,906	91,356	−96.1% (Rounds 2 → 3)
Logical disk writes	44.8 KB/s	0 in sample	0 in sample	none observed
nvpair.jsonl growth	~7.5 KB per 10 s	~450 B per 24 s	0 B in sample	none observed

Overview Window Open (5-second steady-state samples)

Metric	Round 1 (published rc.1)	Round 2	Round 3	Round 1 → 3
Steady-state CPU, owned processes	~5.20%	3.40%	not reported	−34.6% (Rounds 1 → 2)
Package idle wakeups	3.79/s	0.00/s	0.00/s	−100%
Interrupt wakeups, all processes	812.40/s	474.12/s	372.65/s	−54.1%
nvpair-ui-broker interrupt wakeups	110.2/s	20.3/s	9.4/s	−91.5%
nvpair-node-scanner interrupt wakeups	135.5/s	113.1/s	58.2/s	−57.0%
ollama-proxy interrupt wakeups	57.2/s	65.1/s	27.0/s	−52.8%
lmstudio-proxy interrupt wakeups	64.5/s	63.6/s	33.4/s	−48.2%
omlx-proxy interrupt wakeups	58.2/s	52.0/s	26.8/s	−54.0%
Logical disk writes	44.8 KB/s	0 in sample	0 in sample	none observed

With the Overview open, GPU sampling is foreground in every round, so idle sampling backoff does not apply to this table.

Reading These Numbers

• Wakeups and avoided I/O are energy-related indicators, not watts or battery runtime. The kernel billed-energy counter's units and attribution were not validated; treat its −96% as directional only, and note it is dominated by the historical sampling experiment.
• "0 in sample" means no writes during that window, not a zero-I/O guarantee. Real warnings and debug-level logging still write.
• Remaining idle activity is intentional: two-second trust-directory checks in seven workers, two-second scheduling telemetry delivery (the GPU collection behind it relaxes to ten seconds only while no workload is queued or running), one-second CPU/memory sampling, and mDNS loops.
• Short samples, warm-up, background load and power state limit causal claims.

Changes Retained in the Candidate

Improvement	Work avoided / intended benefit	Validation boundary
No-op endorsement merges	Redundant trust-change notifications and downstream refreshes	Regression coverage; first-round profiling reported lower churn
Stable manual-node identity with operational failover	Alias oscillation and unnecessary discovery/engine refreshes, without suppressing a healthy address	Corrected after profiling; alias/failover regressions pass; final profile pending
TLS-aware peer probing	Plaintext engine probes that clustered peers will reject	Probe regression coverage; whole-app effect not isolated quantitatively
Verbose/debug log gating and duplicate-log removal	Routine serialization and synchronous file writes	Logging/installer-independent tests; historical Round 2 I/O observation above
Repeated rejection-warning deduplication	Recurrent warning writes from unchanged legacy-peer probes	Included after Round 2; no standalone measured saving claimed
Filtered macOS kernel network events	Routing-cache-miss traffic and periodic socket-timeout wakeups	Source/tests and historical Round 3 observations; no isolated effect visible in the unclustered 2026-10-20 profile; clustered-peer profile pending
Validated trust-pin byte caching	Repeated JSON/PEM/X.509 parsing of unchanged pins	Trust/race coverage; periodic directory/file reads remain
Lease-driven GPU sampling and restored idle backoff	Reduces unattended GPU collection while strictly preserving scheduler freshness budget	Restored behind X-NVPAIR-Poll leases and separate ~2 s manual telemetry; scheduling leases are now renewed only while a workload is active, and a tray-only desktop reads node-info as background; profiled 2026-10-20 (tray only, unclustered): idle rate reached, same ioreg cost as rc.1. rc.1 peers that poll without the header keep a node at 2 s

The historical aggregate changes cannot be allocated precisely to individual fixes. Intentional discovery, trust refresh, telemetry delivery, and supervision still perform background work.

Final Artifact Profiling Gate

The retained release profiling harness collects reproducible macOS arm64 artifact comparisons and documents the gate checks that still require controller evidence.

Before publishing a new quantitative release claim:

1. Preserve the baseline and final artifact hashes, source commits, service versions, harness, machine/OS/GPU details, power source/mode, screen state, cluster/peer versions, collector settings, engines/models, and workload.
2. Use equivalent warm-up and topology; alternate baseline/candidate runs with at least five repetitions per scenario and at least 120 seconds of steady sampling after warm-up. Test tray-only, visible Overview, and controlled inference, with tracing both disabled and enabled in its supported modes.
3. Discover and sample the whole owned process tree dynamically, including short-lived hardware collectors. Separate app CPU, child CPU, GPU counters, physical footprint, RSS, logical writes, file growth, and wakeup counters.
4. Measure physical energy/power only with a documented, supported instrument and attribution method. Otherwise report the energy proxy metrics explicitly, without converting kernel billed-energy counters into battery claims.
5. Report per-run data, median and spread, failures/skips, and sampling limits. Verify routing eligibility, telemetry freshness and alias failover during the same scenarios; a lower counter with broken scheduling is not an improvement.
6. Inspect the final candidate and repeat profile/lifecycle checks after any runtime correction. A build or component test is not a packaged-runtime pass.

Do not replace the installed application, alter real cluster membership, start engines, or enable payload capture solely to produce a benchmark without an explicitly authorized test environment.