Alt-overview tab: tok/s speedometer dials + vLLM serving lanes - #87
Open
yr75myjnqx-tech wants to merge 4 commits into
Open
Alt-overview tab: tok/s speedometer dials + vLLM serving lanes#87yr75myjnqx-tech wants to merge 4 commits into
yr75myjnqx-tech wants to merge 4 commits into
Conversation
…lanes
Adds a fleet-level 'Alt-overview' tab that renders the same node cards
as Overview, with the LLM workload section replaced by two side-by-side
speedometer dials (prefill left, generation right) plus a serving-lanes
widget per node.
Frontend:
- SpeedGauge: hand-rolled SVG speedometer, EMA-smoothed needle driven by
requestAnimationFrame, theme-aware tick colours, pinned-at-max pointer
with warn colour when value exceeds scale.
- Gauge scales are model-keyed via MODEL_SCALES in src/config/display.js
(per served model name), with FALLBACK_SCALE + visible 'Default Scale'
badge for unknown models. A per-Spark manual override can be set from a
gear popover on each dial and is persisted in server settings.
- ServingLanes widget: lane occupancy pips with n/N numeral, queue depth,
estimated wait (queue x mean decode of completed requests, last 15 m),
TTFT p95, KV cache and prefix cache.
- New /gauges route + tab entry in desktop pill row and mobile menu.
Backend:
- LlmProbe: avgDecodeSeconds computed from windowed histogram sum/count
deltas over a rolling window (same pattern as ttftSeconds), replacing
the lifetime mean.
- SparkRegistry/SparkMonitor: optional maxNumSeqs per Spark (for lane
totals), surfaced in snapshots.
- settings: gaugeScales map (spark id -> {prefill, generation}).
Tests: SparkTabs tab/menu counts updated (23 frontend tests pass,
typecheck clean).
The gauges variant hid the page h1, so the Alt-overview tab lacked the 'Overview' heading and the header row layout differed from Overview. The header (title left, action buttons right) now renders identically in both variants.
…adings - useSnapshot: every WS snapshot reset activeId to Overview unless it was a spark id, so the /gauges view bounced back to Overview within one poll cycle. GAUGES_ID is now treated as always-valid, matching the Overview reservation. - SpeedGauge: the viewBox (-4..136) was not centred on the dial's geometric centre (CX=60), so the PREFILL/GENERATION headings and the digital readout sat ~4% of width left of the dial centre. viewBox is now -10..130, symmetric around CX.
Ports the complete Alt-overview tab experience from the Operator's production fork so the PR matches what the tab looks and behaves like in daily use: - Fixed-domain sparklines replace temperature/usage bars on the overview cards (GPU/CPU temp 20-95 C with warn band, usage 0-100), riding the existing WebSocket push cadence via the shared metrics store. Sparkline gains an optional domain prop: omitted, it keeps the legacy auto-scale for the existing detail-page call sites. - CLUSTER WORKER empty state for TP worker nodes (metrics served by the cluster head) — a worker is never rendered as 'no workload'. - Data freshness: 'updated Ns ago' on every card, STALE/DEAD states. - Share-safe mode (?share=1 or settings toggle, in-memory only): identifiers alias to Node A/B/C + model-a/b, capacity renders as percentages, Wake All/Shutdown All and settings gears are removed from the DOM, and a persistent banner marks the redacted view. - ShareModeProvider + banner wired into App. - The fork's acceptance suite (10 tests: gauge scales, idle/placeholder states, topology honesty, share mode, injection safety) ported verbatim — 33/33 frontend tests pass, typecheck clean, build OK.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a fleet-level Alt-overview tab (
/gauges) — the same node cards as Overview, with the LLM workload section replaced by per-node instrumentation:MODEL_SCALESinsrc/config/display.js, keyed by served model name) with a visible "Default Scale" badge for unknown models; a gear popover sets a per-Spark manual override persisted in server settings. Values above scale pin the pointer at max in the warn colour.Sparklinecomponent gains an optionaldomainprop — when omitted it keeps its legacy auto-scale, so the detail-page call sites are untouched.CLUSTER WORKER — metrics served by <head> (head)instead of looking idle or absent.updated Ns ago, with STALE/DEAD fallbacks.?share=1or settings toggle, in-memory only): hostnames alias toNode A/B/C, models tomodel-a/b, capacity renders as percentages, Wake All/Shutdown All and settings gears are removed from the DOM (not hidden), and a persistent banner marks the redacted view — safe to screenshot.Backend changes
LlmProbe:avgDecodeSecondsfrom windowed histogram sum/count deltas over a rolling window (same pattern asttftSeconds), replacing the lifetime mean.SparkRegistry/SparkMonitor: optionalmaxNumSeqsper Spark, surfaced in snapshots for lane totals.settings: newgaugeScalesmap (spark id → {prefill, generation}).Why
On a multi-node fleet serving different models, a fleet view that emphasises instantaneous tok/s rate, queue state, and thermal/usage trends is more useful for live-ops than aggregate numbers alone. The gauge scales are model-keyed (not per-node, not auto-scaled), so needle angles stay honestly comparable across nodes serving the same model. This has been running in production on a 3-node GB10 fleet (one TP=2 pair + one standalone) for several weeks.
Testing
npm test: server (node --test) + frontend (vitest) — 33/33 frontend tests pass, including the ported acceptance suite (honestVisuals.acceptance.test.tsx): gauge scale provenance, empty-state skeleton, topology honesty, share-mode redaction, and injection safety. Typecheck clean,vite buildsucceeds.?share=1redacted view.No new runtime dependencies. MIT-licensed, same as upstream.