diff --git a/CHANGELOG.md b/CHANGELOG.md index fd6d4e5..26fc52e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,7 @@ ## Unreleased ### Added +- **TensorFold engines are discovered from the process table**: the TensorFold status reader (heartbeat and `ide-agent-kit tensorfold-status`) now also finds this user's running `tensorfold-native serve` / `tensorfold serve` engines (one `ps` on macOS, `/proc` on Linux, no sudo), merges them with the registry by port and publishes them as `source: "discovered"`, so TensorFold Studio's engine on its random port gets a card. A command line is treated as a claim, not configuration: what a process is comes from its real executable (`lsof -d txt` on macOS, `/proc//exe` on Linux), and it is read only at a loopback or wildcard TCP socket its own pid holds in LISTEN state, never at a `--host`; a `--port` it does not listen on, several ports with no `--port`, a non-loopback-only socket, or no way to check (no `lsof`) means the engine is not read. On by default with `INTENT_TENSORFOLD_REGISTRY`; `INTENT_TENSORFOLD_DISCOVER=1` runs it without a registry, `=0` or `--no-discover` turns it off. No key is ever taken from a command line, a key file a command line names, or a process environment: a discovered engine gets a key only through a registry entry marked `"engine": "tensorfold"` on its port with a `keyFile`, so the Studio engine (key in memory only) reports `metrics: "auth_required"` with its model from the command line. New fields: `name`, `app`, `model_source`, `memory.footprint_gb`, and a top-level `discovery` (`ok` / `off` / `unavailable`). Key files for TensorFold status reads, including a registry `keyFile`, are now refused when other users can read them, as the engine refuses them. - **TensorFold status on the device heartbeat**: `uik-daemon` and `ide-agent-kit intent daemon` can publish the live status of each TensorFold server in the model registry under the device's `tensorfold` key, so CodeWatch can show a card per machine. Opt in with `INTENT_TENSORFOLD_REGISTRY=`; `INTENT_TENSORFOLD_SCOPE=local` (default, loopback entries only) or `all`. Read with GET `/health`, `/metrics` and `/v1/slide/graph` only, never a completion. Publishes model, warming, version/backend/device when the server reports them, memory, running and waiting requests, token counters, tok/s from counter deltas between reads, draft acceptance, TTFT p50, KV cache use, Living Weights fact counts (never fact text) and the metrics auth state. An unreachable server is `reachable: false`; an unreadable registry is an `error`, never an empty list. New `ide-agent-kit tensorfold-status [--json]` prints the same object. The registry gains `engine` (`"tensorfold"` marks an entry, any other name skips it, absent means auto-detect). A key is sent only to an entry marked TensorFold that names its own `keyFile`, and only on the two gated routes; the fleet-wide `LLM_API_KEY` fallbacks are never used for a status read. - **What is still waiting on you** (`bin/iak-pending.mjs`): asks every daemon the fleet knows about - this machine's plus every `watchdog-roster.json` entry with a `gate` - for its pending intents and prints them oldest first with the age in the leading column. Motivation: confirmation cards post to the room correctly and then scroll away; on 2026-09-20 four approval requests sat pending on one machine for up to 9 hours 12 minutes and the owner asked twice in one night what was waiting on him, because no surface anywhere could answer it. A host that could not be reached is reported under its own `COULD NOT ASK` heading and is never folded into "nothing pending"; a gate naming another machine by LAN IP is refused rather than probed and reported the same way. A daemon whose rows carry no `status` this version understands is a shape mismatch rather than an empty queue, and a fleet roster that exists but cannot be parsed or read exits `2` instead of silently shrinking the fleet to this machine (only a *missing* roster means "single-machine install"). An age that cannot be believed - missing, future-dated, or epoch-seconds - renders as `age unknown`, sorts to the top and survives `--older-than`, rather than being clamped to "fresh". Exit codes: `0` all clear, `10` pending, `11` some hosts unreachable, `12` no host reachable, `2` cannot run, precedence `12 > 11 > 10 > 0`; `--older-than` narrows what is printed and announced but can never produce a `0`. `--json` for machines. `--room` requires `--older-than` and announces each item at most once per escalation (threshold, then each doubling of its age), keeping a ledger at `~/.iak/pending-announced.json` that is written atomically before the post, carries forward entries for hosts it could not ask, rolls back when a post fails, and degrades to silence - never a storm - when it is missing, corrupt or unwritable. `examples/pending-launchd.plist` wires it to a timer. The gate token is resolved by the existing `resolveGateToken` helper, is sent only to hosts that helper already trusts, and never appears in output or argv. - **Session degradation watch** (`bin/iak-degradation-watch.mjs`): a per-machine daemon that tails watched agents' Claude Code session transcripts and posts a room alert (pushed to the owner's phone via CodeWatch) the moment a session's `message.model` leaves the configured accepted set or its `permissionMode` differs from the expected one. Motivation: on 2026-07-18 an agent's session silently slipped to a small fallback model with manual permission prompts while the owner was away; the degraded agent kept posting — looping on stale tasks and reposting completed instructions — and cost an evening to diagnose. Alerts fire immediately on first detection (a wrong model in a transcript is a fact, not a flap) with a 30-minute per-signal cooldown and a recovery note. Config: `degradation_watch` block (`interval_sec`, `alert_room`, `agents[]` with `transcript_dir`, `accepted_models` prefix list, `expected_permission_mode`); API key reuses `poller.api_key`. `--once` and `--dry-run` flags for testing. Run one instance per machine under process supervision. diff --git a/README.md b/README.md index a00eef2..d20c101 100644 --- a/README.md +++ b/README.md @@ -496,7 +496,9 @@ can show a card for it on each machine. 1. List the server in `config/models.json` with `"engine": "tensorfold"` (or leave `engine` out and let it be auto-detected). See - `config/models.README.md`. + `config/models.README.md`. Engines running on this machine, including the + one TensorFold Studio starts on a random port, are found from the process + table without an entry. 2. Check what the card will say: `ide-agent-kit tensorfold-status` (add `--json` for the exact object). 3. Turn it on for the publisher: set @@ -505,7 +507,9 @@ can show a card for it on each machine. It is read with GET requests only (`/health`, `/metrics`, `/v1/slide/graph`), never a completion, and an unreachable server shows as unreachable instead of -stopping the heartbeat. The payload shape is in +stopping the heartbeat. Keys are read only from a registry entry's +`keyFile`, never from a command line; TensorFold Studio keeps its engine key +in memory, so its card says `metrics: auth required`. The payload shape is in `packages/user-intent-kit/README.md`. UIK v0.2.2 or later recommended for the `uik-daemon` bin. v0.2.0 introduced the daemon, v0.2.1 fixed a silent-exit in the example file, and v0.2.2 added agent-status republish on the heartbeat cadence so the agent slot does not expire while the daemon is running. Pre-0.2.2 daemons publish device state correctly but the agent slot still goes stale after its TTL. @@ -1207,7 +1211,7 @@ ide-agent-kit gateway [options] ide-agent-kit memory [options] ide-agent-kit rooms archive [--backfill] [--max-pages N] [--room R] ide-agent-kit rooms search "words" [--from handle] [--since 2026-09-20] [--regex] [--limit N] -ide-agent-kit tensorfold-status [--json] [--registry ] [--local] [--timeout ] +ide-agent-kit tensorfold-status [--json] [--registry ] [--local] [--no-discover] [--timeout ] ide-agent-kit init [--ide ] [--profile ] ide-agent-kit acp [options] ide-agent-kit keepalive [--pid-file ] [--heartbeat-sec ] diff --git a/bin/cli.mjs b/bin/cli.mjs index 61ad5c7..97d63c8 100755 --- a/bin/cli.mjs +++ b/bin/cli.mjs @@ -102,9 +102,10 @@ Usage: daemon: long-running publisher (desktop state + agent heartbeat) [--interval ] Add INTENT_TENSORFOLD_REGISTRY= to publish TensorFold status on each beat. - ide-agent-kit tensorfold-status [--json] [--registry ] [--local] [--timeout ] [--config ] - Live status of every TensorFold server in the model registry, read with GET only - (/health, /metrics, /v1/slide/graph). Same object the device heartbeat publishes. + ide-agent-kit tensorfold-status [--json] [--registry ] [--local] [--no-discover] [--timeout ] [--config ] + Live status of every TensorFold server in the model registry and every TensorFold engine + running on this machine, read with GET only (/health, /metrics, /v1/slide/graph). + Same object the device heartbeat publishes. ide-agent-kit poll --rooms --api-key --handle <@handle> [--interval ] [--config ] (Legacy) Poll GroupMind rooms with explicit CLI args. Prefer "rooms watch". @@ -578,7 +579,12 @@ async function main() { console.error('Error: --timeout must be a positive number of milliseconds'); process.exit(2); } - const result = await runTensorFoldStatus({ registryPath, scope: opts.local ? 'local' : 'all', json: !!opts.json, timeoutMs }); + const envRegistry = String(process.env.INTENT_TENSORFOLD_REGISTRY || '').trim(); + const registryRequired = typeof opts.registry === 'string' || (envRegistry !== '' && !['0', 'off', 'false', 'no'].includes(envRegistry.toLowerCase())); + const result = await runTensorFoldStatus({ + registryPath, registryRequired, discover: !opts['no-discover'], + scope: opts.local ? 'local' : 'all', json: !!opts.json, timeoutMs, + }); (result.exitCode === 0 ? console.log : console.error)(result.text); process.exit(result.exitCode); } diff --git a/config/models.README.md b/config/models.README.md index 24b0281..0d82b15 100644 --- a/config/models.README.md +++ b/config/models.README.md @@ -82,13 +82,31 @@ Which entries count as TensorFold: both, so mark a keyed server `"engine": "tensorfold"` explicitly. * any other `engine`: never asked. +Engines running on this machine do not need an entry at all: the status +reader also finds this user's `tensorfold-native serve` / `tensorfold serve` +processes and reads them the same way (`source: "discovered"`). It checks the +process's real executable, not just its command line, and reads it only at a +loopback TCP socket that process is listening on; `--host` is ignored. An +entry on the same loopback port wins: its `id` is used (and its `keyFile` +when it is marked `"engine": "tensorfold"`), and an entry with no `engine` +needs no auto-detection. An entry with another `engine` on that port stays +skipped. Turn discovery off with `INTENT_TENSORFOLD_DISCOVER=0` or +`tensorfold-status --no-discover`. + **Keys.** A TensorFold server may gate `/metrics` and `/v1/slide/graph` behind a key (`/health` stays open, and says only `{"status": "ok"}` while -keys are on). The status reader sends a key only to an entry marked -`"engine": "tensorfold"` that names its own `keyFile`, and only on those two -routes. It never uses `LLM_API_KEY` / `LLM_API_KEY_FILE`: those belong to some -other server as far as a status read knows. Without a usable key it reports -`metrics: "auth_required"` and the rest of the card reads unknown. +keys are on). The status reader sends a key only on those two routes, and +only from the `keyFile` of an entry marked `"engine": "tensorfold"`. The file +is read the way the engine reads it (one key per line, `label: key` or a +bare key, `#` comments) and refused, as the engine refuses it, when other +users can read it (`chmod 600`). It never uses `LLM_API_KEY` / +`LLM_API_KEY_FILE`, a key on a command line, a key file a command line names +(`--api-key-file`: any process can name any file there), or a process's +environment. For a keyed engine you started yourself, add an entry for its +port with `"engine": "tensorfold"` and the same file as `keyFile`. Without a usable key it reports `metrics: "auth_required"` and +the rest of the card reads unknown. TensorFold Studio's engine is always in +that state: the app makes a fresh key on every launch, keeps it in memory and +passes it to the engine in the environment, never in a file. ```json { diff --git a/packages/user-intent-kit/README.md b/packages/user-intent-kit/README.md index 6366c43..c981453 100644 --- a/packages/user-intent-kit/README.md +++ b/packages/user-intent-kit/README.md @@ -132,11 +132,25 @@ device card: ```bash export INTENT_TENSORFOLD_REGISTRY=~/ide-agent-kit/config/models.json export INTENT_TENSORFOLD_SCOPE=local # default; `all` also reads remote entries +export INTENT_TENSORFOLD_DISCOVER=1 # default with a registry; 0 turns it off; 1 alone runs without one ``` - **GET only.** `/health`, `/metrics` and `/v1/slide/graph`, never a completion, so a status read cannot interfere with a Living Weights lesson. -- **Off unless set.** Nothing is read or published without the registry path. +- **Off unless set.** Nothing is read or published without the registry path + or `INTENT_TENSORFOLD_DISCOVER=1`. +- **Running engines are found too.** Each read also scans this user's + processes for `tensorfold-native serve` and `tensorfold serve` (one `ps` on + macOS, `/proc` on Linux, no sudo). A command line is only a claim: the + process must really be running `tensorfold-native` (or Python running + TensorFold), checked against its executable (`lsof -d txt`, + `/proc//exe`), and it is read only at a loopback TCP socket it is + listening on, matching its `--port` when it names one; `--host` is + ignored, and without `lsof` nothing is read. The model and name come from + the command line. Engines are merged with the registry by port and + published as `source: "discovered"`. This is how TensorFold Studio's + engine, which gets a new random port every launch, gets a card. Process + environments are never read. - **Not on the heartbeat's path.** It reads on its own timer at the heartbeat interval; the beat publishes the last reading and never waits on a socket. - **One key, replaced whole.** Everything goes under `tensorfold`, so a server @@ -146,6 +160,13 @@ export INTENT_TENSORFOLD_SCOPE=local # default; `all` also reads remote entrie - **Which entries:** marked `"engine": "tensorfold"`, or auto-detected from their `/metrics`. See `config/models.README.md` in ide-agent-kit, including how a keyed server names its key. +- **Keys come from the registry only.** The `keyFile` of an entry marked + `"engine": "tensorfold"`, read in the engine's key-file format and refused + when other users can read the file. Never a command line, a key file a + command line names, or an environment. TensorFold Studio keeps its engine + key in memory and hands it to the engine through the environment, so its + engine shows `metrics: "auth_required"`, the model from its command line, + and the reason. - **TensorFold's default port is 8080, and so is `INTENT_MODEL_ENDPOINT`'s.** The served-model probe above may send one short generation request to a loopback endpoint to prove what is loaded. On a machine where TensorFold @@ -160,12 +181,13 @@ The payload, per device: "checked_at": "2026-10-14T09:00:00.000Z", "scope": "local", "error": null, + "discovery": "ok", "servers": [{ "id": "mb-tensorfold", "host": "localhost", "port": 8091, "local": true, - "source": "registry", "reachable": true, "reason": null, - "model": "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit", "warming": false, + "source": "registry", "name": null, "app": null, "reachable": true, "reason": null, + "model": "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit", "model_source": "health", "warming": false, "version": null, "backend": null, "device": null, - "memory": { "used_gb": 18.7, "peak_gb": 40.7, "budget_gb": 96.2 }, + "memory": { "used_gb": 18.7, "peak_gb": 40.7, "budget_gb": 96.2, "footprint_gb": 19.2 }, "requests": { "running": 0, "waiting": 0 }, "tokens": { "prompt_total": 0, "generation_total": 0 }, "tok_s": null, "tok_s_window_s": null, "decode_tok_s": null, @@ -184,10 +206,19 @@ and "could not tell" never look the same. What the numbers mean: -- `local`: the server runs on this machine (loopback entry). `source` is - `registry` (marked `"engine": "tensorfold"`) or `detected`. +- `local`: the server runs on this machine (a loopback entry or a discovered + engine). `source` is `registry` (marked `"engine": "tensorfold"`), + `detected` (auto-detected from its `/metrics` or `/health`) or `discovered` + (found running in the process table). `name` is the engine's `--name`; + `app` is `"TensorFold Studio"` for the engine the app bundles, else null. +- `model_source`: `health` when `/health` named the model, `args` when it came + from the engine's command line (a keyed `/health` says only "ok"). +- `discovery` (top level): `ok`, `off`, or `unavailable` when the process scan + failed. An `unavailable` scan is not the same as "no engines running". - `memory`: GB, base 10, one decimal, like `mem_total_gb`. `used` is the - engine's active memory, `budget` its configured budget. + engine's active memory, `budget` its configured budget, `footprint` the + process's physical footprint (macOS, including Metal buffers). The native + MLX build reports only the footprint. - `tokens`: counters since the server started, for finished requests. - `tok_s`: generated tokens of requests that finished between two reads, divided by the seconds between them (`tok_s_window_s`). `null` on the first diff --git a/packages/user-intent-kit/bin/uik-daemon.js b/packages/user-intent-kit/bin/uik-daemon.js index 6b4193d..fd1f0be 100755 --- a/packages/user-intent-kit/bin/uik-daemon.js +++ b/packages/user-intent-kit/bin/uik-daemon.js @@ -76,6 +76,16 @@ * Never a generation request. An unreachable server * publishes reachable:false; the heartbeat never * waits on it. + * INTENT_TENSORFOLD_DISCOVER 1 | 0. Find TensorFold engines running on this + * machine from the process table (this user's + * `tensorfold-native serve` / `tensorfold serve`, + * checked against the real executable, read only at + * a loopback socket the process listens on, never + * with a key from its command line) and publish them + * as source "discovered", merged with the registry by + * port. Default: on when + * INTENT_TENSORFOLD_REGISTRY is set; set 1 to run it + * with no registry at all, 0 to turn it off. * INTENT_TENSORFOLD_SCOPE local (default) | all. local = only entries on * loopback, i.e. servers on THIS machine, so each * machine's card carries its own servers. all = every diff --git a/packages/user-intent-kit/src/index.js b/packages/user-intent-kit/src/index.js index c08ebc6..9ad2b87 100644 --- a/packages/user-intent-kit/src/index.js +++ b/packages/user-intent-kit/src/index.js @@ -17,6 +17,7 @@ export { STATUS_SCHEMA as TENSORFOLD_STATUS_SCHEMA, METRICS_STATES as TENSORFOLD_METRICS_STATES, } from './tensorfold-status.js'; +export { discoverTensorFoldEngines } from './tensorfold-discovery.js'; export { ModelAvailabilityProbe, describeLocalModels, diff --git a/packages/user-intent-kit/src/tensorfold-discovery.js b/packages/user-intent-kit/src/tensorfold-discovery.js new file mode 100644 index 0000000..82f9b0c --- /dev/null +++ b/packages/user-intent-kit/src/tensorfold-discovery.js @@ -0,0 +1,446 @@ +// SPDX-License-Identifier: AGPL-3.0-only +// +// tensorfold-discovery.js - which TensorFold engines are running on THIS +// machine right now, found from the process table, so a server started on a +// port nobody wrote down still gets a card. +// +// The case that made this necessary: TensorFold Studio (the macOS app) starts +// its own `tensorfold-native serve ... --port ` and picks a new free +// loopback port on every launch. A registry entry cannot name it. +// +// A COMMAND LINE IS A CLAIM, NOT CONFIGURATION. Any process this user runs +// can put any words in its argv, so nothing here acts on argv alone: +// +// * What the process IS comes from the kernel: the executable `lsof -d txt` +// reports on macOS (`ps` `comm` is argv[0] there, so it can be faked), and +// /proc//exe on Linux. It must be `tensorfold-native`, or a Python +// interpreter whose next words are `-m tensorfold serve` or an absolute +// `.../tensorfold serve` entry script. `printf .../tensorfold-native serve` +// is printf. +// * Where it is read comes from its own listening sockets, never from +// `--host`: a TCP socket the pid holds in LISTEN state on a loopback or +// wildcard address, connected to on loopback. A `--port` must be one of +// those sockets; with no `--port` the pid must listen on exactly one port. +// An engine that listens only on another address, does not listen where +// it says, or cannot be checked is not returned (fail closed), and the +// reason is in `skipped`. +// * Keys: none are read here, and the status reader never reads a key file +// a command line names (`--api-key-file`); only that one was given is +// kept, so the card can say how to supply it. A key comes only from a +// registry entry the user wrote. +// +// WHAT IS READ, AND WHAT IS NOT. +// +// * Command lines of this user's processes only: `ps -U -o pid=,args=` +// on macOS (one exec, about 50 ms), /proc//cmdline on Linux (no exec; +// `comm` is checked first so only candidate processes have their argv +// read). Never `ps -E`, never /proc//environ: an engine's key may be +// in its environment (TensorFold Studio passes it as TENSORFOLD_API_KEY), +// and reading another process's environment to get a key it was not given +// is taking a credential, not reading a status. +// * `--api-key` values on a command line are never kept: the parser records +// only that one was there. +// * For each candidate only: two `lsof` calls on macOS (its executable, its +// listening TCP sockets); /proc//exe, its fd links and +// /proc//net/tcp{,6} on Linux. +// * Nothing here prints a command line. The raw process table never leaves +// this module; only the fields below do. +// +// Never throws: a failed scan is {ok: false, error} and an empty list, and +// the caller says so on the card instead of showing no servers as if none ran. + +import { execFile } from 'node:child_process'; +import { readdir, readFile, readlink, stat } from 'node:fs/promises'; +import { platform as osPlatform } from 'node:os'; +import { isAbsolute } from 'node:path'; + +/** The engine's own defaults (cli_args.py and zig/src/server/cli.zig). */ +export const DEFAULT_ENGINE_HOST = '127.0.0.1'; +export const DEFAULT_ENGINE_PORT = 8080; +export const STUDIO_APP_NAME = 'TensorFold Studio'; + +const NATIVE_BINARY = 'tensorfold-native'; +const PYTHON_BINARY = /^python[0-9.]*$/i; +// TensorFold Studio runs the engine it bundles from inside the app. +const STUDIO_BINARY = /\.app\/Contents\/Resources\/engine\/bin\/tensorfold-native$/; +// Linux truncates `comm` to 15 characters: "tensorfold-nati". +const CANDIDATE_COMM = /^(tensorfold|python)/i; +// A `ps` line worth asking the kernel about. Only a prefilter: the decision +// is made against the real executable. +const CANDIDATE_LINE = /(^|\/|-m )(tensorfold-native|tensorfold) serve(?= |$)/; +// Flags whose value matters here. Everything else on the line is ignored. +const WANTED = new Set(['--port', '--name', '--api-key-file', '--slide']); + +function basename(p) { + const s = String(p || ''); + return s.slice(s.lastIndexOf('/') + 1); +} + +/** + * Group the words after `serve` into a positional (the model) and flags. + * `exact` is true for a real argv array (Linux): each flag takes at most the + * next element. For a joined `ps` line (macOS) a value runs until the next + * `--flag`, so a path with spaces in it ("Application Support") survives. + */ +function groupArgs(words, exact) { + const flags = new Map(); + const positional = []; + let current = null; + let seenFlag = false; + let keyOnArgv = false; + for (const word of words) { + if (word.startsWith('--')) { + seenFlag = true; + const eq = word.indexOf('='); + const name = eq > 0 ? word.slice(0, eq) : word; + if (name === '--api-key') keyOnArgv = true; + // A key on the command line is noted and dropped, never stored. + if (name === '--api-key') { current = { name, values: null }; continue; } + current = { name, values: eq > 0 ? [word.slice(eq + 1)] : [] }; + if (WANTED.has(name) && !flags.has(name)) flags.set(name, current); + if (eq > 0) current = null; + continue; + } + if (current) { + if (current.values !== null && (!exact || current.values.length === 0)) current.values.push(word); + if (exact) current = null; + continue; + } + if (!seenFlag) positional.push(word); + } + const value = name => { + const f = flags.get(name); + return f && f.values && f.values.length ? f.values.join(' ') : null; + }; + return { + model: positional.length ? positional.join(' ') : null, + port: value('--port'), + name: value('--name'), + apiKeyFile: flags.has('--api-key-file'), + slide: flags.has('--slide'), + keyOnArgv, + }; +} + +/** "models--Owner--Name/snapshots/" (the Hugging Face cache) -> "Owner/Name". */ +export function repoFromPath(path) { + const m = /(?:^|\/)models--([A-Za-z0-9][A-Za-z0-9._-]*?)--([A-Za-z0-9][A-Za-z0-9._-]*?)(?:\/|$)/.exec(String(path || '')); + return m ? `${m[1]}/${m[2]}` : null; +} + +function finishEngine(pid, binary, g) { + const port = g.port !== null && /^\d{1,5}$/.test(g.port) ? Number(g.port) : null; + // What the server calls itself (--name) first, as its /health would; then + // the model argument as a repo id, or the repo a cache path belongs to. + const repoId = g.model && /^[A-Za-z0-9][A-Za-z0-9._-]*\/[A-Za-z0-9][A-Za-z0-9._-]*$/.test(g.model) ? g.model : null; + const model = g.name || repoId || repoFromPath(g.model) || (g.model ? basename(g.model.replace(/\/+$/, '')) : null); + return { + pid, + binary, + app: STUDIO_BINARY.test(binary) ? STUDIO_APP_NAME : null, + // 0 means "any free port": the socket decides, as with no --port. + port: port && port <= 65535 ? port : null, + name: g.name, + model: model ? model.slice(0, 200) : null, + apiKeyFile: g.apiKeyFile, + keyOnArgv: g.keyOnArgv, + slide: g.slide, + }; +} + +/** + * An exact argv array (Linux /proc) and the process's real executable into + * an engine, or null when it is not `tensorfold serve`. argv[0] is not + * trusted for identity: `exe` is. + */ +export function parseServeArgv(pid, argv, exe) { + if (!Array.isArray(argv) || typeof exe !== 'string' || !isAbsolute(exe)) return null; + const binary = basename(exe); + if (binary === NATIVE_BINARY) { + return argv[1] === 'serve' ? finishEngine(pid, exe, groupArgs(argv.slice(2), true)) : null; + } + if (!PYTHON_BINARY.test(binary)) return null; + if (argv[1] === '-m' && argv[2] === 'tensorfold' && argv[3] === 'serve') { + return finishEngine(pid, exe, groupArgs(argv.slice(4), true)); + } + if (isAbsolute(argv[1] || '') && basename(argv[1]) === 'tensorfold' && argv[2] === 'serve') { + return finishEngine(pid, exe, groupArgs(argv.slice(3), true)); + } + return null; +} + +/** + * One `ps -o args=` line (macOS) and the process's real executable into an + * engine, or null when it is not `tensorfold serve`. The line is argv joined + * with spaces, so argv[0] is found by matching the executable's own path + * (which may contain spaces, as Studio's does) or as one word. + */ +export function parseServeLine(pid, line, exe) { + const text = String(line || ''); + if (typeof exe !== 'string' || !isAbsolute(exe)) return null; + const binary = basename(exe); + let rest; + if (text === exe || text.startsWith(`${exe} `)) rest = text.slice(exe.length + 1); + else rest = text.slice(text.indexOf(' ') + 1 || text.length); + const argv0 = text.slice(0, text.length - rest.length).trim(); + if (binary === NATIVE_BINARY) { + if (argv0 !== exe && basename(argv0) !== NATIVE_BINARY) return null; + const m = /^serve(?: |$)/.exec(rest); + return m ? finishEngine(pid, exe, groupArgs(splitWords(rest.slice(m[0].length)), false)) : null; + } + if (!PYTHON_BINARY.test(binary)) return null; + if (argv0 !== exe && !PYTHON_BINARY.test(basename(argv0))) return null; + const m = /^(?:-m tensorfold|\/\S*\/tensorfold) serve(?: |$)/.exec(rest); + return m ? finishEngine(pid, exe, groupArgs(splitWords(rest.slice(m[0].length)), false)) : null; +} + +function splitWords(s) { + const t = String(s || '').trim(); + return t ? t.split(/\s+/) : []; +} + +/** `ps -o pid=,args=` output into [{pid, args}]. */ +export function parsePsOutput(text) { + const out = []; + for (const raw of String(text || '').split('\n')) { + const m = /^\s*(\d+)\s+(.*)$/.exec(raw); + if (m) out.push({ pid: Number(m[1]), args: m[2] }); + } + return out; +} + +/** `lsof -Fn` lines ("n127.0.0.1:54635", "n*:8080", "n[::1]:8080") into [{host, port}]. */ +export function parseLsofListen(text) { + const out = []; + for (const raw of String(text || '').split('\n')) { + if (!raw.startsWith('n')) continue; + const m = /^n(\*|\[[0-9a-fA-F:.]+\]|[^:\s]+):(\d{1,5})$/.exec(raw.trim()); + if (m) out.push({ host: m[1].replace(/^\[|\]$/g, ''), port: Number(m[2]) }); + } + return out; +} + +/** `lsof -d txt -Fn`: the first name is the process's executable. */ +export function parseLsofExecutable(text) { + for (const raw of String(text || '').split('\n')) { + if (raw.startsWith('n')) return raw.slice(1) || null; + } + return null; +} + +function hexToIpv4(hex) { + // /proc/net/tcp holds the address as one little-endian 32-bit word. + const b = hex.match(/../g).map(x => parseInt(x, 16)).reverse(); + return b.join('.'); +} + +function hexToIpv6(hex) { + // Four little-endian 32-bit words. + const bytes = []; + for (let w = 0; w < 4; w++) bytes.push(...hex.slice(w * 8, w * 8 + 8).match(/../g).reverse()); + const groups = []; + for (let i = 0; i < 16; i += 2) groups.push(parseInt(bytes[i] + bytes[i + 1], 16).toString(16)); + const full = groups.join(':'); + if (full === '0:0:0:0:0:0:0:0') return '::'; + if (full === '0:0:0:0:0:0:0:1') return '::1'; + if (full.startsWith('0:0:0:0:0:ffff:')) { + const v4 = bytes.slice(12).map(x => parseInt(x, 16)).join('.'); + return `::ffff:${v4}`; + } + return full; +} + +/** + * /proc//net/tcp or tcp6 text into the LISTEN sockets whose inode is in + * `inodes`: [{host, port}]. + */ +export function parseProcNetTcp(text, inodes, v6 = false) { + const out = []; + for (const raw of String(text || '').split('\n').slice(1)) { + const f = raw.trim().split(/\s+/); + if (f.length < 10 || f[3] !== '0A' || !inodes.has(f[9])) continue; + const [addr, port] = f[1].split(':'); + if (!addr || !port || addr.length !== (v6 ? 32 : 8)) continue; + out.push({ host: v6 ? hexToIpv6(addr) : hexToIpv4(addr), port: parseInt(port, 16) }); + } + return out; +} + +/** + * Where to connect for a listen address, or null when it is not reachable on + * loopback: a wildcard means this machine's loopback; any other address is + * not read. + */ +export function connectHost(listenHost) { + const h = String(listenHost ?? '').trim().toLowerCase().replace(/^\[|\]$/g, ''); + if (h === '*' || h === '0.0.0.0') return DEFAULT_ENGINE_HOST; + if (h === '::' || h === '::1') return '::1'; + const v4 = h.startsWith('::ffff:') ? h.slice(7) : h; + if (/^127\.\d{1,3}\.\d{1,3}\.\d{1,3}$/.test(v4)) return v4; + return null; +} + +/** + * The one socket to read an engine at, from the sockets its pid listens on. + * Returns {host, port, portFrom} or {reason}; `quiet` marks a reason that is + * not worth a line on the card (an engine still loading its model). + */ +export function chooseSocket(engine, sockets) { + const usable = sockets + .map(s => ({ host: connectHost(s.host), port: s.port })) + .filter(s => s.host !== null && Number.isInteger(s.port) && s.port > 0); + // IPv4 loopback first: what a wildcard socket and most clients use. + usable.sort((a, b) => Number(a.host.includes(':')) - Number(b.host.includes(':'))); + if (engine.port !== null) { + const match = usable.find(s => s.port === engine.port); + if (match) return { ...match, portFrom: 'args' }; + return sockets.length + ? { reason: `does not listen on its --port ${engine.port} on a loopback address; not read` } + : { reason: `not listening on its --port ${engine.port} yet`, quiet: true }; + } + const ports = [...new Set(usable.map(s => s.port))]; + if (ports.length === 1) return { ...usable[0], portFrom: 'socket' }; + if (ports.length > 1) return { reason: `listens on ${ports.length} ports and names none with --port; not read` }; + return sockets.length + ? { reason: 'listens only on non-loopback addresses; not read' } + : { reason: 'not listening yet', quiet: true }; +} + +function run(command, args, { timeoutMs = 3000, maxBuffer = 8 << 20 } = {}) { + return new Promise(resolve => { + execFile(command, args, { timeout: timeoutMs, maxBuffer, env: { PATH: process.env.PATH || '/usr/bin:/bin:/usr/sbin:/sbin', LC_ALL: 'C' } }, + (err, stdout) => resolve({ ok: !err || Boolean(stdout), stdout: String(stdout || ''), error: err })); + }); +} + +class VerifyUnavailable extends Error {} + +function lsofMissing(r) { + return r.error && (r.error.code === 'ENOENT' || r.error.code === 'EACCES'); +} + +/** macOS: the pid's real executable and listening TCP sockets, from lsof. */ +async function inspectWithLsof(pid, exec) { + const exeOut = await exec('lsof', ['-nP', '-w', '-a', '-p', String(pid), '-d', 'txt', '-Fn'], { timeoutMs: 3000 }); + if (lsofMissing(exeOut)) throw new VerifyUnavailable(`lsof unavailable (${exeOut.error.code})`); + const exe = parseLsofExecutable(exeOut.stdout); + if (!exe) return { exe: null, sockets: [] }; // exited, or not ours to look at + const sockOut = await exec('lsof', ['-nP', '-w', '-a', '-p', String(pid), '-iTCP', '-sTCP:LISTEN', '-Fn'], { timeoutMs: 3000 }); + if (lsofMissing(sockOut)) throw new VerifyUnavailable(`lsof unavailable (${sockOut.error.code})`); + return { exe, sockets: parseLsofListen(sockOut.stdout) }; +} + +/** macOS (and any ps that takes -U/-ww/-o): this user's processes. */ +async function processesFromPs({ exec = run, uid = process.getuid?.() } = {}) { + const args = uid === undefined ? ['-ww', '-x', '-o', 'pid=,args='] : ['-ww', '-U', String(uid), '-o', 'pid=,args=']; + const r = await exec('ps', args); + if (!r.ok) throw new Error(`ps failed (${r.error?.code || 'error'})`); + const found = []; + for (const p of parsePsOutput(r.stdout)) { + if (!CANDIDATE_LINE.test(p.args)) continue; + const { exe, sockets } = await inspectWithLsof(p.pid, exec); + const engine = parseServeLine(p.pid, p.args, exe); + if (engine) found.push({ engine, sockets }); + } + return found; +} + +/** Linux: the LISTEN sockets a pid holds, from its fd links and its net namespace's tcp tables. */ +async function socketsFromProc(base, fs) { + const inodes = new Set(); + for (const fd of await fs.readdir(`${base}/fd`)) { + const link = await fs.readlink(`${base}/fd/${fd}`).catch(() => ''); + const m = /^socket:\[(\d+)\]$/.exec(link); + if (m) inodes.add(m[1]); + } + if (!inodes.size) return []; + const sockets = []; + for (const [file, v6] of [['tcp', false], ['tcp6', true]]) { + const text = await fs.readFile(`${base}/net/${file}`, 'utf8').catch(() => ''); + sockets.push(...parseProcNetTcp(text, inodes, v6)); + } + return sockets; +} + +/** Linux: /proc, this user's processes, argv exactly as the kernel holds it. */ +async function processesFromProc({ procRoot = '/proc', uid = process.getuid?.(), fs = { readdir, readFile, readlink, stat } } = {}) { + const found = []; + for (const entry of await fs.readdir(procRoot)) { + if (!/^\d+$/.test(entry)) continue; + const base = `${procRoot}/${entry}`; + try { + if (uid !== undefined && (await fs.stat(base)).uid !== uid) continue; + const comm = String(await fs.readFile(`${base}/comm`, 'utf8')).trim(); + if (!CANDIDATE_COMM.test(comm)) continue; + const exe = await fs.readlink(`${base}/exe`); + const argv = String(await fs.readFile(`${base}/cmdline`, 'utf8')).split('\0').filter(Boolean); + const engine = parseServeArgv(Number(entry), argv, exe); + if (!engine) continue; + let sockets; + try { + sockets = await socketsFromProc(base, fs); + } catch (err) { + found.push({ engine, sockets: [], unverifiable: `cannot read its sockets (${err?.code || 'error'})` }); + continue; + } + found.push({ engine, sockets }); + } catch { + // Exited between readdir and read, or not ours: nothing to report. + } + } + return found; +} + +/** + * Every TensorFold engine this user runs on this machine, each at a loopback + * socket its own pid listens on. + * + * @returns {Promise<{ok: boolean, error: string|null, + * engines: Array<{pid, host, port, portFrom, name, model, app, apiKeyFile, keyOnArgv, slide}>, + * skipped: Array<{pid, model, reason}>}>} + * `apiKeyFile` is a boolean: whether the engine was started with one. + */ +export async function discoverTensorFoldEngines({ + platform = osPlatform(), + exec = run, + uid = process.getuid?.(), + procRoot = '/proc', + fs, +} = {}) { + let found; + try { + found = platform === 'linux' + ? await processesFromProc({ procRoot, uid, ...(fs ? { fs } : {}) }) + : await processesFromPs({ exec, uid }); + } catch (err) { + // Without a way to check what a process is and where it listens, no + // engine is reported rather than any engine trusted on its word. + const why = err instanceof VerifyUnavailable ? `cannot verify engines: ${err.message}` : err?.message; + return { ok: false, error: String(why || 'process scan failed').slice(0, 120), engines: [], skipped: [] }; + } + const engines = []; + const skipped = []; + const ports = new Set(); + for (const { engine: e, sockets, unverifiable } of found) { + const choice = unverifiable ? { reason: unverifiable } : chooseSocket(e, sockets); + if (choice.reason) { + if (!choice.quiet) skipped.push({ pid: e.pid, model: e.model, reason: choice.reason }); + continue; + } + if (ports.has(choice.port)) continue; // one card per port + ports.add(choice.port); + engines.push({ + pid: e.pid, + host: choice.host, + port: choice.port, + portFrom: choice.portFrom, + name: e.name, + model: e.model, + app: e.app, + apiKeyFile: e.apiKeyFile, + keyOnArgv: e.keyOnArgv, + slide: e.slide, + }); + } + return { ok: true, error: null, engines, skipped }; +} diff --git a/packages/user-intent-kit/src/tensorfold-status.js b/packages/user-intent-kit/src/tensorfold-status.js index 9fbcc49..12c9e10 100644 --- a/packages/user-intent-kit/src/tensorfold-status.js +++ b/packages/user-intent-kit/src/tensorfold-status.js @@ -19,19 +19,33 @@ // // KEYS. A key is sent only to an entry the registry marks // `"engine": "tensorfold"`, only when that same entry names its own key -// (`keyFile`, a path), and only on the two gated routes. The fleet-wide -// LLM_API_KEY / LLM_API_KEY_FILE fallbacks are NOT used here: those are some -// other server's credential as far as this reader knows. Auto-detected -// entries are probed without any key at all. Nothing here logs or returns a -// token; Prometheus `key="..."` labels (the native server labels its request -// counter by API key label) are never copied into the output either. +// (`keyFile`, a path), and only on the two gated routes. The file is read in +// the engine's own key-file format and refused, as the engine refuses it, +// when other users can read it. The fleet-wide LLM_API_KEY / LLM_API_KEY_FILE +// fallbacks are NOT used here: those are some other server's credential as +// far as this reader knows. Auto-detected entries are probed without any key +// at all. A discovered engine's command line is never a source of keys: a +// `--api-key-file` on it is not read (any process of this user can name any +// file there), a `--api-key` value is not kept, and a key held only in a +// process's memory or environment (TensorFold Studio's) is not taken. Those +// engines report `metrics: "auth_required"` with the reason. Nothing here +// logs or returns a token; Prometheus `key="..."` labels (the native server +// labels its request counter by API key label) are never copied into the +// output either. +// +// DISCOVERY. Engines running on this machine are found from the process table +// (tensorfold-discovery.js), each at a loopback socket its own pid listens +// on, merged with the registry by port, and published with +// `source: "discovered"`. Same three GETs, same rules. // // FACT TEXT STAYS ON THE BOX. The slide graph carries what the model was // taught, which can be private. Only counts leave this module. import { homedir } from 'node:os'; import { join } from 'node:path'; -import { readRegistryFile, resolveEntryToken } from './model-capacity.js'; +import { readFile as fsReadFile, stat as fsStat } from 'node:fs/promises'; +import { readRegistryFile } from './model-capacity.js'; +import { discoverTensorFoldEngines } from './tensorfold-discovery.js'; export const TENSORFOLD_ENGINE = 'tensorfold'; export const STATUS_SCHEMA = 1; @@ -152,7 +166,7 @@ export function summarizeHealth(body) { version: healthLabel(h.version), backend: healthLabel(h.backend), device: healthLabel(h.device), - memory: { active: finite(mem.active), peak: finite(mem.peak), budget: finite(mem.budget) }, + memory: { active: finite(mem.active), peak: finite(mem.peak), budget: finite(mem.budget), footprint: finite(mem.footprint) }, // The native server's own live decode rate (`live` block in /health). decodeTokS: finite(isObject(h.live) ? h.live.decode_tokens_per_second : undefined), }; @@ -186,6 +200,9 @@ export function summarizeMetrics(series) { // its memory block when keys are on, so this is the fallback for `used`. deviceMemory: finite(one('device_memory_bytes')), deviceMemoryPeak: finite(one('device_memory_peak_bytes')), + // macOS: the process's physical footprint including Metal buffers. Its + // own field, never folded into `used`: it measures something else. + processFootprint: finite(one('process_footprint_bytes')), }; } @@ -359,15 +376,22 @@ export function buildStatus(target, { health, metrics, slide }, { // and the new value is the baseline for the next one. const sample = generation !== null ? { generation_total: generation, at: clockMs } : previous; + // /health names what the server serves; a keyed server's /health says only + // "ok", and then the command line the engine was started with is the next + // best evidence, labelled as such. + const argsModel = typeof target.argsModel === 'string' && target.argsModel ? target.argsModel : null; const status = { id: target.id ?? null, host: target.host, port: target.port, local: isLoopbackHost(target.host), source: target.source ?? 'registry', + name: target.name ?? null, + app: target.app ?? null, reachable, reason: reasons.length ? reasons.join('; ').slice(0, 300) : null, - model: h.model, + model: h.model ?? argsModel, + model_source: h.model ? 'health' : (argsModel ? 'args' : null), warming: h.warming, version: h.version, backend: h.backend, @@ -376,6 +400,7 @@ export function buildStatus(target, { health, metrics, slide }, { used_gb: gb(h.memory.active ?? (metricsOk ? m.deviceMemory : null)), peak_gb: gb(h.memory.peak ?? (metricsOk ? m.deviceMemoryPeak : null)), budget_gb: gb(h.memory.budget), + footprint_gb: gb(h.memory.footprint ?? (metricsOk ? m.processFootprint : null)), }, requests: metricsOk ? m.requests : { running: null, waiting: null }, tokens: metricsOk ? m.tokens : { prompt_total: null, generation_total: null }, @@ -424,21 +449,75 @@ function expandHome(path) { return s; } +const MAX_KEY_FILE_BYTES = 1 << 20; + /** - * The key for an entry the registry marks TensorFold, from that entry's own - * keyFile and nothing wider. Returns {token, note}; `note` says why a key - * could not be used, and never contains the key. + * The first key in a TensorFold key file, read the way the engine reads it + * (zig/src/server/auth.zig): one key per line, `label: key` or a bare key, + * blank lines and `#` comments skipped. Refused, as the engine refuses it, + * when it is not a regular file or other users can read it (mode & 044). + * + * Returns {token, note}. `note` says why no key came out, names the path and + * never the contents. */ -async function entryToken(entry, env) { - if (entry.auth !== 'bearer') return { token: null, note: null }; - const credential = await resolveEntryToken(entry, { env, entryOnly: true }); - if (credential.token) return { token: credential.token, note: null }; - if (!entry.keyFile) return { token: null, note: 'the entry names no keyFile' }; - return { token: null, note: credential.keyWarning || null }; +export async function readEngineKeyFile(rawPath, { readFileImpl = fsReadFile, statImpl = fsStat } = {}) { + const path = expandHome(rawPath); + let info; + try { + info = await statImpl(path); + } catch (err) { + return { token: null, note: `cannot read key file ${path} (${err.code || 'error'})` }; + } + if (!info.isFile()) return { token: null, note: `key file ${path} is not a regular file` }; + if (info.mode & 0o044) return { token: null, note: `key file ${path} is readable by other users; chmod 600 it (the engine refuses it too)` }; + if (info.size > MAX_KEY_FILE_BYTES) return { token: null, note: `key file ${path} exceeds 1 MiB` }; + let text; + try { + text = String(await readFileImpl(path, 'utf8')); + } catch (err) { + return { token: null, note: `cannot read key file ${path} (${err.code || 'error'})` }; + } + for (const raw of text.split(/[\r\n]/)) { + const line = raw.trim(); + if (!line || line.startsWith('#')) continue; + const colon = line.indexOf(':'); + const key = (colon >= 0 ? line.slice(colon + 1) : line).trim(); + if (key && !/\s/.test(key)) return { token: key, note: null }; + } + return { token: null, note: `key file ${path} holds no key` }; +} + +/** + * The key for one server, from the registry and nothing wider: the keyFile + * of an entry marked TensorFold (auth "bearer"). A discovered engine's + * command line only explains why there is none. Returns {token, note}; + * `note` never contains a key. + */ +async function serverToken(entry, engine, readKeyFile) { + if (entry?.engine === TENSORFOLD_ENGINE && entry.auth === 'bearer') { + if (entry.keyFile) return readKeyFile(entry.keyFile); + return { token: null, note: 'the entry names no keyFile' }; + } + if (!engine) return { token: null, note: null }; + if (engine.app) return { token: null, note: `${engine.app} keeps this engine's key in memory only, so its metrics stay locked` }; + if (engine.apiKeyFile) { + return { + token: null, + note: `the engine was started with --api-key-file, which is not read from a command line; add a models.json entry for port ${engine.port} with "engine": "tensorfold" and that keyFile`, + }; + } + if (engine.keyOnArgv) return { token: null, note: 'the key was given on the engine\'s command line; it is not read from there' }; + return { token: null, note: 'the engine was started without a key file' }; +} + +/** What a discovered engine adds to the target it is probed as. */ +function engineTarget(engine) { + return engine ? { name: engine.name ?? null, app: engine.app ?? null, argsModel: engine.model ?? null } : {}; } /** - * Status for every TensorFold server in a loaded registry. + * Status for every TensorFold server in a loaded registry, plus every engine + * discovery found running on this machine. * * engine "tensorfold" always reported, reachable or not. * engine absent auto-detected: reported when /metrics carries @@ -448,36 +527,68 @@ async function entryToken(entry, env) { * TensorFold earlier in this process and stops * answering stays on the list as unreachable. * any other engine skipped, no request sent. + * discovered an engine from the process table, at a loopback + * socket its pid was seen listening on. Merged with + * a loopback registry entry on the same port (the + * entry keeps its id, and its keyFile when it is + * marked TensorFold; an entry without an engine + * needs no auto-detection then), otherwise reported + * on its own as `source: "discovered"`, without a + * key. One card per port. A non-loopback address in + * `discovered` is ignored. * * `samples` and `detected` are Maps the caller keeps between calls (the - * heartbeat does; a one-shot CLI passes fresh ones). Never throws. + * heartbeat does; a one-shot CLI passes fresh ones). `readKeyFile` is for + * tests. Never throws. */ export async function collectTensorFoldStatus(entries, { scope = 'all', + discovered = null, + readKeyFile = readEngineKeyFile, fetchImpl = (...a) => fetch(...a), timeoutMs = DEFAULT_STATUS_TIMEOUT_MS, now = () => Date.now(), clock = () => performance.now(), - env = process.env, samples = new Map(), detected = new Map(), } = {}) { const io = { fetchImpl, timeoutMs }; const inScope = (entries || []).filter(e => scope === 'all' || isLoopbackHost(e.host)); - const results = await Promise.all(inScope.map(async entry => { + const engines = new Map(); + for (const engine of Array.isArray(discovered) ? discovered : []) { + // Discovery returns loopback addresses only; anything else is not read. + if (!Number.isInteger(engine?.port) || !isLoopbackHost(engine.host) || engines.has(engine.port)) continue; + engines.set(engine.port, engine); + } + const claimed = new Set(); + const stamp = () => new Date(now()).toISOString(); + + const probeKnown = async (target, key, entry, engine) => { + const { token, note } = await serverToken(entry, engine, readKeyFile); + const { status, sample } = await probeTensorFold(target, { + ...io, now, clock, token, keyNote: note, previous: samples.get(key) ?? null, + }); + if (sample) samples.set(key, sample); + return { server: status }; + }; + + const fromRegistry = inScope.map(async entry => { const key = `${String(entry.host).toLowerCase()}:${entry.port}`; const target = { id: entry.id, host: entry.host, port: entry.port }; + const engine = isLoopbackHost(entry.host) ? engines.get(entry.port) : undefined; + if (engine) claimed.add(engine.port); try { if (entry.engine && entry.engine !== TENSORFOLD_ENGINE) { - return { skipped: { id: entry.id, reason: `registry says engine "${entry.engine}"` } }; + const also = engine ? ' (a TensorFold process listens on this port)' : ''; + return { skipped: { id: entry.id, reason: `registry says engine "${entry.engine}"${also}` } }; } if (entry.engine === TENSORFOLD_ENGINE) { - const { token, note } = await entryToken(entry, env); - const { status, sample } = await probeTensorFold({ ...target, source: 'registry' }, { - ...io, now, clock, token, keyNote: note, previous: samples.get(key) ?? null, - }); - if (sample) samples.set(key, sample); - return { server: status }; + return await probeKnown({ ...target, ...engineTarget(engine), source: 'registry' }, key, entry, engine); + } + if (engine) { + // The process table already says TensorFold: no detection needed. + detected.set(key, { tensorfold: true, at: now() }); + return await probeKnown({ ...target, ...engineTarget(engine), source: 'discovered' }, key, entry, engine); } const seen = detected.get(key); if (seen && seen.tensorfold === false && now() - seen.at < DETECT_RECHECK_MS) { @@ -485,7 +596,7 @@ export async function collectTensorFoldStatus(entries, { } const base = baseUrlOf(target); const core = await readCore(base, io, null); - const first = buildStatus(target, core, { clockMs: clock(), checkedAt: new Date(now()).toISOString() }); + const first = buildStatus(target, core, { clockMs: clock(), checkedAt: stamp() }); if (!first.status.reachable) { if (seen?.tensorfold) return { server: { ...first.status, source: 'detected' } }; return { skipped: { id: entry.id, reason: `unreachable, engine unknown (${first.status.reason})` } }; @@ -497,7 +608,7 @@ export async function collectTensorFoldStatus(entries, { detected.set(key, { tensorfold: true, at: now() }); const slide = await readSlide(base, io, null); const { status, sample } = buildStatus({ ...target, source: 'detected' }, { ...core, slide }, { - previous: samples.get(key) ?? null, clockMs: clock(), checkedAt: new Date(now()).toISOString(), + previous: samples.get(key) ?? null, clockMs: clock(), checkedAt: stamp(), }); if (sample) samples.set(key, sample); return { server: status }; @@ -506,7 +617,24 @@ export async function collectTensorFoldStatus(entries, { // heartbeat that dies on a status read is worse than a missing card. return { skipped: { id: entry.id, reason: `status read failed (${err?.name || 'error'})` } }; } - })); + }); + + // Registry entries claim their ports synchronously above, before any await, + // so what is left here is the engines no entry names. + const fromDiscovery = [...engines.values()].filter(e => !claimed.has(e.port)).map(async engine => { + const key = `${String(engine.host).toLowerCase()}:${engine.port}`; + const target = { + id: engine.name || `tensorfold-${engine.port}`, host: engine.host, port: engine.port, + ...engineTarget(engine), source: 'discovered', + }; + try { + return await probeKnown(target, key, null, engine); + } catch (err) { + return { skipped: { id: target.id, reason: `status read failed (${err?.name || 'error'})` } }; + } + }); + + const results = await Promise.all([...fromRegistry, ...fromDiscovery]); return { servers: results.filter(r => r.server).map(r => r.server), skipped: results.filter(r => r.skipped).map(r => r.skipped), @@ -518,19 +646,22 @@ export async function collectTensorFoldStatus(entries, { /** * Keeps a cached TensorFold snapshot for the device heartbeat. * - * OFF unless a registry path is given (INTENT_TENSORFOLD_REGISTRY). It runs - * on its own timer and the heartbeat reads `fields()` synchronously, so a - * slow or dead server costs a stale card, never a late heartbeat. The - * registry file is re-read every refresh, so an edit applies without a - * restart. + * OFF unless a registry path is given (INTENT_TENSORFOLD_REGISTRY) or + * discovery is asked for (INTENT_TENSORFOLD_DISCOVER=1). With a registry, + * discovery of engines running on this machine is on unless + * INTENT_TENSORFOLD_DISCOVER=0. It runs on its own timer and the heartbeat + * reads `fields()` synchronously, so a slow or dead server costs a stale + * card, never a late heartbeat. The registry file is re-read and the process + * table re-scanned every refresh, so an edit or a new engine applies without + * a restart. * * Published as ONE key, `tensorfold`, replaced whole on every beat. The * intent server merges a PATCH key by key, so a field this monitor stopped * sending would otherwise linger on the card forever. */ export class TensorFoldMonitor { - #registryPath; #entries; #scope; #intervalMs; #timeoutMs; #fetchImpl; #now; #clock; #env; #warn; - #readRegistry; #timer = null; #inFlight = null; #snapshot = null; + #registryPath; #entries; #scope; #intervalMs; #timeoutMs; #fetchImpl; #now; #clock; #warn; + #readRegistry; #discover; #discoverImpl; #timer = null; #inFlight = null; #snapshot = null; #samples = new Map(); #detected = new Map(); #lastWarning = -Infinity; constructor({ @@ -542,9 +673,10 @@ export class TensorFoldMonitor { fetchImpl = (...a) => fetch(...a), now = () => Date.now(), clock = () => performance.now(), - env = process.env, warn = message => console.warn(message), readRegistry = readRegistryFile, + discover = process.env.INTENT_TENSORFOLD_DISCOVER, + discoverImpl = discoverTensorFoldEngines, } = {}) { const path = typeof registryPath === 'string' ? registryPath.trim() : ''; this.#registryPath = path && !['0', 'off', 'false', 'no'].includes(path.toLowerCase()) ? expandHome(path) : null; @@ -558,19 +690,32 @@ export class TensorFoldMonitor { this.#fetchImpl = fetchImpl; this.#now = now; this.#clock = clock; - this.#env = env; this.#warn = warn; this.#readRegistry = readRegistry; + // true/false as given; a string from the environment; unset means "with + // a registry, yes". In-memory entries (tests, embedders) never scan + // unless asked: a test must not read the developer's process table. + let wantDiscover = null; + if (typeof discover === 'boolean') wantDiscover = discover; + else if (typeof discover === 'string' && discover.trim()) { + const v = discover.trim().toLowerCase(); + if (['1', 'on', 'yes', 'true'].includes(v)) wantDiscover = true; + else if (['0', 'off', 'no', 'false'].includes(v)) wantDiscover = false; + else throw new Error('INTENT_TENSORFOLD_DISCOVER must be 1 or 0'); + } + this.#discover = wantDiscover ?? Boolean(this.#registryPath); + this.#discoverImpl = discoverImpl; } - get enabled() { return Boolean(this.#registryPath || this.#entries); } + get enabled() { return Boolean(this.#registryPath || this.#entries || this.#discover); } get scope() { return this.#scope; } /** For a log line. A path and a scope, never a key. */ describe() { - if (!this.enabled) return 'off (set INTENT_TENSORFOLD_REGISTRY to a models.json to turn it on)'; - return `${this.#registryPath ?? 'in-memory registry'} (scope ${this.#scope}, every ${Math.round(this.#intervalMs / 1000)}s, GET only)`; + if (!this.enabled) return 'off (set INTENT_TENSORFOLD_REGISTRY to a models.json, or INTENT_TENSORFOLD_DISCOVER=1, to turn it on)'; + const source = this.#registryPath ?? (this.#entries ? 'in-memory registry' : 'no registry'); + return `${source} (scope ${this.#scope}, every ${Math.round(this.#intervalMs / 1000)}s, GET only, discovery ${this.#discover ? 'on' : 'off'})`; } /** What the heartbeat publishes: {} when off or before the first read. */ @@ -588,21 +733,37 @@ export class TensorFoldMonitor { async #read() { const checkedAt = new Date(this.#now()).toISOString(); - let entries; - try { - entries = this.#entries ?? await this.#readRegistry(this.#registryPath); - } catch (err) { - const message = String(err?.message || 'unknown error').slice(0, 200); - this.#warning(`registry unreadable, publishing the error instead of servers: ${message}`); - // An error is not an empty list: `error` says the list is unknown. - this.#snapshot = { schema: STATUS_SCHEMA, checked_at: checkedAt, scope: this.#scope, error: `registry unreadable: ${message}`, servers: [] }; - return this.#snapshot; + let entries = []; + let error = null; + if (this.#entries || this.#registryPath) { + try { + entries = this.#entries ?? await this.#readRegistry(this.#registryPath); + } catch (err) { + const message = String(err?.message || 'unknown error').slice(0, 200); + this.#warning(`registry unreadable, publishing the error with whatever discovery found: ${message}`); + // An error is not an empty list: `error` says the registry's part + // of the list is unknown. + error = `registry unreadable: ${message}`; + } + } + let discovered = null; + let discovery = 'off'; + if (this.#discover) { + let found; + try { found = await this.#discoverImpl(); } catch { found = { ok: false, error: 'process scan failed', engines: [] }; } + discovery = found?.ok ? 'ok' : 'unavailable'; + if (!found?.ok) this.#warning(`engine discovery unavailable: ${found?.error || 'unknown error'}`); + discovered = found?.engines ?? []; } const { servers } = await collectTensorFoldStatus(entries, { - scope: this.#scope, fetchImpl: this.#fetchImpl, timeoutMs: this.#timeoutMs, - now: this.#now, clock: this.#clock, env: this.#env, samples: this.#samples, detected: this.#detected, + scope: this.#scope, discovered, fetchImpl: this.#fetchImpl, timeoutMs: this.#timeoutMs, + now: this.#now, clock: this.#clock, samples: this.#samples, detected: this.#detected, }); - this.#snapshot = { schema: STATUS_SCHEMA, checked_at: checkedAt, scope: this.#scope, error: null, servers }; + // A discovered engine's port changes on every relaunch (TensorFold Studio + // picks a free one), so rate samples for ports no longer on the card go. + const live = new Set(servers.map(s => `${String(s.host).toLowerCase()}:${s.port}`)); + for (const key of this.#samples.keys()) if (!live.has(key)) this.#samples.delete(key); + this.#snapshot = { schema: STATUS_SCHEMA, checked_at: checkedAt, scope: this.#scope, error, discovery, servers }; return this.#snapshot; } diff --git a/src/tensorfold-status.mjs b/src/tensorfold-status.mjs index 2b80ace..9c84c61 100644 --- a/src/tensorfold-status.mjs +++ b/src/tensorfold-status.mjs @@ -19,6 +19,7 @@ import { fileURLToPath } from 'node:url'; import { readRegistryFile, RegistryError, resolveCallerHost } from '../packages/user-intent-kit/src/model-capacity.js'; import { collectTensorFoldStatus, STATUS_SCHEMA } from '../packages/user-intent-kit/src/tensorfold-status.js'; +import { discoverTensorFoldEngines } from '../packages/user-intent-kit/src/tensorfold-discovery.js'; import { resolveModelRegistryPath } from './model-selection.mjs'; const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..'); @@ -65,14 +66,21 @@ function metricsLine(state) { /** One server, several short lines. Plain words, nothing derived silently. */ export function formatServer(s) { const where = `${s.host}:${s.port}`; - const head = `${s.id ?? where} ${where} ${s.reachable ? 'reachable' : 'UNREACHABLE'}${s.source === 'detected' ? ' (auto-detected)' : ''}`; + const how = s.source === 'detected' ? 'auto-detected' : (s.source === 'discovered' ? 'discovered running' : null); + const tags = [how, s.app].filter(Boolean).join(', '); + const head = `${s.id ?? where} ${where} ${s.reachable ? 'reachable' : 'UNREACHABLE'}${tags ? ` (${tags})` : ''}`; const lines = [head]; if (s.reason) lines.push(` note: ${s.reason}`); - if (!s.reachable) return lines.join('\n'); + const fromArgs = s.model_source === 'args' ? ' (from its command line)' : ''; + if (!s.reachable) { + if (s.model) lines.push(` model: ${s.model}${fromArgs}`); + return lines.join('\n'); + } const warming = s.warming === null ? 'unknown' : (s.warming ? 'yes' : 'no'); - lines.push(` model: ${show(s.model)} warming: ${warming}`); + lines.push(` model: ${show(s.model)}${fromArgs} warming: ${warming}`); lines.push(` version: ${show(s.version)} backend: ${show(s.backend)} device: ${show(s.device)}`); - lines.push(` memory: ${num(s.memory.used_gb, ' GB')} used, ${num(s.memory.peak_gb, ' GB')} peak, ${num(s.memory.budget_gb, ' GB')} budget`); + const footprint = s.memory.footprint_gb === null ? '' : `, process footprint ${s.memory.footprint_gb} GB`; + lines.push(` memory: ${num(s.memory.used_gb, ' GB')} used, ${num(s.memory.peak_gb, ' GB')} peak, ${num(s.memory.budget_gb, ' GB')} budget${footprint}`); lines.push(` requests: ${num(s.requests.running)} running, ${num(s.requests.waiting)} waiting ` + `tokens since start: ${num(s.tokens.prompt_total)} prompt, ${num(s.tokens.generation_total)} generated`); const rate = s.tok_s === null ? 'needs two samples (the heartbeat publishes it)' : `${s.tok_s} over ${s.tok_s_window_s} s`; @@ -86,7 +94,10 @@ export function formatServer(s) { } export function formatReport(report) { - const out = [`TensorFold servers in ${report.registry} (read from ${report.from} at ${report.checked_at}, GET only):`]; + const discovery = { ok: 'running engines discovered', off: 'discovery off', unavailable: 'discovery UNAVAILABLE' }[report.discovery] ?? report.discovery; + const out = [`TensorFold servers in ${report.registry} plus this machine's processes (${discovery}; read from ${report.from} at ${report.checked_at}, GET only):`]; + if (report.registry_error) out.push(`Registry not read: ${report.registry_error}`); + if (report.discovery_error) out.push(`Discovery failed: ${report.discovery_error}`); if (!report.servers.length) out.push('none found.'); for (const s of report.servers) out.push('', formatServer(s)); if (report.skipped.length) { @@ -97,30 +108,48 @@ export function formatReport(report) { } /** - * Run one read. Returns {exitCode, report, text}. Exit 0 when the registry - * was read (unreachable servers are reported, not fatal), 2 when it could - * not be. + * Run one read. Returns {exitCode, report, text}. Exit 0 when it ran + * (unreachable servers are reported, not fatal). Exit 2 when the registry + * could not be read and it was required: named with --registry or + * INTENT_TENSORFOLD_REGISTRY, or discovery is off. A default registry path + * that does not exist is not an error while discovery can still find + * engines; the report says the registry was not read. */ export async function runTensorFoldStatus({ registryPath, + registryRequired = true, + discover = true, + discoverImpl = discoverTensorFoldEngines, scope = 'all', json = false, fetchImpl, timeoutMs, - env = process.env, now = () => Date.now(), callerHost, readRegistry = readRegistryFile, } = {}) { - let entries; + let entries = []; + let registryError = null; try { entries = await readRegistry(registryPath); } catch (err) { const message = err instanceof RegistryError ? err.message : `cannot read registry ${registryPath}: ${err.message}`; - return { exitCode: 2, report: null, text: `Error: ${message}\nSee config/models.README.md for the registry format.` }; + if (registryRequired || !discover) { + return { exitCode: 2, report: null, text: `Error: ${message}\nSee config/models.README.md for the registry format.` }; + } + registryError = message; + } + let discovered = null; + let discoveryError = null; + let discoverySkipped = []; + if (discover) { + const found = await discoverImpl().catch(() => ({ ok: false, error: 'process scan failed', engines: [] })); + discovered = found.engines ?? []; + discoverySkipped = (found.skipped ?? []).map(e => ({ id: `pid ${e.pid}${e.model ? ` (${e.model})` : ''}`, reason: e.reason })); + if (!found.ok) discoveryError = found.error || 'process scan failed'; } const { servers, skipped } = await collectTensorFoldStatus(entries, { - scope, env, now, + scope, now, discovered, ...(fetchImpl ? { fetchImpl } : {}), ...(timeoutMs ? { timeoutMs } : {}), }); @@ -130,8 +159,11 @@ export async function runTensorFoldStatus({ from: callerHost ?? await resolveCallerHost(), checked_at: new Date(now()).toISOString(), scope, + discovery: !discover ? 'off' : (discoveryError ? 'unavailable' : 'ok'), + ...(registryError ? { registry_error: registryError } : {}), + ...(discoveryError ? { discovery_error: discoveryError } : {}), servers, - skipped, + skipped: [...skipped, ...discoverySkipped], }; return { exitCode: 0, report, text: json ? JSON.stringify(report, null, 2) : formatReport(report) }; } diff --git a/test/fixtures/tensorfold/health-native-1.0.10.json b/test/fixtures/tensorfold/health-native-1.0.10.json new file mode 100644 index 0000000..8d9b5a6 --- /dev/null +++ b/test/fixtures/tensorfold/health-native-1.0.10.json @@ -0,0 +1 @@ +{"status": "ok", "model": "bench", "model_ids": ["bench"], "max_batch_size": 1, "warming": false, "memory": {}, "live": {"connections": 1, "waiting": 0, "decode_tokens_per_second": 0.0, "prefill_tokens_per_second": 0.0}} \ No newline at end of file diff --git a/test/fixtures/tensorfold/health-studio-keyed.json b/test/fixtures/tensorfold/health-studio-keyed.json new file mode 100644 index 0000000..f73cb51 --- /dev/null +++ b/test/fixtures/tensorfold/health-studio-keyed.json @@ -0,0 +1 @@ +{"status": "ok"} \ No newline at end of file diff --git a/test/fixtures/tensorfold/lsof-studio.txt b/test/fixtures/tensorfold/lsof-studio.txt new file mode 100644 index 0000000..abbaeaa --- /dev/null +++ b/test/fixtures/tensorfold/lsof-studio.txt @@ -0,0 +1,3 @@ +p14779 +f3 +n127.0.0.1:54635 diff --git a/test/fixtures/tensorfold/metrics-native-1.0.10.prom b/test/fixtures/tensorfold/metrics-native-1.0.10.prom new file mode 100644 index 0000000..fe32455 --- /dev/null +++ b/test/fixtures/tensorfold/metrics-native-1.0.10.prom @@ -0,0 +1,202 @@ +# HELP tensorfold:requests_running Requests in prefill or decode. +# TYPE tensorfold:requests_running gauge +tensorfold:requests_running 1 +# HELP tensorfold:requests_waiting Requests queued or held until a lane is free. +# TYPE tensorfold:requests_waiting gauge +tensorfold:requests_waiting 0 +# HELP tensorfold:generation_tokens_running Generated tokens held by live streams. +# TYPE tensorfold:generation_tokens_running gauge +tensorfold:generation_tokens_running 0 +# HELP tensorfold:prompt_tokens_total Prompt tokens of finished requests. +# TYPE tensorfold:prompt_tokens_total counter +tensorfold:prompt_tokens_total 856530 +# HELP tensorfold:generation_tokens_total Generated tokens of finished requests. +# TYPE tensorfold:generation_tokens_total counter +tensorfold:generation_tokens_total 6134 +# HELP tensorfold:kv_cache_usage_ratio Tokens in a stream cache divided by that stream's context window. +# TYPE tensorfold:kv_cache_usage_ratio gauge +tensorfold:kv_cache_usage_ratio{pool="0"} 0 +# HELP tensorfold:mtp_drafted_total Draft tokens verified on finished requests. +# TYPE tensorfold:mtp_drafted_total counter +tensorfold:mtp_drafted_total 10231 +# HELP tensorfold:mtp_accepted_total Draft tokens kept on finished requests. +# TYPE tensorfold:mtp_accepted_total counter +tensorfold:mtp_accepted_total 2580 +# HELP tensorfold:request_latency_seconds Seconds from arrival to the reply leaving. +# TYPE tensorfold:request_latency_seconds histogram +tensorfold:request_latency_seconds_bucket{le="0.01"} 0 +tensorfold:request_latency_seconds_bucket{le="0.05"} 0 +tensorfold:request_latency_seconds_bucket{le="0.1"} 0 +tensorfold:request_latency_seconds_bucket{le="0.25"} 0 +tensorfold:request_latency_seconds_bucket{le="0.5"} 0 +tensorfold:request_latency_seconds_bucket{le="1"} 0 +tensorfold:request_latency_seconds_bucket{le="2.5"} 0 +tensorfold:request_latency_seconds_bucket{le="5"} 0 +tensorfold:request_latency_seconds_bucket{le="10"} 3 +tensorfold:request_latency_seconds_bucket{le="30"} 9 +tensorfold:request_latency_seconds_bucket{le="60"} 29 +tensorfold:request_latency_seconds_bucket{le="120"} 41 +tensorfold:request_latency_seconds_bucket{le="300"} 43 +tensorfold:request_latency_seconds_bucket{le="+Inf"} 44 +tensorfold:request_latency_seconds_sum 2945.728436 +tensorfold:request_latency_seconds_count 44 +# HELP tensorfold:time_to_first_token_seconds Seconds from arrival to the first generated token. +# TYPE tensorfold:time_to_first_token_seconds histogram +tensorfold:time_to_first_token_seconds_bucket{le="0.01"} 0 +tensorfold:time_to_first_token_seconds_bucket{le="0.05"} 0 +tensorfold:time_to_first_token_seconds_bucket{le="0.1"} 0 +tensorfold:time_to_first_token_seconds_bucket{le="0.25"} 0 +tensorfold:time_to_first_token_seconds_bucket{le="0.5"} 0 +tensorfold:time_to_first_token_seconds_bucket{le="1"} 0 +tensorfold:time_to_first_token_seconds_bucket{le="2.5"} 0 +tensorfold:time_to_first_token_seconds_bucket{le="5"} 2 +tensorfold:time_to_first_token_seconds_bucket{le="10"} 3 +tensorfold:time_to_first_token_seconds_bucket{le="30"} 13 +tensorfold:time_to_first_token_seconds_bucket{le="60"} 31 +tensorfold:time_to_first_token_seconds_bucket{le="120"} 41 +tensorfold:time_to_first_token_seconds_bucket{le="300"} 43 +tensorfold:time_to_first_token_seconds_bucket{le="+Inf"} 44 +tensorfold:time_to_first_token_seconds_sum 2693.275104 +tensorfold:time_to_first_token_seconds_count 44 +# HELP tensorfold:request_decode_seconds Seconds a finished request spent decoding. Its sum over generation_tokens_total is the decode rate. +# TYPE tensorfold:request_decode_seconds histogram +tensorfold:request_decode_seconds_bucket{le="0.01"} 0 +tensorfold:request_decode_seconds_bucket{le="0.05"} 0 +tensorfold:request_decode_seconds_bucket{le="0.1"} 0 +tensorfold:request_decode_seconds_bucket{le="0.25"} 0 +tensorfold:request_decode_seconds_bucket{le="0.5"} 0 +tensorfold:request_decode_seconds_bucket{le="1"} 0 +tensorfold:request_decode_seconds_bucket{le="2.5"} 11 +tensorfold:request_decode_seconds_bucket{le="5"} 25 +tensorfold:request_decode_seconds_bucket{le="10"} 34 +tensorfold:request_decode_seconds_bucket{le="30"} 44 +tensorfold:request_decode_seconds_bucket{le="60"} 44 +tensorfold:request_decode_seconds_bucket{le="120"} 44 +tensorfold:request_decode_seconds_bucket{le="300"} 44 +tensorfold:request_decode_seconds_bucket{le="+Inf"} 44 +tensorfold:request_decode_seconds_sum 252.453333 +tensorfold:request_decode_seconds_count 44 +# HELP tensorfold:request_time_per_output_token_seconds Seconds from the first generated token to the last, divided by the tokens less one. +# TYPE tensorfold:request_time_per_output_token_seconds histogram +tensorfold:request_time_per_output_token_seconds_bucket{le="0.0025"} 0 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.005"} 0 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.0075"} 0 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.01"} 0 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.015"} 2 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.02"} 15 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.025"} 21 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.03"} 22 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.04"} 24 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.05"} 24 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.075"} 41 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.1"} 44 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.15"} 44 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.2"} 44 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.3"} 44 +tensorfold:request_time_per_output_token_seconds_bucket{le="0.5"} 44 +tensorfold:request_time_per_output_token_seconds_bucket{le="1"} 44 +tensorfold:request_time_per_output_token_seconds_bucket{le="+Inf"} 44 +tensorfold:request_time_per_output_token_seconds_sum 1.823908 +tensorfold:request_time_per_output_token_seconds_count 44 +# HELP tensorfold:decode_rounds_total Decode rounds used by finished requests. +# TYPE tensorfold:decode_rounds_total counter +tensorfold:decode_rounds_total 3533 +# HELP tensorfold:request_prefill_seconds Seconds a finished request's prompt pass took. +# TYPE tensorfold:request_prefill_seconds histogram +tensorfold:request_prefill_seconds_bucket{le="0.01"} 0 +tensorfold:request_prefill_seconds_bucket{le="0.05"} 0 +tensorfold:request_prefill_seconds_bucket{le="0.1"} 0 +tensorfold:request_prefill_seconds_bucket{le="0.25"} 0 +tensorfold:request_prefill_seconds_bucket{le="0.5"} 0 +tensorfold:request_prefill_seconds_bucket{le="1"} 0 +tensorfold:request_prefill_seconds_bucket{le="2.5"} 0 +tensorfold:request_prefill_seconds_bucket{le="5"} 2 +tensorfold:request_prefill_seconds_bucket{le="10"} 3 +tensorfold:request_prefill_seconds_bucket{le="30"} 13 +tensorfold:request_prefill_seconds_bucket{le="60"} 32 +tensorfold:request_prefill_seconds_bucket{le="120"} 42 +tensorfold:request_prefill_seconds_bucket{le="300"} 43 +tensorfold:request_prefill_seconds_bucket{le="+Inf"} 44 +tensorfold:request_prefill_seconds_sum 2624.64772 +tensorfold:request_prefill_seconds_count 44 +# HELP tensorfold:process_footprint_bytes Current OS physical footprint including Metal buffers (macOS). +# TYPE tensorfold:process_footprint_bytes gauge +tensorfold:process_footprint_bytes 43578886120 +# HELP tensorfold:process_footprint_peak_bytes Kernel lifetime peak physical footprint including Metal buffers; never reset by health (macOS). +# TYPE tensorfold:process_footprint_peak_bytes gauge +tensorfold:process_footprint_peak_bytes 43629185000 +# HELP tensorfold:num_requests_running Requests in prefill or decode. A mirror of tensorfold:requests_running. +# TYPE tensorfold:num_requests_running gauge +tensorfold:num_requests_running 1 +# HELP tensorfold:num_requests_waiting Requests queued or held until a lane is free. A mirror of tensorfold:requests_waiting. +# TYPE tensorfold:num_requests_waiting gauge +tensorfold:num_requests_waiting 0 +# HELP tensorfold:kv_cache_usage_perc A stream's cache occupancy under vLLM's name; same streams and ratios as tensorfold:kv_cache_usage_ratio. +# TYPE tensorfold:kv_cache_usage_perc gauge +tensorfold:kv_cache_usage_perc{stream="0"} 0 +# HELP tensorfold:spec_decode_num_draft_tokens_total Draft tokens verified on finished requests, this server's single draft counter. +# TYPE tensorfold:spec_decode_num_draft_tokens_total counter +tensorfold:spec_decode_num_draft_tokens_total 10231 +# HELP tensorfold:spec_decode_num_accepted_tokens_total Draft tokens kept on finished requests. +# TYPE tensorfold:spec_decode_num_accepted_tokens_total counter +tensorfold:spec_decode_num_accepted_tokens_total 2580 +# HELP tensorfold:e2e_request_latency_seconds Seconds from arrival to the reply leaving, under vLLM's name. +# TYPE tensorfold:e2e_request_latency_seconds histogram +tensorfold:e2e_request_latency_seconds_bucket{le="0.01"} 0 +tensorfold:e2e_request_latency_seconds_bucket{le="0.05"} 0 +tensorfold:e2e_request_latency_seconds_bucket{le="0.1"} 0 +tensorfold:e2e_request_latency_seconds_bucket{le="0.25"} 0 +tensorfold:e2e_request_latency_seconds_bucket{le="0.5"} 0 +tensorfold:e2e_request_latency_seconds_bucket{le="1"} 0 +tensorfold:e2e_request_latency_seconds_bucket{le="2.5"} 0 +tensorfold:e2e_request_latency_seconds_bucket{le="5"} 0 +tensorfold:e2e_request_latency_seconds_bucket{le="10"} 3 +tensorfold:e2e_request_latency_seconds_bucket{le="30"} 9 +tensorfold:e2e_request_latency_seconds_bucket{le="60"} 29 +tensorfold:e2e_request_latency_seconds_bucket{le="120"} 41 +tensorfold:e2e_request_latency_seconds_bucket{le="300"} 43 +tensorfold:e2e_request_latency_seconds_bucket{le="+Inf"} 44 +tensorfold:e2e_request_latency_seconds_sum 2945.728436 +tensorfold:e2e_request_latency_seconds_count 44 +# HELP tensorfold:request_decode_time_seconds Seconds a finished request spent decoding, under vLLM's name. +# TYPE tensorfold:request_decode_time_seconds histogram +tensorfold:request_decode_time_seconds_bucket{le="0.01"} 0 +tensorfold:request_decode_time_seconds_bucket{le="0.05"} 0 +tensorfold:request_decode_time_seconds_bucket{le="0.1"} 0 +tensorfold:request_decode_time_seconds_bucket{le="0.25"} 0 +tensorfold:request_decode_time_seconds_bucket{le="0.5"} 0 +tensorfold:request_decode_time_seconds_bucket{le="1"} 0 +tensorfold:request_decode_time_seconds_bucket{le="2.5"} 11 +tensorfold:request_decode_time_seconds_bucket{le="5"} 25 +tensorfold:request_decode_time_seconds_bucket{le="10"} 34 +tensorfold:request_decode_time_seconds_bucket{le="30"} 44 +tensorfold:request_decode_time_seconds_bucket{le="60"} 44 +tensorfold:request_decode_time_seconds_bucket{le="120"} 44 +tensorfold:request_decode_time_seconds_bucket{le="300"} 44 +tensorfold:request_decode_time_seconds_bucket{le="+Inf"} 44 +tensorfold:request_decode_time_seconds_sum 252.453333 +tensorfold:request_decode_time_seconds_count 44 +# HELP tensorfold:request_prefill_time_seconds Seconds a finished request's prompt pass took, under vLLM's name. +# TYPE tensorfold:request_prefill_time_seconds histogram +tensorfold:request_prefill_time_seconds_bucket{le="0.01"} 0 +tensorfold:request_prefill_time_seconds_bucket{le="0.05"} 0 +tensorfold:request_prefill_time_seconds_bucket{le="0.1"} 0 +tensorfold:request_prefill_time_seconds_bucket{le="0.25"} 0 +tensorfold:request_prefill_time_seconds_bucket{le="0.5"} 0 +tensorfold:request_prefill_time_seconds_bucket{le="1"} 0 +tensorfold:request_prefill_time_seconds_bucket{le="2.5"} 0 +tensorfold:request_prefill_time_seconds_bucket{le="5"} 2 +tensorfold:request_prefill_time_seconds_bucket{le="10"} 3 +tensorfold:request_prefill_time_seconds_bucket{le="30"} 13 +tensorfold:request_prefill_time_seconds_bucket{le="60"} 32 +tensorfold:request_prefill_time_seconds_bucket{le="120"} 42 +tensorfold:request_prefill_time_seconds_bucket{le="300"} 43 +tensorfold:request_prefill_time_seconds_bucket{le="+Inf"} 44 +tensorfold:request_prefill_time_seconds_sum 2624.64772 +tensorfold:request_prefill_time_seconds_count 44 +# HELP tensorfold:client_disconnections_total Requests a client left before the reply left the server. +# TYPE tensorfold:client_disconnections_total counter +tensorfold:client_disconnections_total 0 +# HELP tensorfold:preemptions_total Requests that had to give a lane up to a later one. +# TYPE tensorfold:preemptions_total counter +tensorfold:preemptions_total 0 diff --git a/test/fixtures/tensorfold/metrics-studio-401.json b/test/fixtures/tensorfold/metrics-studio-401.json new file mode 100644 index 0000000..5ca952c --- /dev/null +++ b/test/fixtures/tensorfold/metrics-studio-401.json @@ -0,0 +1 @@ +{"error": {"message": "A valid API key is required", "type": "invalid_request_error", "param": null, "code": "invalid_api_key"}} \ No newline at end of file diff --git a/test/fixtures/tensorfold/ps-macos-decoys.txt b/test/fixtures/tensorfold/ps-macos-decoys.txt new file mode 100644 index 0000000..d3ca384 --- /dev/null +++ b/test/fixtures/tensorfold/ps-macos-decoys.txt @@ -0,0 +1,13 @@ + 501 /usr/bin/grep -E tensorfold serve + 502 vim /Users/me/notes/tensorfold serve.md + 503 /usr/bin/python3 /Users/me/tools/report.py --about tensorfold serve + 504 /Users/me/.venv/bin/tensorfold pull Vontra/Small + 505 /Applications/TensorFold Studio.app/Contents/Resources/engine/bin/tensorfold-native serve /Users/me/Library/Application Support/models/My Model --name Owner/Model --host 0.0.0.0 --port 9123 --slide-graph /Users/me/Library/Application Support/graph.json --slide --api-key-file /Users/me/.tensorfold/keys + 506 /opt/tf/bin/tensorfold-native serve Vontra/Small --port 9200 --api-key not-a-real-key-x --name small + 507 /opt/tf/bin/tensorfold-native serve Vontra/NoPort --name noport + 508 /usr/bin/python3 -m tensorfold serve Vontra/ViaModule --port 9300 + 509 /opt/tf/bin/tensorfold-native serve Vontra/Dup --port 9200 + 510 /opt/tf/bin/tensorfold-native serve Vontra/Eq --port=9400 --host=:: + 511 /usr/bin/printf /tmp/tensorfold-native serve Fake/Model --host example.invalid --port 1234 --api-key-file /tmp/dummy-key + 512 /tmp/tensorfold-native serve Fake/Argv0 --port 1235 + 513 /opt/tf/bin/tensorfold-native serve Fake/Remote --host 192.0.2.7 --port 9500 --api-key-file /tmp/dummy-key diff --git a/test/fixtures/tensorfold/ps-macos-live.txt b/test/fixtures/tensorfold/ps-macos-live.txt new file mode 100644 index 0000000..cbf29da --- /dev/null +++ b/test/fixtures/tensorfold/ps-macos-live.txt @@ -0,0 +1,3 @@ + 1361 /Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python /Users/me/tmp/tensorfold/.venv/bin/tensorfold serve Vontra/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit --host 127.0.0.1 --port 8091 --context 32768 +10169 /Users/me/tmp/tf-rel/v1.0.10/zig-out/native/bin/tensorfold-native serve /Users/me/.cache/huggingface/hub/models--Vontra--Qwen3.8-27B-MLX-4bit/snapshots/70ae7fac63274ff2eac54152031433374cb80f2f --name bench --host 127.0.0.1 --port 8101 --drafter z-lab/Qwen3.8-27B-DFlash2 --context 262144 --parallel 1 --prompt-cache-gib 0 +14779 /Volumes/TensorFold/TensorFold.app/Contents/Resources/engine/bin/tensorfold-native serve TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit --host 127.0.0.1 --port 54635 --name TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit --prompt-cache-gib 8 diff --git a/test/fixtures/tensorfold/slide-graph-native-1.0.10.json b/test/fixtures/tensorfold/slide-graph-native-1.0.10.json new file mode 100644 index 0000000..d03baa4 --- /dev/null +++ b/test/fixtures/tensorfold/slide-graph-native-1.0.10.json @@ -0,0 +1 @@ +{"next":1,"facts":[],"model":{"layers":0,"kinds":"","pocket":{"neurons":0,"layers":[],"used":[]},"shards":[{"name":"model-00001-of-00003.safetensors","bytes":5343268662,"pocket":false,"layers":[]},{"name":"model-00002-of-00003.safetensors","bytes":5354185130,"pocket":false,"layers":[]},{"name":"model-00003-of-00003.safetensors","bytes":5357087557,"pocket":false,"layers":[]}]}} \ No newline at end of file diff --git a/test/tensorfold-discovery.test.mjs b/test/tensorfold-discovery.test.mjs new file mode 100644 index 0000000..b7dc86b --- /dev/null +++ b/test/tensorfold-discovery.test.mjs @@ -0,0 +1,531 @@ +// SPDX-License-Identifier: AGPL-3.0-only +// +// Finding TensorFold engines from the process table, and reading them with +// the same GET-only rules as registry entries. +// +// Fixtures (test/fixtures/tensorfold/): +// ps-macos-live.txt the three `serve` lines `ps -U -o pid=,args=` +// printed on 2026-10-14 (home directory renamed to +// /Users/me): the 0.6.0 Python engine, a native +// 1.0.10 bench engine, and TensorFold Studio's +// bundled engine on its random port. +// ps-macos-decoys.txt synthetic: lines that must NOT be read as engines +// (501-504; 511 is @codexmb's repro, printf with an +// engine-shaped argv; 512 an argv[0] that lies about +// its executable; 513 an engine claiming a remote +// --host), plus the shapes a launcher could produce +// (spaces in paths, --port=, -m, a key on argv, no +// --port, a --port it does not listen on). +// lsof-studio.txt `lsof -Fn` for the Studio engine's listening socket. +// +// What each process really runs comes from the kernel (`lsof -d txt` here, +// faked per pid in EXE): `ps` shows argv, which any process can set. +// health-studio-keyed.json, metrics-studio-401.json +// the Studio engine's live GET answers: /health is +// {"status": "ok"} and /metrics a 401, because the +// app starts it with a key it keeps in memory. +// health/metrics/slide-graph-native-1.0.10.* +// the native bench engine's live GET answers. +// +// No test reads this machine's process table or talks to a live engine: the +// process scan is injected, and every server is a local test server that +// records each request so the suite can assert only the three GETs arrive. + +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { createServer } from 'node:http'; +import { readFileSync } from 'node:fs'; +import { mkdtemp, mkdir, writeFile, rm, symlink } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { + parsePsOutput, parseServeLine, parseServeArgv, parseLsofListen, parseLsofExecutable, parseProcNetTcp, + connectHost, chooseSocket, repoFromPath, discoverTensorFoldEngines, STUDIO_APP_NAME, +} from '../packages/user-intent-kit/src/tensorfold-discovery.js'; +import { + collectTensorFoldStatus, readEngineKeyFile, TensorFoldMonitor, METRICS_STATES, +} from '../packages/user-intent-kit/src/tensorfold-status.js'; +import { loadRegistry } from '../packages/user-intent-kit/src/model-capacity.js'; + +const fixture = name => readFileSync(new URL(`./fixtures/tensorfold/${name}`, import.meta.url), 'utf8'); +const PS_LIVE = fixture('ps-macos-live.txt'); +const PS_DECOYS = fixture('ps-macos-decoys.txt'); +const LSOF_STUDIO = fixture('lsof-studio.txt'); +const STUDIO_HEALTH = fixture('health-studio-keyed.json'); +const STUDIO_401 = fixture('metrics-studio-401.json'); +const NATIVE_HEALTH = fixture('health-native-1.0.10.json'); +const NATIVE_METRICS = fixture('metrics-native-1.0.10.prom'); +const NATIVE_GRAPH = fixture('slide-graph-native-1.0.10.json'); +const STUDIO_MODEL = 'TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit'; + +const DUMMY_KEY = 'tf-dummy-not-real'; +const ALLOWED = new Set(['GET /health', 'GET /metrics', 'GET /v1/slide/graph']); + +async function tfServer(t, routes) { + const seen = []; + const server = createServer((req, res) => { + seen.push({ line: `${req.method} ${req.url}`, auth: req.headers.authorization ?? null }); + let route = routes[`${req.method} ${req.url}`]; + if (typeof route === 'function') route = route(req); + if (!route) { res.writeHead(404, { 'content-type': 'application/json' }); res.end('{"error": {"message": "unknown path"}}'); return; } + res.writeHead(route.status ?? 200, { 'content-type': route.type ?? 'application/json', ...(route.headers || {}) }); + res.end(route.body ?? ''); + }); + await new Promise(r => server.listen(0, '127.0.0.1', r)); + t.after(() => new Promise(r => { server.closeAllConnections(); server.close(r); })); + return { port: server.address().port, seen }; +} + +function assertGetOnly(seen) { + for (const r of seen) assert.ok(ALLOWED.has(r.line), `status reader sent ${r.line}`); +} + +// The Studio engine as it answers live: open /health that says only "ok", +// 401 on the gated routes unless the right key comes with the request. +const keyedRoutes = () => { + const gate = body => req => (req.headers.authorization === `Bearer ${DUMMY_KEY}` + ? { body, type: 'text/plain' } + : { status: 401, body: STUDIO_401, headers: { 'www-authenticate': 'Bearer' } }); + return { + 'GET /health': { body: STUDIO_HEALTH }, + 'GET /metrics': gate(NATIVE_METRICS), + 'GET /v1/slide/graph': gate(NATIVE_GRAPH), + }; +}; +const openNativeRoutes = () => ({ + 'GET /health': { body: NATIVE_HEALTH }, + 'GET /metrics': { body: NATIVE_METRICS, type: 'text/plain' }, + 'GET /v1/slide/graph': { body: NATIVE_GRAPH }, +}); + +/** A discovered engine as discoverTensorFoldEngines returns it. */ +const engine = (port, extra = {}) => ({ + pid: 4242, host: '127.0.0.1', port, portFrom: 'args', name: null, model: null, + app: null, apiKeyFile: false, keyOnArgv: false, slide: false, ...extra, +}); + +/** The real executable of every pid in the fixtures, as `lsof -d txt` reports it. */ +const STUDIO_EXE = '/Applications/TensorFold Studio.app/Contents/Resources/engine/bin/tensorfold-native'; +const EXE = { + 1361: '/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python', + 10169: '/Users/me/tmp/tf-rel/v1.0.10/zig-out/native/bin/tensorfold-native', + 14779: '/Volumes/TensorFold/TensorFold.app/Contents/Resources/engine/bin/tensorfold-native', + 501: '/usr/bin/grep', 502: '/usr/bin/vim', 503: '/usr/bin/python3', 504: '/usr/bin/python3', + 505: STUDIO_EXE, 506: '/opt/tf/bin/tensorfold-native', 507: '/opt/tf/bin/tensorfold-native', + 508: '/usr/bin/python3', 509: '/opt/tf/bin/tensorfold-native', 510: '/opt/tf/bin/tensorfold-native', + 511: '/usr/bin/printf', 512: '/bin/sleep', 513: '/opt/tf/bin/tensorfold-native', +}; +const parseLines = text => Object.fromEntries(parsePsOutput(text).map(p => [p.pid, parseServeLine(p.pid, p.args, EXE[p.pid])])); + +async function keyFile(t, text, mode = 0o600) { + const dir = await mkdtemp(join(tmpdir(), 'iak-tf-engine-key-')); + t.after(() => rm(dir, { recursive: true, force: true })); + const path = join(dir, 'keys'); + await writeFile(path, text, { mode }); + return path; +} + +// --- parsing the process table -------------------------------------------------- + +test('live macOS ps lines: Python 0.6.0, native bench, and the Studio engine on its random port', () => { + const engines = Object.values(parseLines(PS_LIVE)).filter(Boolean); + const simple = engines.map(e => ({ pid: e.pid, port: e.port, name: e.name, model: e.model, app: e.app, slide: e.slide, keyOnArgv: e.keyOnArgv })); + assert.deepEqual(simple, [ + { pid: 1361, port: 8091, name: null, model: 'Vontra/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit', app: null, slide: false, keyOnArgv: false }, + { pid: 10169, port: 8101, name: 'bench', model: 'bench', app: null, slide: false, keyOnArgv: false }, + { pid: 14779, port: 54635, name: STUDIO_MODEL, model: STUDIO_MODEL, app: STUDIO_APP_NAME, slide: false, keyOnArgv: false }, + ]); +}); + +test('decoys are not engines; launcher shapes parse; a key or key file on argv is noted and never kept', () => { + const byPid = parseLines(PS_DECOYS); + for (const pid of [501, 502, 503, 504]) assert.equal(byPid[pid], null, `pid ${pid} is not a TensorFold engine`); + assert.equal(byPid[511], null, 'printf with an engine-shaped argv is printf (@codexmb repro)'); + assert.equal(byPid[512], null, 'an argv[0] naming tensorfold-native does not make /bin/sleep an engine'); + const studio = byPid[505]; + assert.equal(studio.app, STUDIO_APP_NAME, 'a bundle path with spaces is still the Studio engine'); + assert.equal(studio.port, 9123); + assert.equal(studio.name, 'Owner/Model'); + assert.equal(studio.slide, true); + assert.equal(studio.apiKeyFile, true, 'only THAT a key file was named'); + assert.ok(!('host' in studio) && !('listenHost' in studio), '--host is never taken from argv'); + assert.doesNotMatch(JSON.stringify(byPid), /\.tensorfold\/keys|dummy-key|example\.invalid|192\.0\.2\.7/, 'no key path or argv host leaves the parser'); + const keyed = byPid[506]; + assert.equal(keyed.keyOnArgv, true); + assert.equal(keyed.name, 'small'); + assert.equal(keyed.port, 9200); + assert.doesNotMatch(JSON.stringify(keyed), /not-a-real-key-x/, 'an --api-key value never leaves the parser'); + assert.equal(byPid[507].port, null, 'no --port: the socket decides'); + assert.equal(byPid[508].model, 'Vontra/ViaModule', 'python -m tensorfold serve'); + assert.equal(byPid[508].port, 9300); + assert.equal(byPid[510].port, 9400, '--port=N'); + // The line matches, but the executable does not: no engine either way. + assert.equal(parseServeLine(1, '/opt/tf/bin/tensorfold-native serve X', undefined), null, 'no executable known, no engine'); + assert.equal(parseServeLine(1, '/usr/bin/python3 /tmp/x.py -m tensorfold serve X', '/usr/bin/python3'), null); +}); + +test('Linux argv (exact, from /proc//cmdline) keeps a value with spaces whole; identity is /proc//exe', () => { + const native = '/opt/tf/bin/tensorfold-native'; + const e = parseServeArgv(7, [native, 'serve', '/home/me/My Models/qwen', '--port', '9500', '--api-key-file', '/home/me/tf keys', '--slide'], native); + assert.equal(e.model, 'qwen'); + assert.equal(e.port, 9500); + assert.equal(e.apiKeyFile, true); + assert.equal(e.slide, true); + const py = parseServeArgv(8, ['/usr/bin/python3', '/home/me/.venv/bin/tensorfold', 'serve', 'Vontra/X', '--host', '0.0.0.0'], '/usr/bin/python3.12'); + assert.equal(py.model, 'Vontra/X'); + const key = parseServeArgv(9, [native, 'serve', 'Vontra/X', '--api-key', 'argv-key-x', '--port', '9600'], native); + assert.equal(key.keyOnArgv, true); + assert.equal(key.port, 9600, 'the flag after a dropped key value still parses'); + assert.doesNotMatch(JSON.stringify(key), /argv-key-x/); + assert.equal(parseServeArgv(10, ['/usr/bin/less', 'tensorfold', 'serve'], '/usr/bin/less'), null); + assert.equal(parseServeArgv(11, [native, 'pull', 'Vontra/X'], native), null); + assert.equal(parseServeArgv(12, [native, 'serve', 'Vontra/X'], '/usr/bin/printf'), null, 'argv[0] is a claim; exe is the process'); + assert.equal(parseServeArgv(13, ['/usr/bin/python3', 'tensorfold', 'serve'], '/usr/bin/python3'), null, 'a relative script is not an entry point we can name'); + assert.equal(parseServeArgv(14, ['/usr/bin/python3', '/tmp/r.py', '-m', 'tensorfold', 'serve'], '/usr/bin/python3'), null); +}); + +test('lsof and /proc socket tables, loopback-only connect hosts, and Hugging Face cache paths', () => { + assert.deepEqual(parseLsofListen(LSOF_STUDIO), [{ host: '127.0.0.1', port: 54635 }]); + assert.deepEqual(parseLsofListen('p1\nf3\nn*:8080\nf4\nn[::1]:8081\n'), [{ host: '*', port: 8080 }, { host: '::1', port: 8081 }]); + assert.equal(parseLsofExecutable(`p1\nftxt\nn${STUDIO_EXE}\nftxt\nn/usr/lib/dyld\n`), STUDIO_EXE); + assert.equal(parseLsofExecutable(''), null); + const header = ' sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode\n'; + const tcp = `${header} 0: 0100007F:2649 00000000:0000 0A 00000000:00000000 00:00000000 00000000 1000 0 5551 1 0\n` + + ' 1: 0100007F:264A 0100007F:9C40 01 00000000:00000000 00:00000000 00000000 1000 0 5551 1 0\n' + + ' 2: 00000000:264B 00000000:0000 0A 00000000:00000000 00:00000000 00000000 1000 0 7777 1 0\n'; + assert.deepEqual(parseProcNetTcp(tcp, new Set(['5551'])), [{ host: '127.0.0.1', port: 9801 }], 'LISTEN rows of our inodes only'); + const tcp6 = `${header} 0: 00000000000000000000000001000000:2649 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 1000 0 42 1 0\n` + + ' 1: 0000000000000000FFFF00000100007F:264A 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 1000 0 43 1 0\n'; + assert.deepEqual(parseProcNetTcp(tcp6, new Set(['42', '43']), true), [{ host: '::1', port: 9801 }, { host: '::ffff:127.0.0.1', port: 9802 }]); + assert.equal(connectHost('*'), '127.0.0.1'); + assert.equal(connectHost('0.0.0.0'), '127.0.0.1'); + assert.equal(connectHost('::'), '::1'); + assert.equal(connectHost('127.0.0.2'), '127.0.0.2'); + assert.equal(connectHost('::ffff:127.0.0.1'), '127.0.0.1'); + assert.equal(connectHost('100.101.102.103'), null, 'a LAN or tailnet address is not read'); + assert.equal(connectHost('example.invalid'), null); + assert.equal(repoFromPath('/Users/me/.cache/huggingface/hub/models--Vontra--Qwen3.8-27B-MLX-4bit/snapshots/70ae7f'), 'Vontra/Qwen3.8-27B-MLX-4bit'); + assert.equal(repoFromPath('/tmp/plain'), null); +}); + +test('chooseSocket: the pid must listen where it says, on loopback, and unambiguously', () => { + const e = port => ({ port }); + assert.deepEqual(chooseSocket(e(9000), [{ host: '::1', port: 9000 }, { host: '*', port: 9000 }]), { host: '127.0.0.1', port: 9000, portFrom: 'args' }); + assert.match(chooseSocket(e(9000), [{ host: '127.0.0.1', port: 9001 }]).reason, /does not listen on its --port 9000/); + assert.match(chooseSocket(e(9000), [{ host: '192.0.2.7', port: 9000 }]).reason, /on a loopback address/); + assert.equal(chooseSocket(e(9000), []).quiet, true, 'not listening yet is not worth a line'); + assert.deepEqual(chooseSocket(e(null), [{ host: '127.0.0.1', port: 9100 }]), { host: '127.0.0.1', port: 9100, portFrom: 'socket' }); + assert.match(chooseSocket(e(null), [{ host: '127.0.0.1', port: 9100 }, { host: '127.0.0.1', port: 9101 }]).reason, /2 ports and names none/); + assert.match(chooseSocket(e(null), [{ host: '10.0.0.5', port: 9100 }]).reason, /only on non-loopback/); +}); + +// --- discovery as a whole --------------------------------------------------------- + +/** + * A fake exec: `ps` answers with `ps`; `lsof -d txt` with the pid's + * executable from `exe`; `lsof -iTCP` with its listen addresses from + * `listen`. Records every call. + */ +function fakeExec({ ps = PS_LIVE, exe = EXE, listen = {}, lsofMissing = false, psFails = false } = {}) { + const calls = []; + const exec = async (command, args) => { + calls.push([command, ...args]); + if (command === 'ps') return psFails ? { ok: false, stdout: '', error: { code: 'EPERM' } } : { ok: true, stdout: ps, error: null }; + if (command === 'lsof') { + if (lsofMissing) return { ok: false, stdout: '', error: { code: 'ENOENT' } }; + const pid = args[args.indexOf('-p') + 1]; + if (args.includes('txt')) { + return exe[pid] ? { ok: true, stdout: `p${pid}\nftxt\nn${exe[pid]}\nftxt\nn/usr/lib/dyld\n`, error: null } : { ok: false, stdout: '', error: { code: 1 } }; + } + const addrs = listen[pid] ?? []; + return addrs.length ? { ok: true, stdout: `p${pid}\n${addrs.map((a, i) => `f${i + 3}\nn${a}`).join('\n')}\n`, error: null } : { ok: false, stdout: '', error: { code: 1 } }; + } + throw new Error(`unexpected exec ${command}`); + }; + return { exec, calls }; +} + +const LISTEN = { + 1361: ['127.0.0.1:8091'], 10169: ['127.0.0.1:8101'], 14779: ['127.0.0.1:54635'], + 505: ['*:9123'], 506: ['127.0.0.1:9200'], 507: ['127.0.0.1:9700'], 508: ['127.0.0.1:9300'], + 509: ['127.0.0.1:9201'], 510: ['[::1]:9400'], 511: ['127.0.0.1:1234'], 512: ['127.0.0.1:1235'], + 513: ['192.0.2.7:9500'], +}; + +test('macOS discovery: one ps for this user, lsof for candidates only, each engine at a loopback socket its pid holds', async () => { + const { exec, calls } = fakeExec({ ps: `${PS_LIVE}${PS_DECOYS}`, listen: LISTEN }); + const r = await discoverTensorFoldEngines({ platform: 'darwin', exec, uid: 501 }); + assert.equal(r.ok, true); + assert.deepEqual(calls[0], ['ps', '-ww', '-U', '501', '-o', 'pid=,args=']); + assert.ok(calls.every(c => !c.includes('-E') && !c.includes('e')), 'never asks ps for environments'); + const asked = [...new Set(calls.filter(c => c[0] === 'lsof').map(c => c[c.indexOf('-p') + 1]))]; + assert.ok(!asked.some(pid => ['501', '502', '503', '504'].includes(pid)), 'grep, vim and friends are never inspected'); + assert.deepEqual(r.engines.map(e => [e.pid, e.port, e.portFrom, e.host]), [ + [1361, 8091, 'args', '127.0.0.1'], [10169, 8101, 'args', '127.0.0.1'], [14779, 54635, 'args', '127.0.0.1'], + [505, 9123, 'args', '127.0.0.1'], [506, 9200, 'args', '127.0.0.1'], [507, 9700, 'socket', '127.0.0.1'], + [508, 9300, 'args', '127.0.0.1'], [510, 9400, 'args', '::1'], + ]); + assert.deepEqual(r.skipped.map(s => [s.pid, s.reason]), [ + [509, 'does not listen on its --port 9200 on a loopback address; not read'], + [513, 'does not listen on its --port 9500 on a loopback address; not read'], + ]); + assert.doesNotMatch(JSON.stringify(r), /example\.invalid|192\.0\.2\.7|dummy-key/); +}); + +test('no --port and no socket yet: left out quietly; without lsof nothing is trusted and the scan says why', async () => { + const line = ' 700 /opt/tf/bin/tensorfold-native serve Vontra/Loading --name loading\n'; + const exe = { 700: '/opt/tf/bin/tensorfold-native' }; + const quiet = await discoverTensorFoldEngines({ platform: 'darwin', exec: fakeExec({ ps: line, exe }).exec, uid: 501 }); + assert.equal(quiet.ok, true); + assert.deepEqual(quiet.engines, [], 'still loading (or not an engine): no card until it listens'); + assert.deepEqual(quiet.skipped, []); + const blind = await discoverTensorFoldEngines({ platform: 'darwin', exec: fakeExec({ ps: line, exe, lsofMissing: true }).exec, uid: 501 }); + assert.equal(blind.ok, false, 'fail closed: no default port, no engine on its word'); + assert.match(blind.error, /cannot verify engines: lsof unavailable \(ENOENT\)/); + assert.deepEqual(blind.engines, []); +}); + +test('a failed process scan is ok:false with the reason, never a throw and never "no engines" silently', async () => { + const r = await discoverTensorFoldEngines({ platform: 'darwin', exec: fakeExec({ psFails: true }).exec, uid: 501 }); + assert.equal(r.ok, false); + assert.match(r.error, /ps failed/); + assert.deepEqual(r.engines, []); +}); + +test('Linux discovery reads /proc: this user only, comm first, exe for identity, sockets from fd links, never environ', async t => { + const root = await mkdtemp(join(tmpdir(), 'iak-tf-proc-')); + t.after(() => rm(root, { recursive: true, force: true })); + const header = ' sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode\n'; + const row = (addr, port, inode) => ` 0: ${addr}:${port.toString(16).toUpperCase()} 00000000:0000 0A 00000000:00000000 00:00000000 00000000 1000 0 ${inode} 1 0\n`; + const proc = async (pid, comm, exe, argv, { tcp = '', tcp6 = '', sockets = [] } = {}) => { + const dir = join(root, String(pid)); + await mkdir(join(dir, 'fd'), { recursive: true }); + await mkdir(join(dir, 'net')); + await writeFile(join(dir, 'comm'), `${comm}\n`); + await writeFile(join(dir, 'cmdline'), `${argv.join('\0')}\0`); + await writeFile(join(dir, 'environ'), `TENSORFOLD_API_KEY=${DUMMY_KEY}\0`); + await symlink(exe, join(dir, 'exe')); + await writeFile(join(dir, 'net', 'tcp'), `${header}${tcp}`); + await writeFile(join(dir, 'net', 'tcp6'), `${header}${tcp6}`); + for (const [i, inode] of sockets.entries()) await symlink(`socket:[${inode}]`, join(dir, 'fd', String(i + 3))); + }; + await proc(100, 'tensorfold-nati', '/opt/tf/bin/tensorfold-native', ['/opt/tf/bin/tensorfold-native', 'serve', 'Vontra/A', '--port', '9801'], + { tcp: row('0100007F', 9801, 5551), sockets: [5551] }); + await proc(101, 'tensorfold', '/usr/bin/python3.12', ['/usr/bin/python3', '/home/me/.venv/bin/tensorfold', 'serve', 'Vontra/B', '--port', '9802'], + { tcp6: row('00000000000000000000000000000000', 9802, 5552), sockets: [5552] }); + await proc(102, 'bash', '/bin/bash', ['/bin/bash', '-c', 'tensorfold-native serve Vontra/C --port 9803']); + await proc(103, 'python3', '/usr/bin/python3', ['/usr/bin/python3', '/tmp/report.py', 'tensorfold', 'serve', '--port', '9805']); + await proc(104, 'tensorfold-nati', '/opt/tf/bin/tensorfold-native', ['/opt/tf/bin/tensorfold-native', 'serve', 'Fake/Remote', '--host', '192.0.2.7', '--port', '9804'], + { tcp: row('070200C0', 9804, 5554), sockets: [5554] }); + await proc(105, 'tensorfold-nati', '/opt/tf/bin/tensorfold-native', ['/opt/tf/bin/tensorfold-native', 'serve', 'Fake/Other', '--port', '9806'], + { tcp: row('0100007F', 9806, 9999), sockets: [5556] }); + await mkdir(join(root, 'self')); + const reads = []; + const real = await import('node:fs/promises'); + const fs = { + readdir: p => real.readdir(p), + stat: p => real.stat(p), + readlink: p => real.readlink(p), + readFile: async (p, enc) => { reads.push(p); return real.readFile(p, enc); }, + }; + const r = await discoverTensorFoldEngines({ platform: 'linux', procRoot: root, uid: process.getuid(), fs, exec: fakeExec().exec }); + assert.deepEqual(r.engines.map(e => [e.port, e.host, e.model]), [[9801, '127.0.0.1', 'Vontra/A'], [9802, '::1', 'Vontra/B']]); + assert.deepEqual(r.skipped.map(s => s.pid), [104], 'a remote-only socket is not read, and says so'); + assert.ok(!r.engines.some(e => e.pid === 105), 'a --port whose socket another process holds is not read (quietly: it may still be loading)'); + assert.ok(!reads.some(p => p.endsWith('/environ')), 'never reads a process environment'); + assert.ok(!reads.some(p => p.endsWith('/102/cmdline')), 'argv is read only for candidate processes'); + const other = await discoverTensorFoldEngines({ platform: 'linux', procRoot: root, uid: process.getuid() + 1, fs, exec: fakeExec().exec }); + assert.deepEqual(other.engines, [], "another user's processes are not scanned"); +}); + +// --- @codexmb's attack, end to end: argv must not choose a host or a key file ------- + +test('a process that only claims to be an engine gets no request; a real one is read on loopback with no key from argv', async t => { + const { port, seen } = await tfServer(t, keyedRoutes()); + const decoyKey = await keyFile(t, `${DUMMY_KEY}\n`); + const ps = [ + ` 601 /usr/bin/printf /tmp/tensorfold-native serve Fake/Model --host example.invalid --port ${port} --api-key-file ${decoyKey}`, + ` 602 /opt/tf/bin/tensorfold-native serve Fake/Claims --host example.invalid --port ${port} --api-key-file ${decoyKey}`, + ` 603 /opt/tf/bin/tensorfold-native serve Fake/Remote --host 192.0.2.7 --port 9500 --api-key-file ${decoyKey}`, + ].join('\n'); + const exe = { 601: '/usr/bin/printf', 602: '/opt/tf/bin/tensorfold-native', 603: '/opt/tf/bin/tensorfold-native' }; + const listen = { 601: [`127.0.0.1:${port}`], 602: [`127.0.0.1:${port}`], 603: ['192.0.2.7:9500'] }; + const found = await discoverTensorFoldEngines({ platform: 'darwin', exec: fakeExec({ ps, exe, listen }).exec, uid: 501 }); + assert.deepEqual(found.engines.map(e => [e.pid, e.host, e.port]), [[602, '127.0.0.1', port]], 'printf is not an engine; the remote one is not read'); + + const urls = []; + const keyReads = []; + const { servers } = await collectTensorFoldStatus([], { + discovered: found.engines, + fetchImpl: (url, opts) => { urls.push(String(url)); return fetch(url, opts); }, + readKeyFile: async path => { keyReads.push(path); return { token: DUMMY_KEY, note: null }; }, + }); + assert.ok(urls.length > 0 && urls.every(u => u.startsWith(`http://127.0.0.1:${port}/`)), `only loopback was asked: ${urls.join(', ')}`); + assert.deepEqual(keyReads, [], 'no key file named on a command line is read'); + assert.ok(seen.every(r => r.auth === null), 'no Bearer header went anywhere'); + assertGetOnly(seen); + assert.equal(servers[0].metrics, METRICS_STATES.AUTH_REQUIRED); + assert.match(servers[0].reason, /--api-key-file, which is not read from a command line; add a models\.json entry for port \d+/); +}); + +test('a non-loopback address handed in as discovered is ignored, whatever the caller passes', async () => { + const urls = []; + const { servers, skipped } = await collectTensorFoldStatus([], { + discovered: [engine(9, { host: 'example.invalid', apiKeyFile: true }), engine(10, { host: '192.0.2.7' })], + fetchImpl: async url => { urls.push(String(url)); throw new Error('no network in this test'); }, + readKeyFile: async () => { throw new Error('no key file may be read'); }, + }); + assert.deepEqual([servers, skipped, urls], [[], [], []]); +}); + +// --- merging with the registry, and reading the engines ------------------------- + +test('merge by port: the registry keeps its id, an unnamed entry needs no detection, other engines stay skipped', async t => { + const a = await tfServer(t, openNativeRoutes()); + const b = await tfServer(t, openNativeRoutes()); + const c = await tfServer(t, openNativeRoutes()); + const d = await tfServer(t, openNativeRoutes()); + const entries = loadRegistry([ + { id: 'plain', host: 'localhost', port: a.port, kind: 'openai' }, + { id: 'marked', host: '127.0.0.1', port: b.port, kind: 'openai', engine: 'tensorfold' }, + { id: 'other', host: '127.0.0.1', port: c.port, kind: 'openai', engine: 'vllm' }, + ]); + const discovered = [ + engine(a.port, { name: 'bench', model: 'bench' }), + engine(b.port, { name: 'marked-engine' }), + engine(c.port), + engine(d.port, { name: STUDIO_MODEL, model: STUDIO_MODEL, app: STUDIO_APP_NAME }), + engine(d.port, { name: 'duplicate' }), + ]; + const { servers, skipped } = await collectTensorFoldStatus(entries, { discovered, scope: 'local' }); + assert.deepEqual(servers.map(s => [s.id, s.source, s.name, s.app, s.local]), [ + ['plain', 'discovered', 'bench', null, true], + ['marked', 'registry', 'marked-engine', null, true], + [STUDIO_MODEL, 'discovered', STUDIO_MODEL, STUDIO_APP_NAME, true], + ]); + assert.equal(skipped.length, 1); + assert.match(skipped[0].reason, /registry says engine "vllm" \(a TensorFold process listens on this port\)/); + assert.equal(c.seen.length, 0, 'an entry the registry calls something else is not asked'); + assert.ok(servers.every(s => s.metrics === METRICS_STATES.OK)); + for (const s of [a, b, d]) assertGetOnly(s.seen); + const plain = servers[0]; + assert.deepEqual(plain.draft, { drafted: 10231, accepted: 2580, acceptance: 0.252 }, 'the native bench recording reads through'); + assert.equal(plain.decode_tok_s, 0); + assert.equal(plain.memory.footprint_gb, 43.6); + assert.equal(plain.memory.used_gb, null, 'footprint is its own field, never passed off as engine memory'); + assert.deepEqual(plain.living_weights, { available: true, facts: 0, learning: 0, learned: 0 }); +}); + +test('the Studio engine: keyed /health, no key file, so metrics auth_required with the reason, and the model from its command line', async t => { + const { port, seen } = await tfServer(t, keyedRoutes()); + const { servers } = await collectTensorFoldStatus([], { + discovered: [engine(port, { name: STUDIO_MODEL, model: STUDIO_MODEL, app: STUDIO_APP_NAME })], + }); + const s = servers[0]; + assertGetOnly(seen); + assert.ok(seen.every(r => r.auth === null), 'no key exists to send, and none is invented'); + assert.equal(s.reachable, true); + assert.equal(s.metrics, METRICS_STATES.AUTH_REQUIRED); + assert.match(s.reason, /metrics: auth required; TensorFold Studio keeps this engine's key in memory only/); + assert.equal(s.model, STUDIO_MODEL); + assert.equal(s.model_source, 'args'); + assert.equal(s.warming, null); + assert.deepEqual(s.requests, { running: null, waiting: null }); + assert.equal(s.living_weights.available, null); +}); + +test('a keyed engine is read with the key the user configured: a marked registry entry on its port, gated routes only', async t => { + const { port, seen } = await tfServer(t, keyedRoutes()); + const path = await keyFile(t, `# engine keys\n\nops: ${DUMMY_KEY}\nother-key-not-used\n`); + const [marked] = loadRegistry([{ id: 'mine', host: '127.0.0.1', port, kind: 'openai', engine: 'tensorfold', keyFile: path }]); + const { servers } = await collectTensorFoldStatus([marked], { discovered: [engine(port, { name: 'keyed', apiKeyFile: true })] }); + assertGetOnly(seen); + const auth = Object.fromEntries(seen.map(r => [r.line, r.auth])); + assert.equal(auth['GET /health'], null); + assert.equal(auth['GET /metrics'], `Bearer ${DUMMY_KEY}`); + assert.equal(auth['GET /v1/slide/graph'], `Bearer ${DUMMY_KEY}`); + assert.deepEqual([servers[0].id, servers[0].source, servers[0].name], ['mine', 'registry', 'keyed']); + assert.equal(servers[0].metrics, METRICS_STATES.OK); + assert.equal(servers[0].tokens.generation_total, 6134); + assert.doesNotMatch(JSON.stringify(servers), new RegExp(DUMMY_KEY)); + + // An entry that does not say TensorFold keeps the old rule: probed without a key. + const before = seen.length; + const [unmarked] = loadRegistry([{ id: 'unmarked', host: '127.0.0.1', port, kind: 'openai', keyFile: path }]); + const keyReads = []; + await collectTensorFoldStatus([unmarked], { + discovered: [engine(port)], readKeyFile: async p => { keyReads.push(p); return { token: DUMMY_KEY, note: null }; }, + }); + assert.deepEqual(keyReads, []); + assert.ok(seen.slice(before).every(r => r.auth === null)); +}); + +test('key files the engine itself would refuse are refused; command-line keys are only explained', async t => { + const { port, seen } = await tfServer(t, keyedRoutes()); + const shared = await keyFile(t, `${DUMMY_KEY}\n`, 0o644); + const marked = keyFilePath => loadRegistry([{ id: 'e', host: '127.0.0.1', port, kind: 'openai', engine: 'tensorfold', keyFile: keyFilePath }]); + const cases = [ + [marked(shared), [engine(port)], /readable by other users; chmod 600/], + [marked(join(tmpdir(), 'iak-tf-no-such', 'keys')), [engine(port)], /cannot read key file .* \(ENOENT\)/], + [[], [engine(port, { keyOnArgv: true })], /given on the engine's command line; it is not read from there/], + [[], [engine(port)], /started without a key file/], + ]; + for (const [entries, discovered, why] of cases) { + const { servers } = await collectTensorFoldStatus(entries, { discovered }); + assert.equal(servers[0].metrics, METRICS_STATES.AUTH_REQUIRED); + assert.match(servers[0].reason, why); + } + assert.ok(seen.every(r => r.auth === null)); +}); + +test('readEngineKeyFile reads the engine format: comments, labels, bare keys, the first key wins', async t => { + assert.equal((await readEngineKeyFile(await keyFile(t, `# c\n lab: ${DUMMY_KEY} \nsecond\n`))).token, DUMMY_KEY); + assert.equal((await readEngineKeyFile(await keyFile(t, `\n${DUMMY_KEY}\n`))).token, DUMMY_KEY); + const empty = await readEngineKeyFile(await keyFile(t, '# nothing\n\n')); + assert.equal(empty.token, null); + assert.match(empty.note, /holds no key/); +}); + +// --- the heartbeat ----------------------------------------------------------------- + +test('monitor: discovery alone turns it on; the snapshot says whether discovery ran', async t => { + const { port } = await tfServer(t, openNativeRoutes()); + const scans = []; + const m = new TensorFoldMonitor({ + registryPath: '', discover: true, intervalMs: 60000, + discoverImpl: async () => { scans.push(1); return { ok: true, error: null, engines: [engine(port, { name: 'bench', model: 'bench' })] }; }, + }); + assert.equal(m.enabled, true); + assert.match(m.describe(), /^no registry .*discovery on/); + await m.refresh(); + const { tensorfold } = m.fields(); + assert.equal(tensorfold.error, null); + assert.equal(tensorfold.discovery, 'ok'); + assert.deepEqual(tensorfold.servers.map(s => [s.id, s.source, s.port]), [['bench', 'discovered', port]]); + assert.equal(scans.length, 1); + assert.throws(() => new TensorFoldMonitor({ discover: 'sometimes' }), /INTENT_TENSORFOLD_DISCOVER/); +}); + +test('monitor: a failed scan is "unavailable" on the card and warned once; a broken registry still shows discovered engines', async t => { + const { port } = await tfServer(t, openNativeRoutes()); + const warnings = []; + const failing = new TensorFoldMonitor({ + entries: [], discover: true, warn: w => warnings.push(w), + discoverImpl: async () => ({ ok: false, error: 'ps failed (EPERM)', engines: [] }), + }); + await failing.refresh(); + await failing.refresh(); + assert.equal(failing.fields().tensorfold.discovery, 'unavailable'); + assert.equal(warnings.length, 1); + assert.match(warnings[0], /engine discovery unavailable: ps failed \(EPERM\)/); + + const broken = new TensorFoldMonitor({ + registryPath: join(tmpdir(), 'iak-tf-no-such-registry', 'models.json'), warn: () => {}, + discoverImpl: async () => ({ ok: true, error: null, engines: [engine(port, { name: 'bench' })] }), + }); + await broken.refresh(); + const snap = broken.fields().tensorfold; + assert.match(snap.error, /^registry unreadable: /); + assert.deepEqual(snap.servers.map(s => s.id), ['bench'], 'discovery is on by default with a registry path, and still reports'); +}); diff --git a/test/tensorfold-status.test.mjs b/test/tensorfold-status.test.mjs index a5eae09..777f427 100644 --- a/test/tensorfold-status.test.mjs +++ b/test/tensorfold-status.test.mjs @@ -212,14 +212,17 @@ test('live recording end to end: the published object, GET only, no key sent', a port, local: true, source: 'registry', + name: null, + app: null, reachable: true, reason: null, model: LIVE_MODEL, + model_source: 'health', warming: false, version: null, backend: null, device: null, - memory: { used_gb: 18.7, peak_gb: 40.7, budget_gb: 96.2 }, + memory: { used_gb: 18.7, peak_gb: 40.7, budget_gb: 96.2, footprint_gb: 19.2 }, requests: { running: 0, waiting: 0 }, tokens: { prompt_total: 0, generation_total: 0 }, tok_s: null, @@ -271,7 +274,7 @@ test('401 on /metrics: "auth required", minimal /health reads as unknown, no key assert.equal(status.metrics, METRICS_STATES.AUTH_REQUIRED); assert.match(status.reason, /metrics: auth required/); assert.equal(status.model, null); - assert.deepEqual(status.memory, { used_gb: null, peak_gb: null, budget_gb: null }); + assert.deepEqual(status.memory, { used_gb: null, peak_gb: null, budget_gb: null, footprint_gb: null }); assert.deepEqual(status.requests, { running: null, waiting: null }, 'unread is null, never 0'); assert.equal(status.living_weights.available, null, 'a 401 graph is unknown, not absent'); assert.match(formatServer(status), /metrics: auth required/); @@ -437,7 +440,8 @@ const offSwitcher = { enabled: false, refresh: async () => ({}), start() {}, sto test('monitor: off by default, network-free, publishes nothing', async () => { for (const registryPath of ['', 'off', undefined]) { - const m = new TensorFoldMonitor({ registryPath, env: {}, fetchImpl: () => { throw new Error('network while off'); } }); + if (process.env.INTENT_TENSORFOLD_DISCOVER) break; + const m = new TensorFoldMonitor({ registryPath, fetchImpl: () => { throw new Error('network while off'); }, discoverImpl: () => { throw new Error('scan while off'); } }); if (registryPath === undefined && process.env.INTENT_TENSORFOLD_REGISTRY) continue; assert.equal(m.enabled, false); assert.deepEqual(m.fields(), {}); @@ -451,13 +455,17 @@ test('monitor: off by default, network-free, publishes nothing', async () => { test('monitor: an unreadable registry is an error on the card, not an empty list', async () => { const warnings = []; - const m = new TensorFoldMonitor({ registryPath: join(tmpdir(), 'no-such-dir-iak', 'models.json'), warn: w => warnings.push(w) }); + const m = new TensorFoldMonitor({ + registryPath: join(tmpdir(), 'no-such-dir-iak', 'models.json'), warn: w => warnings.push(w), + discoverImpl: async () => ({ ok: true, error: null, engines: [] }), + }); await m.refresh(); await m.refresh(); const { tensorfold } = m.fields(); assert.equal(tensorfold.schema, STATUS_SCHEMA); assert.match(tensorfold.error, /^registry unreadable: /); assert.deepEqual(tensorfold.servers, []); + assert.equal(tensorfold.discovery, 'ok', 'discovery still ran beside the broken registry'); assert.equal(warnings.length, 1, 'warned once, not once per beat'); }); @@ -471,8 +479,8 @@ test('monitor: reads the registry file each refresh and keeps the rate between r const path = join(dir, 'models.json'); await writeFile(path, JSON.stringify({ models: [{ id: 'mb-tf', host: 'localhost', port, kind: 'openai', engine: 'tensorfold' }] })); let clock = 0; - const m = new TensorFoldMonitor({ registryPath: path, scope: 'local', clock: () => clock, intervalMs: 30000 }); - assert.match(m.describe(), /scope local, every 30s, GET only/); + const m = new TensorFoldMonitor({ registryPath: path, scope: 'local', clock: () => clock, intervalMs: 30000, discover: false }); + assert.match(m.describe(), /scope local, every 30s, GET only, discovery off/); await m.refresh(); clock = 30_000; state.generation = 1600; @@ -545,20 +553,20 @@ test('read-out: human text says "unknown" for absent fields; --json is the heart t.after(() => rm(dir, { recursive: true, force: true })); const path = join(dir, 'models.json'); await writeFile(path, JSON.stringify({ models: [{ id: 'mb-tf', host: '127.0.0.1', port, kind: 'openai' }] })); - const human = await runTensorFoldStatus({ registryPath: path, callerHost: 'testhost', env: {} }); + const human = await runTensorFoldStatus({ registryPath: path, callerHost: 'testhost', discover: false }); assert.equal(human.exitCode, 0); assert.match(human.text, /mb-tf {2}127\.0\.0\.1:\d+ {2}reachable {2}\(auto-detected\)/); assert.match(human.text, /version: unknown {3}backend: unknown {3}device: unknown/); assert.match(human.text, /memory: 18\.7 GB used, 40\.7 GB peak, 96\.2 GB budget/); assert.match(human.text, /tok\/s: needs two samples/); assert.match(human.text, /living weights: not available/); - const json = await runTensorFoldStatus({ registryPath: path, callerHost: 'testhost', json: true, env: {} }); + const json = await runTensorFoldStatus({ registryPath: path, callerHost: 'testhost', json: true, discover: false }); const report = JSON.parse(json.text); assert.equal(report.schema, STATUS_SCHEMA); assert.equal(report.servers[0].model, LIVE_MODEL); assertGetOnly(seen); - const missing = await runTensorFoldStatus({ registryPath: join(dir, 'nope.json'), callerHost: 'testhost' }); + const missing = await runTensorFoldStatus({ registryPath: join(dir, 'nope.json'), callerHost: 'testhost', discover: false }); assert.equal(missing.exitCode, 2); assert.match(missing.text, /cannot read registry/); }); @@ -571,15 +579,16 @@ test('cli: ide-agent-kit tensorfold-status --json runs end to end', async t => { await writeFile(path, JSON.stringify({ models: [{ id: 'mb-tf', host: '127.0.0.1', port, kind: 'openai', engine: 'tensorfold' }] })); const env = { ...process.env }; delete env.INTENT_TENSORFOLD_REGISTRY; + delete env.INTENT_TENSORFOLD_DISCOVER; const run = args => new Promise(resolve => { execFile(process.execPath, [join(ROOT, 'bin', 'cli.mjs'), 'tensorfold-status', ...args], { cwd: dir, env, timeout: 20000 }, (err, stdout, stderr) => resolve({ code: err ? err.code : 0, stdout, stderr })); }); - const ok = await run(['--registry', path, '--json']); + const ok = await run(['--registry', path, '--json', '--no-discover']); assert.equal(ok.code, 0, ok.stderr); const report = JSON.parse(ok.stdout); assert.equal(report.servers[0].id, 'mb-tf'); assert.equal(report.servers[0].source, 'registry'); - const bad = await run(['--registry', join(dir, 'missing.json')]); + const bad = await run(['--registry', join(dir, 'missing.json'), '--no-discover']); assert.equal(bad.code, 2); });