Discover running TensorFold engines, including TensorFold Studio's

CHANGELOG.md

@@ -3,6 +3,7 @@
 ## Unreleased
 
 ### Added
+- **TensorFold engines are discovered from the process table**: the TensorFold status reader (heartbeat and `ide-agent-kit tensorfold-status`) now also finds this user's running `tensorfold-native serve` / `tensorfold serve` engines (one `ps` on macOS, `/proc` on Linux, no sudo), merges them with the registry by port and publishes them as `source: "discovered"`, so TensorFold Studio's engine on its random port gets a card. A command line is treated as a claim, not configuration: what a process is comes from its real executable (`lsof -d txt` on macOS, `/proc/<pid>/exe` on Linux), and it is read only at a loopback or wildcard TCP socket its own pid holds in LISTEN state, never at a `--host`; a `--port` it does not listen on, several ports with no `--port`, a non-loopback-only socket, or no way to check (no `lsof`) means the engine is not read. On by default with `INTENT_TENSORFOLD_REGISTRY`; `INTENT_TENSORFOLD_DISCOVER=1` runs it without a registry, `=0` or `--no-discover` turns it off. No key is ever taken from a command line, a key file a command line names, or a process environment: a discovered engine gets a key only through a registry entry marked `"engine": "tensorfold"` on its port with a `keyFile`, so the Studio engine (key in memory only) reports `metrics: "auth_required"` with its model from the command line. New fields: `name`, `app`, `model_source`, `memory.footprint_gb`, and a top-level `discovery` (`ok` / `off` / `unavailable`). Key files for TensorFold status reads, including a registry `keyFile`, are now refused when other users can read them, as the engine refuses them.
 - **TensorFold status on the device heartbeat**: `uik-daemon` and `ide-agent-kit intent daemon` can publish the live status of each TensorFold server in the model registry under the device's `tensorfold` key, so CodeWatch can show a card per machine. Opt in with `INTENT_TENSORFOLD_REGISTRY=<path to models.json>`; `INTENT_TENSORFOLD_SCOPE=local` (default, loopback entries only) or `all`. Read with GET `/health`, `/metrics` and `/v1/slide/graph` only, never a completion. Publishes model, warming, version/backend/device when the server reports them, memory, running and waiting requests, token counters, tok/s from counter deltas between reads, draft acceptance, TTFT p50, KV cache use, Living Weights fact counts (never fact text) and the metrics auth state. An unreachable server is `reachable: false`; an unreadable registry is an `error`, never an empty list. New `ide-agent-kit tensorfold-status [--json]` prints the same object. The registry gains `engine` (`"tensorfold"` marks an entry, any other name skips it, absent means auto-detect). A key is sent only to an entry marked TensorFold that names its own `keyFile`, and only on the two gated routes; the fleet-wide `LLM_API_KEY` fallbacks are never used for a status read.
 - **What is still waiting on you** (`bin/iak-pending.mjs`): asks every daemon the fleet knows about - this machine's plus every `watchdog-roster.json` entry with a `gate` - for its pending intents and prints them oldest first with the age in the leading column. Motivation: confirmation cards post to the room correctly and then scroll away; on 2026-09-20 four approval requests sat pending on one machine for up to 9 hours 12 minutes and the owner asked twice in one night what was waiting on him, because no surface anywhere could answer it. A host that could not be reached is reported under its own `COULD NOT ASK` heading and is never folded into "nothing pending"; a gate naming another machine by LAN IP is refused rather than probed and reported the same way. A daemon whose rows carry no `status` this version understands is a shape mismatch rather than an empty queue, and a fleet roster that exists but cannot be parsed or read exits `2` instead of silently shrinking the fleet to this machine (only a *missing* roster means "single-machine install"). An age that cannot be believed - missing, future-dated, or epoch-seconds - renders as `age unknown`, sorts to the top and survives `--older-than`, rather than being clamped to "fresh". Exit codes: `0` all clear, `10` pending, `11` some hosts unreachable, `12` no host reachable, `2` cannot run, precedence `12 > 11 > 10 > 0`; `--older-than` narrows what is printed and announced but can never produce a `0`. `--json` for machines. `--room` requires `--older-than` and announces each item at most once per escalation (threshold, then each doubling of its age), keeping a ledger at `~/.iak/pending-announced.json` that is written atomically before the post, carries forward entries for hosts it could not ask, rolls back when a post fails, and degrades to silence - never a storm - when it is missing, corrupt or unwritable. `examples/pending-launchd.plist` wires it to a timer. The gate token is resolved by the existing `resolveGateToken` helper, is sent only to hosts that helper already trusts, and never appears in output or argv.
 - **Session degradation watch** (`bin/iak-degradation-watch.mjs`): a per-machine daemon that tails watched agents' Claude Code session transcripts and posts a room alert (pushed to the owner's phone via CodeWatch) the moment a session's `message.model` leaves the configured accepted set or its `permissionMode` differs from the expected one. Motivation: on 2026-07-18 an agent's session silently slipped to a small fallback model with manual permission prompts while the owner was away; the degraded agent kept posting — looping on stale tasks and reposting completed instructions — and cost an evening to diagnose. Alerts fire immediately on first detection (a wrong model in a transcript is a fact, not a flap) with a 30-minute per-signal cooldown and a recovery note. Config: `degradation_watch` block (`interval_sec`, `alert_room`, `agents[]` with `transcript_dir`, `accepted_models` prefix list, `expected_permission_mode`); API key reuses `poller.api_key`. `--once` and `--dry-run` flags for testing. Run one instance per machine under process supervision.

README.md

@@ -496,7 +496,9 @@ can show a card for it on each machine.
 
 1. List the server in `config/models.json` with `"engine": "tensorfold"` (or
    leave `engine` out and let it be auto-detected). See
-   `config/models.README.md`.
+   `config/models.README.md`. Engines running on this machine, including the
+   one TensorFold Studio starts on a random port, are found from the process
+   table without an entry.
 2. Check what the card will say: `ide-agent-kit tensorfold-status` (add
    `--json` for the exact object).
 3. Turn it on for the publisher: set
@@ -505,7 +507,9 @@ can show a card for it on each machine.
 
 It is read with GET requests only (`/health`, `/metrics`, `/v1/slide/graph`),
 never a completion, and an unreachable server shows as unreachable instead of
-stopping the heartbeat. The payload shape is in
+stopping the heartbeat. Keys are read only from a registry entry's
+`keyFile`, never from a command line; TensorFold Studio keeps its engine key
+in memory, so its card says `metrics: auth required`. The payload shape is in
 `packages/user-intent-kit/README.md`.
 
 UIK v0.2.2 or later recommended for the `uik-daemon` bin. v0.2.0 introduced the daemon, v0.2.1 fixed a silent-exit in the example file, and v0.2.2 added agent-status republish on the heartbeat cadence so the agent slot does not expire while the daemon is running. Pre-0.2.2 daemons publish device state correctly but the agent slot still goes stale after its TTL.
@@ -1207,7 +1211,7 @@ ide-agent-kit gateway <health|agents|trigger|wake> [options]
 ide-agent-kit memory <list|get|set|append|delete|search> [options]
 ide-agent-kit rooms archive [--backfill] [--max-pages N] [--room R]
 ide-agent-kit rooms search "words" [--from handle] [--since 2026-09-20] [--regex] [--limit N]
-ide-agent-kit tensorfold-status [--json] [--registry <path>] [--local] [--timeout <ms>]
+ide-agent-kit tensorfold-status [--json] [--registry <path>] [--local] [--no-discover] [--timeout <ms>]
 ide-agent-kit init [--ide <claude-code|codex|cursor|vscode|gemini>] [--profile <balanced|low-friction>]
 ide-agent-kit acp <spawn|list|status|send|close> [options]
 ide-agent-kit keepalive <start|stop|status> [--pid-file <path>] [--heartbeat-sec <sec>]

bin/cli.mjs

@@ -102,9 +102,10 @@ Usage:
     daemon:    long-running publisher (desktop state + agent heartbeat) [--interval <sec>]
                Add INTENT_TENSORFOLD_REGISTRY=<models.json> to publish TensorFold status on each beat.
 
-  ide-agent-kit tensorfold-status [--json] [--registry <path>] [--local] [--timeout <ms>] [--config <path>]
-    Live status of every TensorFold server in the model registry, read with GET only
-    (/health, /metrics, /v1/slide/graph). Same object the device heartbeat publishes.
+  ide-agent-kit tensorfold-status [--json] [--registry <path>] [--local] [--no-discover] [--timeout <ms>] [--config <path>]
+    Live status of every TensorFold server in the model registry and every TensorFold engine
+    running on this machine, read with GET only (/health, /metrics, /v1/slide/graph).
+    Same object the device heartbeat publishes.
 
   ide-agent-kit poll --rooms <room1,room2> --api-key <key> --handle <@handle> [--interval <sec>] [--config <path>]
     (Legacy) Poll GroupMind rooms with explicit CLI args. Prefer "rooms watch".
@@ -578,7 +579,12 @@ async function main() {
       console.error('Error: --timeout must be a positive number of milliseconds');
       process.exit(2);
     }
-    const result = await runTensorFoldStatus({ registryPath, scope: opts.local ? 'local' : 'all', json: !!opts.json, timeoutMs });
+    const envRegistry = String(process.env.INTENT_TENSORFOLD_REGISTRY || '').trim();
+    const registryRequired = typeof opts.registry === 'string' || (envRegistry !== '' && !['0', 'off', 'false', 'no'].includes(envRegistry.toLowerCase()));
+    const result = await runTensorFoldStatus({
+      registryPath, registryRequired, discover: !opts['no-discover'],
+      scope: opts.local ? 'local' : 'all', json: !!opts.json, timeoutMs,
+    });
     (result.exitCode === 0 ? console.log : console.error)(result.text);
     process.exit(result.exitCode);
   }

config/models.README.md

@@ -82,13 +82,31 @@ Which entries count as TensorFold:
   both, so mark a keyed server `"engine": "tensorfold"` explicitly.
 * any other `engine`: never asked.
 
+Engines running on this machine do not need an entry at all: the status
+reader also finds this user's `tensorfold-native serve` / `tensorfold serve`
+processes and reads them the same way (`source: "discovered"`). It checks the
+process's real executable, not just its command line, and reads it only at a
+loopback TCP socket that process is listening on; `--host` is ignored. An
+entry on the same loopback port wins: its `id` is used (and its `keyFile`
+when it is marked `"engine": "tensorfold"`), and an entry with no `engine`
+needs no auto-detection. An entry with another `engine` on that port stays
+skipped. Turn discovery off with `INTENT_TENSORFOLD_DISCOVER=0` or
+`tensorfold-status --no-discover`.
+
 **Keys.** A TensorFold server may gate `/metrics` and `/v1/slide/graph`
 behind a key (`/health` stays open, and says only `{"status": "ok"}` while
-keys are on). The status reader sends a key only to an entry marked
-`"engine": "tensorfold"` that names its own `keyFile`, and only on those two
-routes. It never uses `LLM_API_KEY` / `LLM_API_KEY_FILE`: those belong to some
-other server as far as a status read knows. Without a usable key it reports
-`metrics: "auth_required"` and the rest of the card reads unknown.
+keys are on). The status reader sends a key only on those two routes, and
+only from the `keyFile` of an entry marked `"engine": "tensorfold"`. The file
+is read the way the engine reads it (one key per line, `label: key` or a
+bare key, `#` comments) and refused, as the engine refuses it, when other
+users can read it (`chmod 600`). It never uses `LLM_API_KEY` /
+`LLM_API_KEY_FILE`, a key on a command line, a key file a command line names
+(`--api-key-file`: any process can name any file there), or a process's
+environment. For a keyed engine you started yourself, add an entry for its
+port with `"engine": "tensorfold"` and the same file as `keyFile`. Without a usable key it reports `metrics: "auth_required"` and
+the rest of the card reads unknown. TensorFold Studio's engine is always in
+that state: the app makes a fresh key on every launch, keeps it in memory and
+passes it to the engine in the environment, never in a file.
 
 ```json
 {

packages/user-intent-kit/README.md

@@ -132,11 +132,25 @@ device card:
 ```bash
 export INTENT_TENSORFOLD_REGISTRY=~/ide-agent-kit/config/models.json
 export INTENT_TENSORFOLD_SCOPE=local   # default; `all` also reads remote entries
+export INTENT_TENSORFOLD_DISCOVER=1    # default with a registry; 0 turns it off; 1 alone runs without one
 ```
 
 - **GET only.** `/health`, `/metrics` and `/v1/slide/graph`, never a
   completion, so a status read cannot interfere with a Living Weights lesson.
-- **Off unless set.** Nothing is read or published without the registry path.
+- **Off unless set.** Nothing is read or published without the registry path
+  or `INTENT_TENSORFOLD_DISCOVER=1`.
+- **Running engines are found too.** Each read also scans this user's
+  processes for `tensorfold-native serve` and `tensorfold serve` (one `ps` on
+  macOS, `/proc` on Linux, no sudo). A command line is only a claim: the
+  process must really be running `tensorfold-native` (or Python running
+  TensorFold), checked against its executable (`lsof -d txt`,
+  `/proc/<pid>/exe`), and it is read only at a loopback TCP socket it is
+  listening on, matching its `--port` when it names one; `--host` is
+  ignored, and without `lsof` nothing is read. The model and name come from
+  the command line. Engines are merged with the registry by port and
+  published as `source: "discovered"`. This is how TensorFold Studio's
+  engine, which gets a new random port every launch, gets a card. Process
+  environments are never read.
 - **Not on the heartbeat's path.** It reads on its own timer at the heartbeat
   interval; the beat publishes the last reading and never waits on a socket.
 - **One key, replaced whole.** Everything goes under `tensorfold`, so a server
@@ -146,6 +160,13 @@ export INTENT_TENSORFOLD_SCOPE=local   # default; `all` also reads remote entrie
 - **Which entries:** marked `"engine": "tensorfold"`, or auto-detected from
   their `/metrics`. See `config/models.README.md` in ide-agent-kit, including
   how a keyed server names its key.
+- **Keys come from the registry only.** The `keyFile` of an entry marked
+  `"engine": "tensorfold"`, read in the engine's key-file format and refused
+  when other users can read the file. Never a command line, a key file a
+  command line names, or an environment. TensorFold Studio keeps its engine
+  key in memory and hands it to the engine through the environment, so its
+  engine shows `metrics: "auth_required"`, the model from its command line,
+  and the reason.
 - **TensorFold's default port is 8080, and so is `INTENT_MODEL_ENDPOINT`'s.**
   The served-model probe above may send one short generation request to a
   loopback endpoint to prove what is loaded. On a machine where TensorFold
@@ -160,12 +181,13 @@ The payload, per device:
   "checked_at": "2026-10-14T09:00:00.000Z",
   "scope": "local",
   "error": null,
+  "discovery": "ok",
   "servers": [{
     "id": "mb-tensorfold", "host": "localhost", "port": 8091, "local": true,
-    "source": "registry", "reachable": true, "reason": null,
-    "model": "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit", "warming": false,
+    "source": "registry", "name": null, "app": null, "reachable": true, "reason": null,
+    "model": "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit", "model_source": "health", "warming": false,
     "version": null, "backend": null, "device": null,
-    "memory": { "used_gb": 18.7, "peak_gb": 40.7, "budget_gb": 96.2 },
+    "memory": { "used_gb": 18.7, "peak_gb": 40.7, "budget_gb": 96.2, "footprint_gb": 19.2 },
     "requests": { "running": 0, "waiting": 0 },
     "tokens": { "prompt_total": 0, "generation_total": 0 },
     "tok_s": null, "tok_s_window_s": null, "decode_tok_s": null,
@@ -184,10 +206,19 @@ and "could not tell" never look the same.
 
 What the numbers mean:
 
-- `local`: the server runs on this machine (loopback entry). `source` is
-  `registry` (marked `"engine": "tensorfold"`) or `detected`.
+- `local`: the server runs on this machine (a loopback entry or a discovered
+  engine). `source` is `registry` (marked `"engine": "tensorfold"`),
+  `detected` (auto-detected from its `/metrics` or `/health`) or `discovered`
+  (found running in the process table). `name` is the engine's `--name`;
+  `app` is `"TensorFold Studio"` for the engine the app bundles, else null.
+- `model_source`: `health` when `/health` named the model, `args` when it came
+  from the engine's command line (a keyed `/health` says only "ok").
+- `discovery` (top level): `ok`, `off`, or `unavailable` when the process scan
+  failed. An `unavailable` scan is not the same as "no engines running".
 - `memory`: GB, base 10, one decimal, like `mem_total_gb`. `used` is the
-  engine's active memory, `budget` its configured budget.
+  engine's active memory, `budget` its configured budget, `footprint` the
+  process's physical footprint (macOS, including Metal buffers). The native
+  MLX build reports only the footprint.
 - `tokens`: counters since the server started, for finished requests.
 - `tok_s`: generated tokens of requests that finished between two reads,
   divided by the seconds between them (`tok_s_window_s`). `null` on the first

packages/user-intent-kit/bin/uik-daemon.js

@@ -76,6 +76,16 @@
  *                        Never a generation request. An unreachable server
  *                        publishes reachable:false; the heartbeat never
  *                        waits on it.
+ *   INTENT_TENSORFOLD_DISCOVER 1 | 0. Find TensorFold engines running on this
+ *                        machine from the process table (this user's
+ *                        `tensorfold-native serve` / `tensorfold serve`,
+ *                        checked against the real executable, read only at
+ *                        a loopback socket the process listens on, never
+ *                        with a key from its command line) and publish them
+ *                        as source "discovered", merged with the registry by
+ *                        port. Default: on when
+ *                        INTENT_TENSORFOLD_REGISTRY is set; set 1 to run it
+ *                        with no registry at all, 0 to turn it off.
  *   INTENT_TENSORFOLD_SCOPE local (default) | all. local = only entries on
  *                        loopback, i.e. servers on THIS machine, so each
  *                        machine's card carries its own servers. all = every

packages/user-intent-kit/src/index.js

@@ -17,6 +17,7 @@ export {
   STATUS_SCHEMA as TENSORFOLD_STATUS_SCHEMA,
   METRICS_STATES as TENSORFOLD_METRICS_STATES,
 } from './tensorfold-status.js';
+export { discoverTensorFoldEngines } from './tensorfold-discovery.js';
 export {
   ModelAvailabilityProbe,
   describeLocalModels,