Benchmarks and Showcase fail with "fetch failed" on Node 22: undici 8.x Agent rejected by built-in fetch (UND_ERR_INVALID_ARG)

[Saolence]

Summary

On Node 22, every SSE-streaming path that goes through runStreamingRequest() fails immediately with fetch failed:

• Decode benchmark (POST /api/sparks/:id/llm/bench)
• Prefill benchmark (POST /api/sparks/:id/llm/prefill-bench)
• Prompt Showcase

All waves report streamsOk: 0, streamsFailed: N, and the job finishes in ~1 ms. Live monitoring (LlmProbe) keeps working, which makes it look like a network problem — it is not.

Environment

Item	Value
sparkDash	39c7f2b (Merge PR #85), v1.8.6
Container base	node:22-bookworm-slim (Dockerfile ARG NODE_IMAGE)
Node	v22.23.2
Built-in undici	6.28.0
package.json dependency	"undici": "^8.9.0" (resolves to 8.9.0)
Host	x86_64, Docker network_mode: host

Steps to reproduce

1. Start sparkDash with an LLM reachable on the LAN, e.g. http://192.168.50.168:8888.
2. Confirm the LLM itself is reachable from inside the container (this succeeds):

// plain built-in fetch, no dispatcher
await fetch("http://192.168.50.168:8888/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ model: "…", messages: [{ role: "user", content: "hi" }], max_tokens: 8 }),
});
// → HTTP 200

3. Run the benchmark:

curl -X POST http://localhost:5555/api/sparks/<id>/llm/bench \
  -H 'Content-Type: application/json' \
  -d '{"port":8888,"concurrencies":[1,8,16,32],"maxTokens":400}'

4. Result — every wave fails, first one in under a millisecond:

c=1  ok=0 fail=1  err='fetch failed' dur=0.81
c=8  ok=0 fail=8  err='fetch failed' dur=5.07
c=16 ok=0 fail=16 err='fetch failed' dur=6.92
c=32 ok=0 fail=32 err='fetch failed' dur=7.57

Root cause

server/collectors/LlmStreaming.js creates an Agent from the npm undici package (8.9.0) and passes it as the dispatcher to the Node built-in fetch (which is undici 6.28.0). Node 22's built-in fetch does not accept an 8.x Dispatcher instance and rejects it with UND_ERR_INVALID_ARG before any socket is opened.

Minimal reproduction (inside the container, with undici@8.9.0 installed):

import { Agent } from "undici";          // 8.9.0
const agent = new Agent({ headersTimeout: 0, bodyTimeout: 0 });
await fetch("http://192.168.50.168:8888/v1/models", { dispatcher: agent });
// TypeError: fetch failed — cause: UND_ERR_INVALID_ARG

Cross-checks (same container, same target URL):

Combination	Result
built-in fetch, no dispatcher	✅ HTTP 200
undici@8.9.0 Agent → built-in fetch	❌ UND_ERR_INVALID_ARG
undici@6.28.0 Agent → built-in fetch	✅ HTTP 200
undici@8.9.0 Agent → undici@8.9.0 own fetch	✅ HTTP 200

Note that even new Agent() with default options is rejected — the failure is the instance/version boundary, not the headersTimeout: 0 options.

Where it was introduced

e03b9d6 — "feat: hide worker nodes and add remote decode/prefill benches" (2026-09-07) is the first commit that adds both the undici dependency and the dispatcher:

• server/collectors/LlmStreaming.js:8 — import { Agent } from "undici";
• server/collectors/LlmStreaming.js:23-26 — LLM_STREAM_AGENT = new Agent({ headersTimeout: 0, bodyTimeout: 0 })
• server/collectors/LlmStreaming.js:564 — dispatcher: LLM_STREAM_AGENT
• package.json — "undici": "^8.9.0"

Before that commit the streaming paths used the built-in fetch only, so benchmarks worked. Deployments still on a pre-e03b9d6 build are unaffected, which is why this can regress silently on upgrade.

The intent of the agent is legitimate: headersTimeout: 0 / bodyTimeout: 0 disables undici's 300 s idle cut so long prefill benchmarks (30–45 min caller-side timeout) are not aborted. The problem is only which package's fetch receives it.

Affected code paths

• DecodeBench.js → runStreamingRequest
• PrefillBench.js → runStreamingRequest
• ShowcaseManager.js → runStreamingRequest

LlmProbe is unaffected because it calls fetch without a dispatcher — this is why the dashboard still shows live metrics while every benchmark fails.

Suggested fixes

1. Use the matching fetch — import fetch from the same undici package that owns the Agent, so both sides come from one module instance:

   import { Agent, fetch as undiciFetch } from "undici";
   // ...
   const response = await undiciFetch(url, { ..., dispatcher: LLM_STREAM_AGENT });

   Verified working on the same Node 22 container (HTTP 200).

2. Pin undici to the 6.x line (e.g. ^6.28.0) so the dispatcher matches Node 22's built-in undici. CVE-2026-12151 is fixed in >= 6.27.0, so 6.28.0 does not reintroduce it — but this does move the dependency away from the 8.x line chosen in fix: upgrade undici to patched version (CVE-2026-12151) #81.

3. Move to a Node image whose built-in undici matches the npm dependency.

Option 1 keeps the current dependency version and the long-timeout behaviour, so it looks like the smallest correct change.

Additional: the error message hides the cause

describeStreamFetchError() (server/collectors/LlmStreaming.js) returns err.message for unrecognised errors, which for undici is just "fetch failed". The actionable detail lives in err.cause.code (UND_ERR_INVALID_ARG). Including the cause code in the returned string would have made this self-diagnosing from the UI:

const code = err.code || err.cause?.code;
// ...
return code ? `${err.message} (${code})` : (err.message || String(err));

Happy to open a PR for either the fix or just the error-message improvement if that helps.

[Lesilva]

Fixed on main in f035ca2: runStreamingRequest() now calls undici's own fetch with the npm Agent instead of passing it as dispatcher to Node's global fetch — the exact mismatch you diagnosed (Node 22's built-in fetch is a different undici build, so the Agent was rejected with UND_ERR_INVALID_ARG). Regression test: server/collectors/__tests__/LlmStreaming.tokens.test.js ("undici Agent works with undici fetch (not Node global fetch)").

Two related fixes landed on the same path afterwards: b950136 (a few retries 429ing the bench) and d0c7f71 (full 1x–32x concurrency sweep against the post-1.8.6 work cap, #93).

One caveat worth stating: this is on main but not in a release — 1.8.6 predates it, so an unchanged 1.8.6 container still fails exactly as you reported. @MikeGibbsOnyx, main has now accumulated a few unreleased fixes that open reports are still hitting (this one, #73, #93, the prefill-bench idle timeout); a 1.8.7 tag would close them with no further code work.