test/mcp-server.test.ts: a timed-out RPC is returned as a partial result, so a slow server and a crashed one fail identically #350

rpc() in test/mcp-server.test.ts gives up after a fixed 15s, returns whatever arrived, and throws the child's stderr away. When it gives up, the four tests built on it fail like this:

✖ a built repo advertises every tool (16835.2945ms)
  TypeError: Cannot read properties of undefined (reading 'result')
      at listTools (test/mcp-server.test.ts:80:35)

Nothing in that says "the server did not answer in time". It is the shape you get from rs.find(r => r.id === 2) returning undefined — and you get the identical error whether the server was slow, crashed on startup, or answered something malformed.

const child = spawn(process.execPath, ['--import', 'tsx', 'src/cli.ts', 'mcp', dir], { stdio: ['pipe', 'pipe', 'pipe'] });
…
const deadline = Date.now() + 15000;
while (responses.length < expected && Date.now() < deadline) await new Promise((r) => setTimeout(r, 50));
child.kill();
await once(child, 'exit').catch(() => {});
return responses;                    // ← partial, silently

Two things make it hard to diagnose rather than merely slow:

• The partial result is returned, not raised. responses.length < expected is known right there and thrown away; every caller then dereferences into undefined.
• stderr is piped and never read. If the child died — a bad import, a missing native build, a throw during startup — the message that says so goes nowhere. A crash and a timeout are indistinguishable from the test output.

How often

Locally, on a Windows 11 box that was also doing other work: 2 of 5 full-suite runs. Failing subset varied — four of the tests in one run, two in another, none in the other three. Run the file alone and it passes every time, in 3–5s per test.

CI has not seen it: the last 20 ci.yml runs on main are green. It is a developer-machine problem, and the cost is that the failure looks like a real defect in the MCP server until you notice the 15s in the timing column.

Headroom

I measured time-to-first-response, spawning the server exactly as the test does (5 spawns each, idle machine):

1738ms  1881ms  1976ms  2019ms  1775ms

So ~2s against a 15s deadline — 7x, which sounds ample and is not, once ~30 test files are running in parallel and each of those spawns children of its own. The two failing runs came in at 15.3–17.6s, i.e. the deadline expiring rather than anything running long.

Disclosure, since it is the same neighbourhood: #337 adds a test file that spawns three short-lived CLI processes, so it does add parallel load. It does not make the server slower — the numbers above are the same on that branch and on main — but it is one more thing competing for the machine, which is how I ran into this.

What would help

Not the number, particularly — the diagnosis:

1. On expiry, throw instead of returning short: how many of expected arrived, and how long it waited.
2. Collect the child's stderr and put it in that error. That alone separates "slow" from "crashed", which is the distinction the current output cannot make.
3. Then the deadline can be as generous as you like, because a real hang reports itself properly instead of masquerading as a missing tool.

Happy to send a patch if you want one — it is a dozen lines in the helper and touches nothing else.

Environment

graft main (de8456e, 0.18.0), Node v24.16.0, Windows 11 x64, 5 local npm test runs.