Repository navigation
test: pin that two chats stream at once through the gateway - #14258
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the test stays green under a loaded shared CI runner — the order assertion's margin is one full 15s slow turn on warm slots. [DESIGN-REVIEWED] 4e942b1 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ⏭️ skippedRevision |
940a065 to
df4f2e4
Compare
|
|
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
df4f2e4 to
d1dad73
Compare
d1dad73 to
660029c
Compare
660029c to
0caa52d
Compare
|
b7310da to
33c2491
Compare
|
33c2491 to
1f5db1f
Compare
1f5db1f to
133c354
Compare
|
Fourth slice of the in-process integration layer. One test in test/integration/test_streaming.py drives two real POST /api/chat SSE streams on two slots with the fake model's [[SLOW]] turn (thirty chunks half a second apart) and asserts the fairness contract behind bug 16: slot B, sent once slot A is streaming, must produce its first chunk before A produces its last, and A must keep producing chunks while B starts. The report was the opposite -- one session streams, every other one sits in "thinking" until it finishes -- and the fix landed as #13203; this is the regression pin for it. Nothing inside the process is patched or timed, only what the two SSE clients see, and the assertions are on ORDER, never on absolute latency (a cold session start costs several seconds before the first chunk). Bug 17 (session/new timing out with every MCP server reported) is not in this slice: agent.session_start_timeout_secs is clamped to at least 90 seconds, so a hang test costs a minute and a half per attempt, and the wording fix that separates the MCP count from the verdict is already on main. Its pin belongs at the unit level. Route coverage moves to 9/1247 and the ratchet tightens from 0.5% to 0.7%. The conventions doc gains the SSE reading pattern. The harness also grants the disposable home unsandboxed-exec consent, as the E2E suite does: the CI container has no sandbox backend, and without it every child the gateway spawns is refused before the fake model runs.
133c354 to
4e942b1
Compare
Problem / Motivation
Goal: Two chats on two slots stream at the same time through the gateway, and a regression to one-at-a-time streaming fails CI.
Bug 16 reported that one session streams while every other one sits in "thinking" until it finishes, then a blob arrives. The fix landed as #13203 with no test that drives two real chat turns concurrently and watches both SSE streams.
Why it matters
Concurrent streaming is the difference between a usable multi-session dashboard and one that serialises every user; a regression is invisible to unit tests and only shows up as user lag.
Not a goal
agent.session_start_timeout_secsis clamped to at least 90s, so a hang test costs 90s per attempt and belongs at the unit level.What changed (motivation -> approach -> change)
Fourth slice of the in-process integration layer (harness #13621; memory seams #13736; spawn admission #14160). One test,
test/integration/test_streaming.py, drives two realPOST /api/chatSSE streams on two slots with the fake model's[[SLOW]]turn and pins the fairness contract behind bug 16.Based on
main(#13736 supplied theintegration_homeagent-spec pin release this slice needs; #14269 supplied the sandbox consent and the 95 ratchet). Independent of #14160.no linked issue: bug 16 comes from a Slack report; its fix landed as #13203 and this is the regression pin for it.
What the test pins
[[SLOW]]; wait for its first chunk[[SLOW]]while A is streaming[DONE])Local timings for the record (not asserted), warm slots: A first chunk +0.02s, B sent +0.05s, B first chunk +0.08s, A done +15.1s, B done +15.2s. The report's symptom would put B's first chunk after A's
[DONE].Why the shape is what it is
[[SLOW]]turn is an existing fake-model behaviour (SLOW_CHUNKS,SLOW_CHUNK_DELAY_SECSare imported, not restated).session/new timed out after 90s (4/4 MCP server(s) reported)+ Resume loop) is deliberately not here:agent.session_start_timeout_secsis clamped to >= 90s in the loader, so a hang test costs 90s+ per attempt, and the wording fix (session-injected MCP server(s) reported) is already on main. Its pin belongs at the unit level.Harness changes
None on this branch: the unsandboxed-exec consent this slice's chat turn needs (the CI container has no sandbox backend, so without it the spawn is refused before the fake stub runs) landed in
integration_homewith #14269 and is inherited here.Coverage
No workflow change: the route sweep (#14269) has merged and holds the
Integration (in-process gateway)ratchet at 95, which this slice keeps (pytest test/integrationalone on this branch: 1240/1252 routes, 99%).POST /api/chatgains a behaviour pin here; the sweep already counted it.docs/system-specs/common/testing-conventions.mdgains the SSE reading pattern.Tests
pytest test/integration -n 2 --dist loadgroup-> 22 passed, 61s.Backwards compatibility
Compatible: tests and a CI ratchet only; no product code changes.