Repository navigation
test: cover the spawn admission seams behind bugs 14, 15 and 20 - #14160
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Confirm the [DESIGN-REVIEWED] fe454ec |
First Principles Review (Fable 5) — ⏭️ skippedRevision |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
5faee32 to
f03d12f
Compare
|
|
f03d12f to
c0b473a
Compare
|
|
|
87323cb to
59c25bd
Compare
c0b473a to
aaba646
Compare
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
d4cf41a to
187b77c
Compare
187b77c to
bd8b888
Compare
|
|
Third slice of the in-process integration layer: six tests in test/integration/test_governance.py drive POST /api/spawn and the /api/security snapshots through a booted gateway with the credentials a real MCP caller presents, and pin what the product answers today. Bug 20 (a memory-held spawn reported as "queued behind concurrency limit"): the memory bar is raised through the operator's own config.local.json (agent.spawn_min_memory_gb), never by patching the guard. A persistent parent gets a durable row, so the guard defers it and the answer must be `queued` with reason `low_memory` and a detail that names the gigabytes, not the cap. An incognito parent has no row to park, so the guard refuses, and the refusal names the memory. Both assert the wording the fix on main now produces. Bug 14 (the parent agent spec's spawn allow-list): kiro-cli reads it from toolsSettings.subagent.availableAgents, with trustedAgents beside it only waiving permission prompts. The spawn gate honours the list: a parent slot runs a spec whose list names one agent, and a spawn of another installed agent is refused with agent_not_available and a message that names the agent, the parent and the list -- pinned as the positive contract through the real route, no child started. Bug 15 (a report that the agent read ~/.aws/credentials): the in-process boot cannot run the OS sandbox or the shell gate's hook process, so the layer pins the baseline a report is triaged against instead -- on a fresh home no denied-command opt-out is recorded, every built-in rule is on, the AWS credential directory is among the sensitive paths, and the three snapshots agree on their counts. Agent specs a slot or a spawn must resolve are written to <home>/kiro/agents before the boot: the loader answers names from a snapshot it scans once, and a spec written afterwards is not dispatchable until the next registration. The conventions doc gains that seam and the config.local.json one. Route coverage moves to 13/1245 and the ratchet tightens from 0.5% to 0.7%. The harness also grants the disposable home unsandboxed-exec consent, as the E2E suite does: the CI container has no sandbox backend, and without it every child the gateway spawns is refused before the fake model runs.
bd8b888 to
fe454ec
Compare
bolichen97
left a comment
There was a problem hiding this comment.
Tests-only slice; read all six cases. Operator seams (agents dir before boot, config.local.json) are documented in testing-conventions.md. LGTM.
Problem / Motivation
Goal: Three reported spawn-admission and security-baseline behaviours (bugs 14, 15, 20) are pinned as tests through the running gateway, so a regression in any of them fails CI by route and by message.
The reports behind bugs 14, 15 and 20 had no test that drove the real route: a spawn outside the parent spec's allow-list, a memory-held spawn described as a capacity queue, and a report that the agent read a credential file were each investigated by hand and left no artefact that would catch the same regression again.
Why it matters
Spawn admission is an authorization boundary and the security snapshots are what a report is triaged against; without a route-level pin a fix can silently regress, and the next report costs the same investigation again.
Not a goal
network.egressfloor or any other governance change.What changed (motivation -> approach -> change)
Third slice of the in-process integration layer (harness: #13621; memory seams: #13736). Six tests in
test/integration/test_governance.pydrivePOST /api/spawnand the/api/security/*snapshots through a booted gateway with the credentials a real MCP caller presents (gw.mcp_headers), and pin what the product answers today for three reported bugs.Based on
main(#13736 supplied the cookie auth andmcp_headersthis slice uses; #14269 supplied the sandbox consent inintegration_homeand the 95 ratchet). Independent of #14258.Fixes #14180 (bug 14: the allow-list gate landed on
mainin #13683; this test pins it through the real route, so the tracking issue closes with this PR). Bugs 15 and 20 come from Slack reports with no GitHub issue; no linked issue for them: 20 is already fixed on main and 15 is a baseline pin, not a defect.What each test pins
test_spawn_is_guarded/api/spawnwithout credentialstest_a_memory_deferral_is_named_as_oneconfig.local.jsonqueued/low_memory/ detail names GB, not the cap (bug 20)test_a_memory_refusal_names_the_memoryspawn refused: only … GB memory available,counted: true(bug 20)test_a_spawn_outside_the_allow_list_is_refusedtoolsSettings.subagent.availableAgents: ["ally"], spawnstrangeragent_not_available; message namesstranger,gatekeeperandally(bug 14)test_security_snapshots_are_guarded/api/security/*reads without credentialstest_a_fresh_home_ships_the_credential_ceiling_intact~/.awslisted, counts agree (bug 15 baseline)Why the shapes are what they are
toolsSettings.subagent.availableAgentsis the spawn allow-list (docs/reference/kiro-cli/chat/subagents.md);trustedAgentsbeside it only waives permission prompts, which is why the reporter's spelling is not the one asserted. The gate (_vet_parent_available_agents, landed in fix: honour the parent agent spec's availableAgents at the spawn gate #13683) refuses before any child starts withagent_not_available; the test asserts the code and the three names in the message (not the prose), so a rewording that keeps them still passes. The_settledguard stays so a regression that admits the spawn cannot leave a child running past the boot.Seams documented
Agent specs a slot or a spawn must resolve are written to
<home>/kiro/agentsbefore the boot (the loader answers names from a snapshot scanned once; a later write is not dispatchable until the next registration). Boot-time config goes in<home>/config.local.json. Both added totesting-conventions.md.Harness change
None. Every spawn this slice makes is refused at a gate that runs before any child starts (the memory guard for bug 20, the allow-list check for bug 14), so the layer's conftest is untouched here;
_pin_memory_barmerges intoconfig.local.jsonrather than replacing it, so whatever the harness writes there for another slice survives. No test in this slice depends on a sandbox backend being present or absent.Coverage
No workflow change: the route sweep (#14269) has merged and holds the
Integration (in-process gateway)ratchet at 95, which this slice keeps (pytest test/integrationalone on this branch: 1240/1252 routes, 99%)./api/spawnand the three/api/security/*snapshots gain behaviour pins here; the sweep already counted them.Tests
main:pytest test/integration -n 2 --dist loadgroup-> 27 passed, 34s. CIIntegration (in-process gateway)green on the previous head with the same 27._settledguard only runs if a regression admits it.Backwards compatibility
Compatible: tests and a CI ratchet only; no product code changes.