Skip to content

test: start one node child for the ssrf-guard module, not four at once - #37716

Merged
bolichen97 merged 1 commit into
mainfrom
crew/linuxflackyguard/ssrf-guard-36261
Nov 16, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
crew/linuxflackyguard/ssrf-guard-36261

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Stop test_ssrf_guard_fail_closed.py from stalling Linux Backend Tests (3.12, 7) and failing unrelated PRs.

On Linux shard 7, TestInstallSsrfGuardRuntime and TestGetPlaywrightVersionGate fail with Failed: Timeout >120.0s (sometimes one assert -9 == 0), together with whatever else is in flight. It has blocked at least five different PRs in seven days, none of which touch design_critique. Latest: job 132937923896 on #37633.

Why it matters

One shard going red fails Coverage Gate and PR Readiness, so every PR whose run hits it needs a rerun, and some runs lose the runner entirely.

Not a goal

  • Changing the shard count, worker count or runner size in ci.yml.
  • Changing ssrf-guard.mjs or ensure-playwright.mjs.
  • Moving these checks to a JavaScript test runner.

What changed (motivation → approach → change)

The four runtime tests each started their own node. xdist sent them to four workers, so four cold Node starts (a ~100 MB binary each) began in the same second, at 98% of shard 7, on a runner near its memory ceiling. The per-test progress artifacts (linux-pytest-progress-7) of six failed attempts all show the same thing: ~50 tests a second up to that second, almost nothing for minutes after it. Passing attempts start the same four in the same second and finish the last ~530 tests in under ten seconds. Each child's deadline was 120 s against --timeout=120, so pytest-timeout always fired first and the stall was reported as bare timeouts.

Before After
node children per run 4, on 4 workers, same second 1, on one worker (xdist_group)
Child deadline 120 s, equal to --timeout kiro_crew.testing.wait.default_timeout(), half the test timeout: 60 s on Linux, 90 s on Windows
A stalled child reads as 4 × Timeout >120.0s node did not finish the ssrf-guard probe within 60s

The one child runs all four cases and prints a verdict per case. test_capture_site_resolver_rule.py, the other design_critique module that started node per test, gets the same shape: one child over every input its five tests use. Both cache cases run copies of ensure-playwright.mjs under tmp, so neither can resolve a playwright above the checkout. The missing-routeWebSocket case also checks that no HTTP route was installed before the refusal.

testing-conventions.md (Determinism D8 and class 6), builtin_skills/kirocrew-dev/writing-tests/SKILL.md and the tests-are-deterministic rule in AUTOSDE.yaml now name this shape: one module-scoped interpreter child, deadline at half the test timeout.

Backwards compatibility

Compatible: only a test file and docs change.
Removes nothing: no product code, key or path changes; the four test names are unchanged.

Tests

Proof: not provable -- test-only change

  • Both changed files: 20/20 at -n0; 10/10 at -n 4 beside the other design_critique tests (173 tests); 5/5 in shuffled order.
  • Each of five mutations of the shipped scripts turns tests red: the guard skipping its capability check, the guard always refusing, the version gate always true or always false, hostResolverRule not bracketing IPv6.
  • Forced conditions on a Linux DevSpace, -n 4 --timeout=120, 10 rounds each, before and after interleaved:
Forced condition Before After
Concurrent starts contend (each start holds one shared lock 35 s) 10/10 red: the 4th child hits Timeout >120.0s (124-132 s) 10/10 green (48-54 s)
node never finishes starting 10/10 red: 4 × Timeout >120.0s, 122 s 10/10 red by name: within 60s, 62 s
Memory pressure (PSI full stall up to 20 s a round) slowest node call 0.2–23 s one child, 0.05–0.08 s unloaded

Manual verification

N/A — unit coverage and the DevSpace rounds above; CI on this PR runs shard 7.

Related Issues

Fixes #36261

Sibling sweep

Swept: git grep -l 'which("node")' -- 'test/.py' 'src/kiro_crew/**/tests/.py'
Sibling: src/kiro_crew/apps/builtins/design_critique/tests/test_capture_site_resolver_rule.py -- changed here
Sibling: test/test_symbols_manifest_contract.py -- tracked in #37714
Sibling: test/test_app_frame.py -- tracked in #37714

Pattern harvest

Rule candidate: review-prompt
Pattern: several tests in one module each spawn a real node/interpreter child, or a child deadline equals the suite's --timeout

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 15, 2026 23:02
@bolichen97
bolichen97 requested a review from CrysisDeu November 15, 2026 23:02
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 15, 2026
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision b687fc13649235ca9b4ef332e331c3a641bdcef4 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of b687fc13649235ca9b4ef332e331c3a641bdcef4 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: one child per module in its own xdist_group, with a deadline at half the test timeout, goes after the measured cause, and the change only touches tests and docs.

Notes

  • design-critique is listed in apps/adoption-pins.json. AGENTS.md says a fix to a pinned built-in lands in its app repository first, and the PR names it with App repo: <owner>/<repo>@<sha>. This description has no such line, so the core copy of these tests can drift from the app repo.
  • The old Windows pin (NODE_TIMEOUT_SECS >= 120) becomes 90 s. That still covers the "more than a minute" cold start the old comment cited, but the first Windows red with within 90s would mean the budget needs a measurement.

[DESIGN-REVIEWED] b687fc1

Description read

[DESCRIPTION-READ] a6c4870204b926675b9c7f58bc0c26cecc4757a70f9fd0cead5f52ab77e16541

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of b687fc13649235ca9b4ef332e331c3a641bdcef4 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Test-only fix for a shard stall that is linked to an issue (#36261) and backed by failing CI jobs; the riders are small extra assertions inside the same two test modules.

What this change ships

Inventory (7 items) — 5 justified

Intent: stop the design-critique node tests from stalling Linux shard 7 and failing unrelated PRs. This is a FIX.

  1. The ssrf-guard module now starts one node child for all four runtime cases, kept on one worker. — justified
  2. The child's deadline drops from 120 s to half the test timeout, and a stalled child now fails with a named message instead of a bare Timeout. — justified
  3. test_capture_site_resolver_rule.py gets the same shape: one child for all inputs, in its own xdist_group. — justified
  4. The usable-cache case now runs a copy of the script under tmp, so it can't pick up a playwright found above the checkout. — rides along, declared
  5. The missing-routeWebSocket case now also asserts that no HTTP route was installed before the refusal. — rides along, declared
  6. The NODE_TIMEOUT_SECS >= 120 pin is replaced by a half-timeout pin plus a fails-by-name test. — justified
  7. D8 in testing-conventions, the writing-tests skill and the tests-are-deterministic rule in AUTOSDE.yaml now state the one-child-per-module rule. — justified

[FIRST-PRINCIPLES-REVIEWED] b687fc1

Description read

[DESCRIPTION-READ] b4cbf165fe6c657626190dabe91b679edb184cdcc1ce93d78be30169d03c5b76

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed b687fc13649235ca9b4ef332e331c3a641bdcef4 — this comment is updated in place on each push.

Review details

No findings. None of the three candidates held up when checked against the code. All three are about how the test suite reports a failure or how much time a child gets, and none of them leads to a reachable defect.

[OPUS-REVIEWED] b687fc1

Verdict parsed from the review's SHA-scoped output markers for commit b687fc13649235ca9b4ef332e331c3a641bdcef4.

False positive or not applicable? A repository writer can comment:
/ai-review override fable b687fc13649235ca9b4ef332e331c3a641bdcef4 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of b687fc13649235ca9b4ef332e331c3a641bdcef4 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] b687fc1

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b687fc13649235ca9b4ef332e331c3a641bdcef4 false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 force-pushed the crew/linuxflackyguard/ssrf-guard-36261 branch from a0f3090 to 88a7c48 Compare November 15, 2026 23:25
@bolichen97

Copy link
Copy Markdown
Contributor Author

FIRST-PRINCIPLES (BLOCK): _node_deadline is a second copy of a half-the-test-timeout deadline that already exists: kiro_crew.testing.wait.default_timeout().

fixed in 88a7c48: _node_deadline and its config readers are gone; both node helpers (test_ssrf_guard_fail_closed.py::_run_node, test_capture_site_resolver_rule.py::_rules) bound their child with kiro_crew.testing.wait.default_timeout(). TestNodeBudget now pins that the child gets half the running test's timeout and that a child past its deadline fails by name. The testing-conventions paragraph names the helper.
self-added: yes
mechanism: removed a duplicate deadline reader in favour of the existing helper

@bolichen97

Copy link
Copy Markdown
Contributor Author

[operator: bolichen97] Backend Tests (3.12, 1) red on test/test_wakatime_handlers.py::test_status_loads_config_off_the_event_loop (assert [False, False] == [False]): https://github.com/kirodotdev/KiroCrew/actions/runs/43774730630/job/133000775350. This diff touches no wakatime file; the flake is being fixed in #37726. Coverage Gate failed only because that shard did. Rerunning the failed jobs once.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 16, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

[operator: bolichen97] The rerun of Backend Tests (3.12, 1) is red again on the same test/test_wakatime_handlers.py::test_status_loads_config_off_the_event_loop (https://github.com/kirodotdev/KiroCrew/actions/runs/43774730630/job/133014711915). It is order-dependent: it fails whenever its shard lacks the tests that warm its caches, which #37726 fixes. Not rerunning again; this PR rebases once #37726 lands. Every other lane is green, including Backend Tests (3.12, 7) (https://github.com/kirodotdev/KiroCrew/actions/runs/43774730630/job/133000775349).

@bolichen97
bolichen97 force-pushed the crew/linuxflackyguard/ssrf-guard-36261 branch from 88a7c48 to 71e0789 Compare November 16, 2026 01:26
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 16, 2026
The four runtime tests in test_ssrf_guard_fail_closed.py each started
their own node, and --dist loadgroup handed them to four workers, so
four cold Node starts began in the same second at 98% of Linux Backend
Tests shard 7. In six of six failed attempts the shard ran ~50 tests a
second up to that second and almost none after it; every test in flight
hit Timeout >120.0s. Each child's 120 s deadline equalled the suite's
--timeout, so pytest-timeout always fired first and the child's own
deadline never named the stall.

One module-scoped child now runs all four cases and prints a verdict per
case, the module has its own xdist group so it lands on one worker, and
the child's deadline is half the test's own timeout
(kiro_crew.testing.wait.default_timeout()). Both cache cases run copies
of ensure-playwright.mjs under tmp, so neither can resolve a playwright
above the checkout, and the missing-routeWebSocket case also checks no
HTTP route was installed before the refusal.
test_capture_site_resolver_rule.py, the other design_critique module
that started node per test, gets the same shape: one child over every
input its five tests use.

The testing conventions, the writing-tests skill and AUTOSDE's tests-
are-deterministic rule now name the shape.

Fixes #36261
@bolichen97
bolichen97 force-pushed the crew/linuxflackyguard/ssrf-guard-36261 branch from 71e0789 to b687fc1 Compare November 16, 2026 02:32
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 16, 2026
@bolichen97
bolichen97 merged commit c4c3d92 into main Nov 16, 2026
97 checks passed
@bolichen97
bolichen97 deleted the crew/linuxflackyguard/ssrf-guard-36261 branch November 16, 2026 03:30
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: test_ssrf_guard_fail_closed node subprocess tests time out on Linux shard 7

1 participant