Skip to content

test(electron): stub the py-spy dump in the gateway supervisor harness - #35635

Merged
rnoack1 merged 1 commit into
mainfrom
fix/flaky-wedge-unlisted-owner
Nov 12, 2026
Merged

rnoack1 merged 1 commit into
mainfrom
fix/flaky-wedge-unlisted-owner

Conversation

@rnoack1

@rnoack1 rnoack1 commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: gateway-supervisor.wedge-unlisted-owner.test.js gives the same verdict on every host, so its red means a real recovery bug.

On a host with py-spy installed, the file failed 10 of 10 solo runs on main, on Node 22 and Node 24. Each run a different subset of the respawn and port-wait tests went red, for example the gateway is respawned once: 0 !== 1. The full electron npm test showed 1-3 reds per run, always from this file.

Why it matters

Anyone running the electron suite on a machine with py-spy (a normal setup for a Kiro Crew developer) gets red runs that have nothing to do with their change. That trains people to rerun and ignore this file, which hides a real regression in wedge recovery when one lands.

Not a goal

  • Changing wedge recovery or the py-spy capture in the app.
  • Changing how settle() drains the scripted clock.
  • Auditing tests that do not use the shared supervisor harness.

What changed (motivation → approach → change)

Step Fact
Symptom settle() returns before recovery reaches the port check, so no timer fires and no respawn is counted
Cause Recovery first awaits capturePySpyDump. The harness never scripted it, so the test ran the real py-spy dump --pid 1234, a real child process
Fix harness() in gateway-supervisor-test-helpers.js now defaults capturePySpyDumpFn to a stub that resolves at once

The real bind check was the first suspect. It is not the cause: at every early return of settle(), no real socket was mid-listen or mid-close, and recovery's last log line was the one before the dump. Without py-spy on the host (CI), the spawn fails with ENOENT within the same few loop turns, which is why CI stayed green.

With no real process left before the port check, settle()'s fixed drain is enough, so it is unchanged. The harness already refused a real execFile; the dump reached the host only because it calls its own execFile. The file's header now names the dump as scripted. As a side effect the file runs in about 0.6 s instead of about 120 s.

Backwards compatibility

Compatible: test-only change; a test that passes its own capturePySpyDumpFn still gets it.

Removes nothing: the harness only gains a default for an option nobody else reads.

Tests

Proof: not provable -- test-only change

Determinism proof, on this host (load average 13-24, 48 cores):

Check Parent This PR
Solo runs, Node 24.18.0 10 of 10 red 20 of 20 green
Solo runs, Node 22.23.1 red (issue) 20 of 20 green
Shuffled order, 5 seeds 5 of 5 green
Forced: HOME with a py-spy that exits after 200 ms 5 of 5 red, same 6 tests each run 5 of 5 green
Full electron npm test 1-3 red (issue) 3 of 3 green (latest: 3664 pass, 4 skipped)

Each of the five issue tests still fails when the behaviour it asserts is removed from gateway-supervisor.js:

  • Delete the respawn after the port check: the three "respawns" tests go red.
  • Remove the stale-install abort from the port wait: the "dispatched and fails" and "ends that wait" tests go red.
  • Remove every install guard on the respawn (that abort, the two in respawn-owner.js, and the one in startGateway): "stops the respawn" goes red too. Any single guard alone still stops the spawn.

Manual verification

N/A — test-only change; the runs above are the verification.

Related Issues

Closes #35503

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@rnoack1
rnoack1 requested review from a team and KiroCrewCaptain as code owners November 11, 2026 22:39
@rnoack1
rnoack1 requested a review from bolichen97 November 11, 2026 22:39
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 11, 2026
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 467e91278fa0ffe2395f5bb5895d115ce56c69b1 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 467e91278fa0ffe2395f5bb5895d115ce56c69b1 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns. The flake comes from the harness running a real py-spy child process, and a default stub in the shared harness fixes it at the source.

[DESIGN-REVIEWED] 467e912

Description read

[DESCRIPTION-READ] 3f6e17c1bcceaff3297ec6f2fc50d51ed39d90f5b6dc82e4eab12fb51daf19b2

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 467e91278fa0ffe2395f5bb5895d115ce56c69b1 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check. The diff only changes Electron test harness files, so there is no user-visible surface and no screenshots are needed.

[UX-REVIEWED] 467e912

Description read

[DESCRIPTION-READ] e6092d80f7b7c18705accacbbbe64ed8b300e6cd65b2231b2cb76d8002e6aa12

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 467e91278fa0ffe2395f5bb5895d115ce56c69b1 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 467e912

Verdict parsed from the review's SHA-scoped output markers for commit 467e91278fa0ffe2395f5bb5895d115ce56c69b1.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 467e91278fa0ffe2395f5bb5895d115ce56c69b1 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 467e91278fa0ffe2395f5bb5895d115ce56c69b1 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 467e912

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 467e91278fa0ffe2395f5bb5895d115ce56c69b1 false-positive: <reason, 20 to 500 characters>

@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Approve

Why: the decider estimates a 5% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes.

Test-only change: the shared gateway-supervisor harness now defaults capturePySpyDumpFn to a stub that resolves at once, so wedge recovery no longer spawns a real py-spy dump whose exit timing varies by host and breaks settle()'s fixed drain. The stub matches the real function's contract ({ok, reason}, never rejects). Explicit overrides still apply (the test at line 275 relies on this). No other harness test checks py-spy logs or the real dump's timing; respawn-owner tests that reach recovery get the same fix. Production code is untouched.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 9d657f8b1b · 6 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@rnoack1

rnoack1 commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Backend Tests (Windows) shards 2 and 4 are red on main, not on this diff (this PR changes only two electron test files). Both reproduce on main's tree at e8283ce:

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 11, 2026
@rnoack1
rnoack1 force-pushed the fix/flaky-wedge-unlisted-owner branch from 9d657f8 to 467e912 Compare November 12, 2026 02:43
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 12, 2026
@rnoack1
rnoack1 enabled auto-merge (squash) November 12, 2026 04:01

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 467e912. I read the whole diff.

Test-only. The gateway supervisor test harness now defaults capturePySpyDumpFn to a stub that answers { ok: false, reason: "py-spy does not run in this harness" }. A test can still pass its own through overrides.capturePySpyDumpFn. So no test spawns the real py-spy, whose exit lands after a host-dependent number of loop turns. The wedge-unlisted-owner test's header now lists the stack dump among what it scripts.

No product code changes. 98 checks green.

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change; the electron gateway supervisor test harness now stubs the py-spy dump by default, and one test file header comment is updated.

@rnoack1
rnoack1 merged commit 4dd046d into main Nov 12, 2026
100 of 102 checks passed
@rnoack1
rnoack1 deleted the fix/flaky-wedge-unlisted-owner branch November 12, 2026 04:02
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: gateway-supervisor.wedge-unlisted-owner.test.js respawn tests go red on a loaded host

3 participants