Skip to content

test(decisions): import the HTTP stack before the end-to-end Jev ask - #29458

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
pearcekieser/fix-decisions-e2e-provider-budget
Nov 5, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
pearcekieser/fix-decisions-e2e-provider-budget

Conversation

@Pearcekieser

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Make test_jev_response_reaches_the_assembled_message pass on slow CI runners again.

Since #26950 merged, the three cases of this test that expect one judge request fail on Linux shards with assert 0 == 1 at test/test_decisions_end_to_end.py:130. The loopback judge gets no request. The failure occurred on shard 5 of PR #14767 two times in a row, and on PR run 41762567848 from another branch. Neither branch changes Python code.

Why it matters

Any PR whose shard 5 gets this file can go red without a cause in its own diff. Each red costs a rerun and a CI slot, and a second failure blocks the PR until main changes.

Not a goal

  • No change to the decision gate, the provider, or the 2 s default budget in the product.
  • No change to the test's 2 s budget or to how the gateway warms the provider at startup.

What changed (motivation → approach → change)

The test asks the gate for a decision with timeout_ms=2000, and the gate wraps the whole provider call in that budget. Since #26950 the provider imports httpx with HTTP/2 inside that call, and httpx imports httpcore and the h2 modules when it builds the client. In a CI worker, this is the first import of these packages. The modules have no bytecode yet, and coverage traces them, so the import can use the whole 2 s budget. The gate then cancels the call before the POST, and the test sees no request.

A new autouse fixture imports h2, httpcore and httpx before the test starts. The cold import then runs outside the timed call, and the 2 s budget only covers the client build and the loopback request. I did not raise the budget, because a larger budget only moves the race to a slower runner.

Backwards compatibility

Compatible: the diff adds one test fixture and changes no product code.

Removes nothing: the test keeps its cases, assertions and budget.

Tests

test/test_decisions_end_to_end.py keeps its five cases, their assertions and the 2 s budget. The new fixture only moves the first import of h2, httpcore and httpx out of the timed call.

Proof: not provable -- test-only change

Manual verification

I reproduced the CI failure locally on origin/main. I installed httpx[http2]==0.28.1 with pip install --no-compile and set PYTHONDONTWRITEBYTECODE=1. Then I ran the file with taskset -c 0,1, -n4, --cov=kiro_crew and --cov-branch. Without the fixture, 2 of 3 runs failed with the CI assertion. With the fixture, 5 of 5 runs passed on this branch.

Related Issues

The failure is the same assertion as closed #17324, which reported it on Windows before #26950.

Sibling sweep

Swept: git grep -l -E "TestServer|make_url" -- 'test/test_decisions*.py' | xargs grep -n "timeout_ms="
Sibling: test/test_decisions_end_to_end.py -- changed here
Sibling: test/test_decisions_impl_jev.py -- not affected -- it calls the provider directly, and the httpx import and client setup run before asyncio.timeout starts

Pattern harvest

Not generalizable: one test sent a real request through the gate while the provider's first import ran inside the budget, and the sweep found no second site.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Fixes #29457

@Pearcekieser
Pearcekieser requested review from a team and KiroCrewCaptain as code owners November 4, 2026 04:27
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision c8db4443dda226935ad3970f0fc2658031217dde ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision c8db4443dda226935ad3970f0fc2658031217dde touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of c8db4443dda226935ad3970f0fc2658031217dde — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: a test-only change that moves the cold httpx/httpcore/h2 imports out of the 2 s timed call, without touching the budget or any product code.

Notes

  • The test is only safe while the provider's lazy imports (impl_systemone.py:189) match this fixture's list. If the provider adds another lazy import inside the timed call, the flake comes back. A comment that names that provider site would tell the next editor to keep the two in step.

[DESIGN-REVIEWED] c8db444

Description read

[DESCRIPTION-READ] 2038f2cac423abd87790ede55fa31276349cbabc6b35f03ce4123f14497cdc45

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed c8db4443dda226935ad3970f0fc2658031217dde — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] c8db444

Verdict parsed from the review's SHA-scoped output markers for commit c8db4443dda226935ad3970f0fc2658031217dde.

False positive or not applicable? A repository writer can comment:
/ai-review override fable c8db4443dda226935ad3970f0fc2658031217dde false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of c8db4443dda226935ad3970f0fc2658031217dde and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] c8db444

False positive or not applicable? A repository writer can comment:
/ai-review override gpt c8db4443dda226935ad3970f0fc2658031217dde false-positive: <reason, 20 to 500 characters>

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): Electron Shell Tests fails on main with the same two linux-desktop-integration.real-fs.test.js cases, in main run 41759965091. #29439 tracks it, and #29444 and #29445 fix it. This PR does not touch Electron.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 4, 2026
Since #26950 the provider imports httpx, and through it httpcore and h2, inside the gate's 2 s budget. The first import in a fresh CI worker can use the whole budget, so the gate cancels the call before the POST and the asked cases see no request. An autouse fixture now imports h2, httpcore and httpx before the timed ask.
@Pearcekieser
Pearcekieser force-pushed the pearcekieser/fix-decisions-e2e-provider-budget branch from 3a26487 to c8db444 Compare November 4, 2026 06:57
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 4, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 4, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): Backend Tests (3.12, 2) fails in leaf mode on 120 s timeouts in tree-scan corpus gates such as test/test_snapshot_refactor_seams.py:864. This is main's CI issue #23450, and #23461 fixes it. Shards 1 and 3 to 8 passed, including the test this PR changes.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Nov 5, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 5, 2026 01:05

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: end-to-end decisions test imports the HTTP stack before the timed ask.

@KiroCrewCaptain
KiroCrewCaptain merged commit afe1e9d into main Nov 5, 2026
249 of 257 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the pearcekieser/fix-decisions-e2e-provider-budget branch November 5, 2026 01:06
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants