Repository navigation
fix(ci): decline container_tests collection for directly-named files too - #10709
Conversation
|
Intent: Make the container suite's existing may-this-environment-collect decline hold for files named explicitly on the pytest command line — the path CI's reduced cross-surface run uses — so frontend-only PRs stop failing the backend shard on the suite's deliberately-absent image deps (#10700). |
collect_ignore_glob only filters files pytest discovers by walking the directory. CI's reduced cross-surface path passes this suite's files as explicit pytest arguments, which pytest treats as overriding every ignore mechanism - so on every frontend-only PR the backend shard imported a module whose deps (httpx/fastapi/uvicorn) are deliberately absent from the app env, and failed with ModuleNotFoundError (issue kirodotdev#10700). Module construction is the one step every path to a test module shares, so the conftest now also substitutes a declining collector in pytest_pycollect_makemodule: it raises the module-level skip instead of importing, scoped to this directory, and returns None wherever the suite may run. The REQUIRED lane (CREW_CONTAINER_TESTS_REQUIRED=1) is unchanged and still hard-errors when the environment cannot run the suite.
6c44a96 to
9a60f3a
Compare
Design Review (Fable 5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Fixes the decline-bypass at the one seam every collection path shares, preserving the required-lane's refusal-to-skip invariant; scoped and pinned. Suggestions
[DESIGN-REVIEWED] 9a60f3a |
First Principles Review (Fable 5, fork) — ✅ PASSPremise-level review of All claims check out. The premise that explicit pytest args bypass ignore mechanisms is already documented as "verified, not assumed" on the base at First-Principles-Verdict: PASS Verify the hook fires in one real pytest run on a directly-named file — the new pins call it in isolation; only the author's manual runs cover the live path. What this change shipsInventory (3 items) — 3 justifiedIntent: unblock frontend-only PRs whose Backend Tests shard fails because CI names container-suite files explicitly in an env deliberately missing their deps. FIX.
[FIRST-PRINCIPLES-REVIEWED] 9a60f3a |
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 4.8 Review (fork) — ✅ no blocking findingsReviewed Review detailsI've validated the single candidate against the file. Let me assess it. The candidate hinges on two premises it explicitly cannot confirm (its own confidence is "low"):
Falsifying (a) — "a concrete input that occurs in practice":
Input (a) is speculative ("if a caller were to"), not something occurring in practice on the path where the hook runs. The candidate drops under Step 1. Step 2: the diff is test-infrastructure only (a pytest No findings. [OPUS-REVIEWED] 9a60f3a |
…too (kirodotdev#10709) collect_ignore_glob only filters files pytest discovers by walking the directory. CI's reduced cross-surface path passes this suite's files as explicit pytest arguments, which pytest treats as overriding every ignore mechanism - so on every frontend-only PR the backend shard imported a module whose deps (httpx/fastapi/uvicorn) are deliberately absent from the app env, and failed with ModuleNotFoundError (issue kirodotdev#10700). Module construction is the one step every path to a test module shares, so the conftest now also substitutes a declining collector in pytest_pycollect_makemodule: it raises the module-level skip instead of importing, scoped to this directory, and returns None wherever the suite may run. The REQUIRED lane (CREW_CONTAINER_TESTS_REQUIRED=1) is unchanged and still hard-errors when the environment cannot run the suite.
Problem / Motivation
Backend Tests (3.12, 1)fails on every frontend-only PR withModuleNotFoundError: No module named 'httpx'fromcontainer_tests/test_review_findings.py, which fails theCoverage Gateclosed and redsPR Readiness. Red right now on at least #10678, #10679, #10680, #10681 and #10698.Why it matters
No frontend-only PR can reach
readiness: passeduntil this lands. The failing suite's deps (httpx/fastapi/uvicorn) are deliberately absent from the app CI env — the environment is right, the guard just has a hole.What changed (motivation → approach → change)
The container suite's
conftest.py(src/kiro_crew/apps/builtins/aws_control/crew/runtime/container_tests/conftest.py) already declines collection when the image-only deps are missing, viacollect_ignore_glob. That glob only filters files pytest discovers by walking the directory. CI's reduced cross-surface path passes this suite's files as explicit pytest arguments, and pytest treats direct arguments as overriding every ignore mechanism — apytest_ignore_collecthook included — so the modules were imported anyway and hit the missing dep.Module construction is the one step every path to a test module shares — directory walk, explicit file argument,
file.py::testnode id. So the conftest now also substitutes a declining collector inpytest_pycollect_makemodule: it raises the module-level skip instead of importing, is scoped to this one directory, and returnsNone(pytest proceeds normally) wherever the suite may run. TheCREW_CONTAINER_TESTS_REQUIRED=1lane is untouched and still hard-errors when the environment cannot run the suite.flowchart LR A[pytest args name a container test file]:::ctx --> B{collect_ignore_glob}:::ctx B -- "walk-discovered: filtered" --> C[not collected]:::ctx B -- "named directly: bypassed" --> D[import module -> ModuleNotFoundError]:::removed B -- "named directly" --> E[makemodule -> declining collector -> skip]:::added classDef added fill:#DCFCE7,stroke:#16A34A classDef removed fill:#FEE2E2,stroke:#DC2626,stroke-dasharray:4 3 classDef ctx fill:#E0F2FE,stroke:#0284C7🟩 added · 🟥 removed · 🟦 unchanged
Tests
Three pins added to
test/test_ci_surface_tests.py, using the file's existing conftest-loading harness: with a dep missing, the hook substitutes the declining collector for an in-suite file; with all deps present, it returnsNone; and it leaves files outside the suite's directory alone. All 49 pre-existing pins in the file still pass.Manual verification
At the tip of main, reproduced the failure and verified the fix for every invocation shape against a dep-less env: explicit file argument and
::node id now report the module skipped instead of failing; CI's exact reduced shape (-q -n auto --timeout=120 --no-cov <container files> <other files>) exits 0 with the other files' tests passing; the plain directory walk is byte-identical to before (glob path untouched);CREW_CONTAINER_TESTS_REQUIRED=1still raises the refusal.Pattern harvest
Rule candidate: review-prompt
Pattern: collect_ignore/collect_ignore_glob only covers walk-discovered files; any suite gated that way is still importable via explicit pytest arguments and needs a collector-level decline.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)Fixes #10700