Repository navigation
test: give every test the cold source-link settings snapshot - #39475
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: this is a test-only floor reset, it follows the existing Notes
[DESIGN-REVIEWED] 432e233 Description read[DESCRIPTION-READ] 532c6a9f3771b48a3bfe173c3537a6c49c7f62b1e812fbef3d47d7295e4ad1ad That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The floor entry fixes the cause: every snapshot that What this change shipsInventory (4 items) — 4 justifiedIntent: stop tests from leaving the source-link chip switch or host allowlists changed for later tests on the same worker, which reddens unrelated shards. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 432e233 Description read[DESCRIPTION-READ] 0803547db2b637edc1adaba0d82843fcbbd2bc31e870ca1905a6ff664829c1f3 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
ac44e2d to
4bc1f9d
Compare
4bc1f9d to
58248f4
Compare
58248f4 to
432e233
Compare
Problem / Motivation
Goal: No test can leave the source-link settings snapshot (session-card chip switch and provider host allowlists) changed for the tests after it on the same xdist worker.
test_source_link_github_case.py::TestOneChipPerObject::test_case_variants_collapse_to_one_chipfails withsource_links_totalassert 0 == 1on Backend Tests shard 7/8, on main (runs 44519135612, 44514765616) and on four PRs that do not touch source links (#39453, #39450, #39301, #36477).Why it matters
Shard 7 goes red on unrelated PRs, so their authors rerun or chase a failure they did not cause.
Not a goal
test_process_cache_floor_ratchet.pyto scalar globals.What changed (motivation → approach → change)
dashboard.source_providers.hostskeeps the chip switch and five host allowlists as module globals. The real/api/dashboard/configPUT publishes the switch.test_session_card_source_links_knob.py::TestConfigEndpoint::test_put_persists_offsendsfalseand puts nothing back, so every laterto_dicton that worker reports no chips. #39470, now on main, pins the switch in that file's fixture. This PR adds the floor entry, so the next test in any file that publishes these settings cannot leak them either.test/conftest_resets.py: new autouse_cold_source_link_settingspins the switch toTrue, the allowlists tofrozenset()and the load stamp to0.0through_floor_monkeypatch, only when the module is already imported. The generation counter stays unpinned because the per-slot link cache keys on it.docs/system-specs/common/testing-conventions.md§ 4 andsrc/kiro_crew/builtin_skills/kirocrew-dev/writing-tests/SKILL.md: this leak shape, a second pre-push grep forTrue/False/frozenset()/ number initializers, and a wrong/right pair.Backwards compatibility
Compatible: test-only floor and docs; no product code changes.
Removes nothing: the diff adds a test fixture and documentation.
Tests
No test added: the change is the floor fixture itself.
Determinism proof on main
8301e68663(which already has #39470): a throwaway test that callshosts._publish_session_card_chips(False)and restores nothing, run beforetest/test_source_link_github_case.pyat-n0, x20: main's conftest 20/20 red (4 failed, 17 passed), this head 0/20 red (21 passed). The probe file was not committed.Proof: not provable -- test-only change
Manual verification
N/A — covered by the real-machine runs below.
Real-machine verification
Machine: DevSpace (KiroCrewBlueprint), Amazon Linux 2023, kernel 6.1, x86_64, 8 vCPU / 30 GB, Python 3.12.16
Before (main 8c917a5):
pytest -n0 -q test/test_session_card_source_links_knob.py::TestConfigEndpoint::test_put_persists_off test/test_source_link_github_case.pyx20 -> 20/20 red,4 failed, 17 passed(assert 0 == 1)After (head f8ea3ca7380d, this change before the rebase onto #39470): same command x20 -> 0/20 red,
21 passed; both files at-n 4x10 -> 0/10 redFull-suite regression: CI's Backend Tests shards on head ac44e2d were green except
test_process_cache_floor_ratchet.py::test_every_new_module_level_cache_has_a_per_test_reset(_NOT_DUE_MEMO), which is red on main itself and is fixed by #39479.Cleanup: DevSpace
ds-chip-floor-verify-jlsk056ldeleted (devspaces deletereported removed)Related Issues
Part of #39460
Relationship to #39470 (merged): #39470 pins the chip switch in
test_session_card_source_links_knob.py'shandler_appfixture, which stops the one polluter found. This PR adds the per-test floor reset intest/conftest_resets.pyfor every snapshot indashboard.source_providers.hosts, plus the convention text. The two touch different files; with both on main, the fixture pin is redundant but harmless.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits