Repository navigation
test: reset the vault read audit's process caches in the test floor - #37754
Conversation
secrets.read_audit keeps _loaded (the audit switch read from config.json, reused for 30 s with no live snapshot) and _seen (per-secret windows) for the process. #37659 reset them only in test_vault_read_audit.py, so any other test on the worker inherited a warm cache, which made test_status_loads_config_off_the_event_loop count config loads by order. Reset both in test/conftest_resets.py and add a two-case guard that fails on whichever case runs second when the reset is missing. Part of #37737
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. This is a test-only reset of process-wide audit caches, and it sits in the shared conftest floor, which is where that kind of reset belongs. [DESIGN-REVIEWED] 3f6fe0f Description read[DESCRIPTION-READ] 36304fb96ae5ac45d4efb639b4220716f9f4dee372a3fcc2e78a5b63230324ee That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
…7659 testing-conventions §4 already says a new process-global ships its floor entry. #37659's secrets.read_audit._loaded did not, and made test_status_loads_config_off_the_event_loop red by order (fixed in #37726, floor entry in #37754). Record why the rule did not stop it and add a pre-push grep that flags new module-level state, plus a writing-tests pointer and checklist item. Part of #37737
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved. On this head the review checked several things. The new _cold_vault_read_audit floor fixture resets read_audit._loaded and read_audit._seen through _floor_monkeypatch, and only when the module is already imported. Those names and value shapes match src/kiro_crew/secrets/read_audit.py. Teardown restores the original values, and the fixture's undo order with the _rig fixture in test_vault_read_audit.py is correct. No production code changes. The guard file was run serially and under -n 2 --dist loadgroup, and both passed. With both reset lines removed, the second case fails. Removing only the _loaded line, or only the _seen line, also fails it, so each line has its own pin.
Done well: the reset sits in the shared test floor where the process-global lives, not in one more victim file, so every test that reaches a vault read starts cold.
Optional:
- The local resets in
test/test_vault_read_audit.py:51-52andtest/test_wakatime_handlers.py:507now duplicate the floor. They do no harm and can stay, or a follow-up can drop them.
…7659 (#37750) testing-conventions §4 already says a new process-global ships its floor entry. #37659's secrets.read_audit._loaded did not, and made test_status_loads_config_off_the_event_loop red by order (fixed in #37726, floor entry in #37754). Record why the rule did not stop it and add a pre-push grep that flags new module-level state, plus a writing-tests pointer and checklist item. Part of #37737
Problem / Motivation
Goal: no test inherits the vault read audit's cached state from an earlier test on the same worker.
#37659 added two process globals to
kiro_crew.secrets.read_audit._loadedholds the audit switch read fromconfig.json, reused forCONFIG_TTL_SECS(30 s) when there is no live config snapshot._seenholds the per-secret rate-limit windows. Every vault read that returns a secret consults both. Onlytest/test_vault_read_audit.pyreset them, in a file-local autouse fixture. So a vault read in one test decided whether a later test on the worker loaded config once or twice. That madetest_status_loads_config_off_the_event_loopred by order (#37737; that test was pinned cold in #37726).Why it matters
#37726 fixed one victim. Any other test that counts config loads or audit rows on a vault-read path can still pass or fail by order.
Not a goal
read_auditor the vault.What changed (motivation -> approach -> change)
The state is process-wide, so its reset belongs in the shared floor, not one file -> reset both globals before every
test/test, through_floor_monkeypatch, only when the module is already imported (the_fresh_aws_bin_dir_trustpattern) ->test/conftest_resets.py: new autouse_cold_vault_read_auditsets_loaded = Noneand_seen = {}.test/conftest.py: binds it in theconftest_resetsimport block.test/test_vault_read_audit_floor.py: two cases in onexdist_group. Each asserts the cold state, then warms both caches the way a vault read does. Whichever case runs second fails when the reset is missing.Backwards compatibility
Compatible: test-only.
Tests
Linux, 8 cores. "base" = main at
5c8ce43638plus only the new guard file. "floor" = this branch.-n 4 --dist loadgroup--random-orderseeds 1-5SecretVault/read_audit/vault.get),-n 6 --random-orderseed 7[first]saw_loaded == (65488.9, True)left by an unrelated testtest_status_loads_config_off_the_event_loop(asserts one load) shuffled withtest_vault_read_audit.pyandtest_connections_status.py, seeds 1-6test_wakatime_handlers.py+test_vault_read_audit.pyThe base row for the 71 files is the leak this closes, seen without any planted warm-up.
Proof: forced condition red on the parent and green on the fix, shown above.
Manual verification
N/A: test-only.
Screenshots / video
N/A: no user-visible UI change.
Related Issues
Part of #37737