Repository navigation
test: ignore other threads' lockdowns in the lockdown-order tests - #33394
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Test-only change. It fixes the cause: the probes now record only calls made on the test's own thread, so the SEL writer thread's lockdowns no longer count. The same-thread premise holds for every writer under test. Notes
[DESIGN-REVIEWED] 0a482a4 Description read[DESCRIPTION-READ] a07bc0a1694f08cc8cfe0d0177094dd69b7634e477fed7b806bc5bca14482871 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
The lockdown-order tests replace the process-global platform_compat.restrict_to_owner with a probe that records the size of every target it is called on, then assert the first recorded call (or the exact list) found an empty file. The SEL writer is a daemon thread that re-locks its non-empty audit log on every append, so a flush of events queued by an earlier test can land inside the write under test and be read as its lockdown: macOS CI saw test_write_lockdown_precedes_content fail with 475797 bytes. Each probe now records only calls made on the test's own thread. The writers under test all lock down on that thread, so every assertion keeps its strength; a probe that saw nothing still fails its premise. Closes #33300
b952566 to
0a482a4
Compare
|
Rebased: main moved in test/test_webhooks_store.py, test/test_apps_registry.py, test/test_apps_registry_coverage_install.py, test/test_apps_routes_coverage_update_uninstall.py, test/test_apps_worker_auto_approve.py since this head went green (2caa69b -> d2c6817); CI re-running. The previous head's only red, [operator: bolichen97] |
buluoray
left a comment
There was a problem hiding this comment.
Approving. Test-only fix for #33300. The eight restrict_to_owner size probes now keep only the test thread's calls (test/test_aws_consent.py:76-95, thread filter at :90; test/test_workflows_store_lockdown.py:30-36), so a sel-writer re-lock of its non-empty audit log (src/kiro_crew/sel_append.py:255) can no longer read as the store's lockdown. Every assertion keeps its strength.
Problem / Motivation
Goal: Stop the lockdown-order tests from failing when another thread locks down its own file while the test's write runs.
test/test_aws_consent.py::TestGrantIsOnTheKeystoneFloor::test_write_lockdown_precedes_contentfailed on macOS CI on #33195 (head 7fb162d), a PR that touches no AWS consent code:AssertionError: the file already held payload bytes when it was locked down: 475797 bytes. The same head passed on Linux.Why it matters
Any backend PR can hit this red, pay a rerun and sit blocked on PR Readiness for a defect that is not in its diff. Seven more tests use the same probe and can fail the same way.
Not a goal
restrict_to_ownerspies that record paths, events or flags. They share the root cause, need a per-site check of which thread locks down, and are tracked in Flaky risk: restrict_to_owner spies in tests also record the SEL writer thread's calls #33379.What changed (motivation → approach → change)
These tests check that a secret-bearing file is locked down while it is still empty. They swap the process-global
platform_compat.restrict_to_ownerfor a probe that records the size of every file it is called on, then assert that the first recorded call (or the exact list) saw zero bytes.The probe records calls from every thread. The SEL writer is a daemon thread (
sel-writer) that re-locks its non-empty audit log on every append (src/kiro_crew/sel_append.py:255). When it flushes events an earlier test queued while the write under test runs, its call is recorded first and the test reads the audit log's size as the store's. The store's own temp file is locked while empty, as the test intends.Each probe now records only calls made on the test's own thread. Every writer under test locks down on that thread, so each assertion keeps its strength, and a probe that sees nothing still fails its
premisecheck.test/test_aws_consent.py(two tests, one shared helper)test/test_kiro_prerequisite.py,test/test_tips.py,test/test_webhooks_store.pytest/test_session_transfer_layer_b.pysizes == [0, 0]test/test_workflows_store_lockdown.py(fixturelockdowns)ops_mission_control/tests/test_policy_store.py,tests/test_security.pyAlternative considered: keep only calls on the store's own staged temp path. It lost because it ties each test to the writer's temp naming, and a leaked thread that writes under the current data home would still count; the thread is what leaks.
Backwards compatibility
Compatible: test-only change; no product code touched.
Removes nothing: the probes still record every lockdown the code under test makes; only other threads' calls are left out.
Tests
No test is added. The twelve changed tests (the probes in the table;
lockdownsserves four workflow-store tests) keep every assertion they had.Determinism proof, on Linux, base
2caa69b7a617(parent) and this head:-n0tempfile.mkstemp, and the workflow store's own lockdown seam)sel-writerthread appends to its non-empty log insiderecord_grant(the twotest_aws_consent.pytests)[1582, 2109] == [0, 0])-n0-n 4 --dist loadgroup --max-worker-restart=2, withtest/test_live_target.pyandtest/test_sel_edge_cases.pyas co-tenantsThe forced conditions and the negative control are throwaway pytest plugins, not committed.
Manual verification
N/A — test-only change; the forced-condition runs under Tests are the verification.
Screenshots / video
N/A — test-only change, no UI.
Related Issues
Closes #33300
Sibling sweep
Swept: git grep -lE "restrict_to_owner" -- test/ src/kiro_crew/apps, then every spy those files install on it (an AST scan of the replacement functions and the asserts that read what they record)
Sibling: test/test_kiro_prerequisite.py:130 -- changed here
Sibling: test/test_tips.py:880 -- changed here
Sibling: test/test_webhooks_store.py:488 -- changed here
Sibling: test/test_session_transfer_layer_b.py:1360 -- changed here
Sibling: test/test_workflows_store_lockdown.py:34 -- changed here
Sibling: src/kiro_crew/apps/builtins/ops_mission_control/tests/test_policy_store.py:686 -- changed here
Sibling: src/kiro_crew/apps/builtins/ops_mission_control/tests/test_security.py:333 -- changed here
Sibling: test/test_atomic_write_capabilities.py:135 -- tracked in #33379
Sibling: test/test_weixin_qr.py:94 -- tracked in #33379
Sibling: test/test_handlers_messaging_coverage_channels.py:825 -- tracked in #33379
Sibling: test/test_cli_setup.py:633 -- tracked in #33379
Sibling: test/test_computer_use_capture.py:247 -- tracked in #33379
Sibling: test/test_computer_use_capture_windows.py:476 -- tracked in #33379
Sibling: test/test_config_rmw_preserves_settings.py:544 -- tracked in #33379
Sibling: test/test_memory_backup.py:350 -- tracked in #33379
Sibling: test/test_platform_compat_coverage_filesystem.py:495 -- tracked in #33379
Sibling: test/test_platform_compat_owner_only.py:560 -- tracked in #33379
Sibling: test/test_vector_memory.py:1079 -- tracked in #33379
Sibling: test/test_lockdown_before_publish.py:879 -- tracked in #33379
Sibling: src/kiro_crew/apps/builtins/code_review_sage/tests/test_store_selfheal.py:313 -- tracked in #33379
Sibling: test/test_live_target.py:454 -- not affected -- keeps only calls under live-target-staging
Sibling: test/test_mcp_gateway_apps_spool.py:226 -- not affected -- keeps only calls under the spool temp dir
Sibling: test/test_sel_edge_cases.py:116 -- not affected -- keeps only files in the trust dir
Sibling: test/test_mcp_gateway_rewriter.py:860 -- not affected -- asserts only on the agents dir's own calls
Sibling: test/test_memory_backup.py:1078 -- not affected -- keeps only .tmp/.partial files in the backup dir
Sibling: test/test_member_memory_backup.py:132 -- not affected -- keeps only .tmp/.partial files in the backup dir
Sibling: test/test_beacon.py:1062 -- not affected -- asserts membership of the config path only
Sibling: test/test_mcp_gateway_oversize.py:151 -- not affected -- asserts one specific path is absent
Sibling: test/test_mcp_gateway_prewarm.py:101 -- not affected -- asserts the spy saw at least one call
Sibling: test/test_memory_index_owner_only.py:43 -- not affected -- asserts membership of named paths only
Sibling: test/test_session_image_repair.py:813 -- not affected -- compares positions of two named events only
Sibling: test/test_snapshot.py:1262 -- not affected -- keeps only calls under the snapshot output dir
Sibling: test/test_token_auth_session_lifecycle.py:509 -- not affected -- asserts the spy saw at least one call
Sibling: test/test_webhooks_store.py:641 -- not affected -- asserts membership of two named calls only
Sibling: test/test_mcp_gateway_rewriter_fingerprint_keep.py:826 -- not affected -- probes fchmod_safe, which the SEL writer never calls
Pattern harvest
Rule candidate: review-prompt
Pattern: "a test patches a process-global and asserts on the order or count of its calls without keeping only the test thread's calls"
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.