Skip to content

test: ignore other threads' lockdowns in the lockdown-order tests - #33394

Merged
bolichen97 merged 1 commit into
mainfrom
fix/33300-consent-lockdown-filter
Nov 9, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/33300-consent-lockdown-filter

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Stop the lockdown-order tests from failing when another thread locks down its own file while the test's write runs.

test/test_aws_consent.py::TestGrantIsOnTheKeystoneFloor::test_write_lockdown_precedes_content failed on macOS CI on #33195 (head 7fb162d), a PR that touches no AWS consent code: AssertionError: the file already held payload bytes when it was locked down: 475797 bytes. The same head passed on Linux.

Why it matters

Any backend PR can hit this red, pay a rerun and sit blocked on PR Readiness for a defect that is not in its diff. Seven more tests use the same probe and can fail the same way.

Not a goal

  • No product code change: every writer under test already locks its temp file down while it is empty.
  • No change to the SEL writer thread or to the directory it writes.
  • No change to the lockdown probes that already keep only their own target's calls.
  • No change to the other restrict_to_owner spies that record paths, events or flags. They share the root cause, need a per-site check of which thread locks down, and are tracked in Flaky risk: restrict_to_owner spies in tests also record the SEL writer thread's calls #33379.

What changed (motivation → approach → change)

These tests check that a secret-bearing file is locked down while it is still empty. They swap the process-global platform_compat.restrict_to_owner for a probe that records the size of every file it is called on, then assert that the first recorded call (or the exact list) saw zero bytes.

The probe records calls from every thread. The SEL writer is a daemon thread (sel-writer) that re-locks its non-empty audit log on every append (src/kiro_crew/sel_append.py:255). When it flushes events an earlier test queued while the write under test runs, its call is recorded first and the test reads the audit log's size as the store's. The store's own temp file is locked while empty, as the test intends.

Each probe now records only calls made on the test's own thread. Every writer under test locks down on that thread, so each assertion keeps its strength, and a probe that sees nothing still fails its premise check.

Probe (each records the test thread's calls only) Assertion another thread's call broke
test/test_aws_consent.py (two tests, one shared helper) first size, first two sizes are 0
test/test_kiro_prerequisite.py, test/test_tips.py, test/test_webhooks_store.py first size is 0
test/test_session_transfer_layer_b.py sizes == [0, 0]
test/test_workflows_store_lockdown.py (fixture lockdowns) exact list of lockdowns
ops_mission_control/tests/test_policy_store.py, tests/test_security.py first size is 0

Alternative considered: keep only calls on the store's own staged temp path. It lost because it ties each test to the writer's temp naming, and a leaked thread that writes under the current data home would still count; the thread is what leaks.

Backwards compatibility

Compatible: test-only change; no product code touched.

Removes nothing: the probes still record every lockdown the code under test makes; only other threads' calls are left out.

Tests

No test is added. The twelve changed tests (the probes in the table; lockdowns serves four workflow-store tests) keep every assertion they had.

Determinism proof, on Linux, base 2caa69b7a617 (parent) and this head:

Run Parent This head
plain, -n0 12 passed 12 passed
forced: another thread locks down a 475,797-byte file at each writer's staging step (tempfile.mkstemp, and the workflow store's own lockdown seam) 12 failed 12 passed
forced: the real sel-writer thread appends to its non-empty log inside record_grant (the two test_aws_consent.py tests) 2 failed ([1582, 2109] == [0, 0]) 2 passed
negative control: the consent store is written first and locked down after n/a 2 failed, as they must
20 repeats at -n0 n/a 0 failures
10 repeats at -n 4 --dist loadgroup --max-worker-restart=2, with test/test_live_target.py and test/test_sel_edge_cases.py as co-tenants n/a 0 failures (110 passed each)
every test in the eight changed files, shuffled (seed 19453) n/a 504 passed

The forced conditions and the negative control are throwaway pytest plugins, not committed.

Manual verification

N/A — test-only change; the forced-condition runs under Tests are the verification.

Screenshots / video

N/A — test-only change, no UI.

Related Issues

Closes #33300

Sibling sweep

Swept: git grep -lE "restrict_to_owner" -- test/ src/kiro_crew/apps, then every spy those files install on it (an AST scan of the replacement functions and the asserts that read what they record)
Sibling: test/test_kiro_prerequisite.py:130 -- changed here
Sibling: test/test_tips.py:880 -- changed here
Sibling: test/test_webhooks_store.py:488 -- changed here
Sibling: test/test_session_transfer_layer_b.py:1360 -- changed here
Sibling: test/test_workflows_store_lockdown.py:34 -- changed here
Sibling: src/kiro_crew/apps/builtins/ops_mission_control/tests/test_policy_store.py:686 -- changed here
Sibling: src/kiro_crew/apps/builtins/ops_mission_control/tests/test_security.py:333 -- changed here
Sibling: test/test_atomic_write_capabilities.py:135 -- tracked in #33379
Sibling: test/test_weixin_qr.py:94 -- tracked in #33379
Sibling: test/test_handlers_messaging_coverage_channels.py:825 -- tracked in #33379
Sibling: test/test_cli_setup.py:633 -- tracked in #33379
Sibling: test/test_computer_use_capture.py:247 -- tracked in #33379
Sibling: test/test_computer_use_capture_windows.py:476 -- tracked in #33379
Sibling: test/test_config_rmw_preserves_settings.py:544 -- tracked in #33379
Sibling: test/test_memory_backup.py:350 -- tracked in #33379
Sibling: test/test_platform_compat_coverage_filesystem.py:495 -- tracked in #33379
Sibling: test/test_platform_compat_owner_only.py:560 -- tracked in #33379
Sibling: test/test_vector_memory.py:1079 -- tracked in #33379
Sibling: test/test_lockdown_before_publish.py:879 -- tracked in #33379
Sibling: src/kiro_crew/apps/builtins/code_review_sage/tests/test_store_selfheal.py:313 -- tracked in #33379
Sibling: test/test_live_target.py:454 -- not affected -- keeps only calls under live-target-staging
Sibling: test/test_mcp_gateway_apps_spool.py:226 -- not affected -- keeps only calls under the spool temp dir
Sibling: test/test_sel_edge_cases.py:116 -- not affected -- keeps only files in the trust dir
Sibling: test/test_mcp_gateway_rewriter.py:860 -- not affected -- asserts only on the agents dir's own calls
Sibling: test/test_memory_backup.py:1078 -- not affected -- keeps only .tmp/.partial files in the backup dir
Sibling: test/test_member_memory_backup.py:132 -- not affected -- keeps only .tmp/.partial files in the backup dir
Sibling: test/test_beacon.py:1062 -- not affected -- asserts membership of the config path only
Sibling: test/test_mcp_gateway_oversize.py:151 -- not affected -- asserts one specific path is absent
Sibling: test/test_mcp_gateway_prewarm.py:101 -- not affected -- asserts the spy saw at least one call
Sibling: test/test_memory_index_owner_only.py:43 -- not affected -- asserts membership of named paths only
Sibling: test/test_session_image_repair.py:813 -- not affected -- compares positions of two named events only
Sibling: test/test_snapshot.py:1262 -- not affected -- keeps only calls under the snapshot output dir
Sibling: test/test_token_auth_session_lifecycle.py:509 -- not affected -- asserts the spy saw at least one call
Sibling: test/test_webhooks_store.py:641 -- not affected -- asserts membership of two named calls only
Sibling: test/test_mcp_gateway_rewriter_fingerprint_keep.py:826 -- not affected -- probes fchmod_safe, which the SEL writer never calls

Pattern harvest

Rule candidate: review-prompt
Pattern: "a test patches a process-global and asserts on the order or count of its calls without keeping only the test thread's calls"

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 8, 2026 20:16
@bolichen97
bolichen97 requested a review from Zedmor November 8, 2026 20:16
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 8, 2026
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 0a482a40ae5dd10ee2952587ff76831bf83f19d1 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 0a482a40ae5dd10ee2952587ff76831bf83f19d1 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 0a482a40ae5dd10ee2952587ff76831bf83f19d1 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Test-only change. It fixes the cause: the probes now record only calls made on the test's own thread, so the SEL writer thread's lockdowns no longer count. The same-thread premise holds for every writer under test.

Notes

[DESIGN-REVIEWED] 0a482a4

Description read

[DESCRIPTION-READ] a07bc0a1694f08cc8cfe0d0177094dd69b7634e477fed7b806bc5bca14482871

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 0a482a40ae5dd10ee2952587ff76831bf83f19d1 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 0a482a4

Verdict parsed from the review's SHA-scoped output markers for commit 0a482a40ae5dd10ee2952587ff76831bf83f19d1.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 0a482a40ae5dd10ee2952587ff76831bf83f19d1 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 0a482a40ae5dd10ee2952587ff76831bf83f19d1 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 0a482a4

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 0a482a40ae5dd10ee2952587ff76831bf83f19d1 false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) November 8, 2026 20:44
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 8, 2026
The lockdown-order tests replace the process-global
platform_compat.restrict_to_owner with a probe that records the size
of every target it is called on, then assert the first recorded call
(or the exact list) found an empty file. The SEL writer is a daemon
thread that re-locks its non-empty audit log on every append, so a
flush of events queued by an earlier test can land inside the write
under test and be read as its lockdown: macOS CI saw
test_write_lockdown_precedes_content fail with 475797 bytes.

Each probe now records only calls made on the test's own thread. The
writers under test all lock down on that thread, so every assertion
keeps its strength; a probe that saw nothing still fails its premise.

Closes #33300
@bolichen97
bolichen97 force-pushed the fix/33300-consent-lockdown-filter branch from b952566 to 0a482a4 Compare November 9, 2026 06:41
@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in test/test_webhooks_store.py, test/test_apps_registry.py, test/test_apps_registry_coverage_install.py, test/test_apps_routes_coverage_update_uninstall.py, test/test_apps_worker_auto_approve.py since this head went green (2caa69b -> d2c6817); CI re-running.

The previous head's only red, test/test_dashboard_server_composition_contract.py::test_the_lazy_imports_stay_inside_the_functions_that_need_them (assert 140 == 139, Backend Tests shard 6 on Linux and Windows), was main's: #29509 added a lazy import and #33377 re-pinned the count. The new base carries #33377; this PR's diff is unchanged.

[operator: bolichen97]

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 9, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Test-only fix for #33300. The eight restrict_to_owner size probes now keep only the test thread's calls (test/test_aws_consent.py:76-95, thread filter at :90; test/test_workflows_store_lockdown.py:30-36), so a sel-writer re-lock of its non-empty audit log (src/kiro_crew/sel_append.py:255) can no longer read as the store's lockdown. Every assertion keeps its strength.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 9, 2026
@bolichen97
bolichen97 merged commit f038398 into main Nov 9, 2026
99 checks passed
@bolichen97
bolichen97 deleted the fix/33300-consent-lockdown-filter branch November 9, 2026 08:29
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: test_write_lockdown_precedes_content on macOS

2 participants