Repository navigation
test(apps): move the settle-window clock only on the listing's own walk - #39010
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS This only changes tests, docs and a review rule. The spy now uses the same caller-frame filter as #38980, which removes the cross-test coupling at its source. Notes
[DESIGN-REVIEWED] 24ef86c Description read[DESCRIPTION-READ] 0152121201f97d9dd5f8a6f29c2d98cadb68b980938979f8a6d959b1ca721c38 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Review detailsNo findings. The one candidate is only a difference in wording between the AUTOSDE rule text and the conventions doc. It does not change what any code does at runtime. The test change works: [OPUS-REVIEWED] 24ef86c Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
76516af to
4fd2b72
Compare
4fd2b72 to
6392928
Compare
The settle-window test advanced its clock on every os.scandir of the apps folder, from any module or thread, because manager_records.os is the shared os module. Filter it by caller frame, as the unchanged-folder test does, and write the rule for counting and recording spies into the testing conventions and the AUTOSDE review list.
6392928 to
24ef86c
Compare
|
Main already fixed the unchanged-folder counter in #38980, using the same caller-frame filter. I rebased this PR onto it. What is left here: the same filter on the settle-window test's clock spy, plus the convention and AUTOSDE rule that keep the class from coming back. |
实机验证 (real-machine verification)
|
Problem / Motivation
Goal:
TestListingReusesTheWalk::test_an_unchanged_folder_is_not_walked_againpasses whatever ran before it on the xdist worker.test/test_app_manager_installed_record.py::TestListingReusesTheWalk::test_an_unchanged_folder_is_not_walked_againfailed withassert 2 == 1(the apps folder was "walked" twice) onBackend Tests (3.12, 8)for five open PRs: #38902, #38891, #38803, #37604, #33979.Why it matters
Shard 8 holds this file, so every PR whose shard 8 put this test after the wrong neighbour went red with no relation to its own diff.
Not a goal
list_apps, its walk cache, the path gate or themc-pathrespool.os.scandir. They are tracked in test: sweep counting spies that patch the stdlib os.scandir for the whole worker #39021.What changed (motivation → approach → change)
Symptom: the spy recorded two scans of the apps folder for two
list_apps()calls on an unchanged folder.Root cause: the spy did
monkeypatch.setattr(manager_records.os, "scandir", counting).manager_records.osis the stdlibosmodule, so the spy counted everyos.scandirof that folder in the worker, from every module and thread.list_apps()caused the second scan itself. Whendetect_orphaned_builtinshad no cached result,list_appsvalidated the built-in manifests. The path gate then listed the home's credential masks on the sharedmc-pathrespool (kiro_crew.apps.credential_masks._installed_manifests, which runs its ownos.scandir(apps)). The second scan's stack is incredential_masks.py, not inmanager_records.py, so the product cache behaved correctly.No thread leaks across tests.
mc-pathresis the process-wide shared executor, and this test's ownlist_apps()call submitted the listing. Whether the orphan cache was cold depended on the tests that ran before on the worker. The CI progress artifacts of all four failing shard-8 jobs show the same order:test/test_app_backend.py::TestBootSpawnLatency, then atest_app_backend_stale_reap.pytest, then this test, within about 0.2 s.Change: the unchanged-folder counter itself was fixed on main by #38980 (it counts only calls whose caller frame is
_recorded_app_names). This PR, rebased on it, covers the rest of the class:test_a_scan_started_inside_the_settle_window_is_not_reusedhad the same spy shape: it advanced itsManualClockon everyos.scandirof the apps folder, from any module or thread. It now moves the clock only when the caller frame is_recorded_app_names, the same filter test(apps): count only the listing walk in the unchanged-folder test #38980 uses.docs/system-specs/common/testing-conventions.mdgets a new rule: a counting spy counts only the calls the code under test makes. It shows the wrong form and the caller-frame form. It also gives the module-binding form, and says whentest/test_facade_mirror_guards.pyrules that form out (osis a name thekiro_crew.apps.managerfacade mirrors). The rule carries the numbers below.AUTOSDE.yamlitem 11 now flags a newly added counting or recording spy through<mod>.os.<name>that counts every caller, the same way it flags a rebound stdlib clock.Only tests and docs/spec files change. No product code, no CI config.
Backwards compatibility
Compatible: only a test, a convention doc and a review rule change. No product behaviour moves.
Removes nothing: the diff only adds a doc rule and a review-rule clause, and narrows which calls a test spy counts.
Tests
Proof: not provable -- test-only change
Evidence on a Linux DevSpace with 8 cores, 30 runs each. "After" is the caller-frame filter on both spies (the same filter #38980 uses for the counter):
TestBootSpawnLatency+test_reap_skips_recycled_pid+ this class,-n0-n 8 -p randomly,test_app_backend.py+test_app_backend_stale_reap.py+ this fileOn a second host (aarch64), with 16 CPU spinners,
-n 8 -p randomlyand the same files, this test failed 14/30 before the change and passed 30/30 after. The-n0order failed 30/30 before and passed 30/30 after.TestBackendLifecycle(4 tests) fails on that host before and after, because it has no sandbox backend (SandboxUnavailableError). That failure is unrelated.This file plus the 69 test files that read
testing-conventions.mdorAUTOSDE.yaml: 3059 passed, 54 skipped.Manual verification
N/A — the repro runs above are the verification. Local
gptandopusreviews of the change: PASS, no findings.Related Issues
N/A — found by the Linux flaky-test scan across the five PRs above.
Sibling sweep
Swept: git grep -nE 'setattr([A-Za-z_.]*os, "scandir"' -- test/
Sibling: test/scale_budget/conftest.py -- tracked in #39021
Sibling: test/scale_budget/test_listing_budget.py -- tracked in #39021
Sibling: test/test_agent_scratch.py -- tracked in #39021
Sibling: test/test_agents_dir_fingerprint_skips_aliases.py -- tracked in #39021
Sibling: test/test_agents_dir_memo.py -- tracked in #39021
Sibling: test/test_app_command_agent_guard.py -- tracked in #39021
Sibling: test/test_backend_tmp.py -- tracked in #39021
Sibling: test/test_chat_folder_steering_dirs.py -- tracked in #39021
Sibling: test/test_config_app_owned_sections.py -- tracked in #39021
Sibling: test/test_crew_log_fullscan_guard.py -- tracked in #39021
Sibling: test/test_crewmate_prune_migration.py -- tracked in #39021
Sibling: test/test_dev_fleet_disk_measure_without_du.py -- tracked in #39021
Sibling: test/test_dev_fleet_npm_preflight.py -- tracked in #39021
Sibling: test/test_display_verdicts.py -- tracked in #39021
Sibling: test/test_folder_steering.py -- tracked in #39021
Sibling: test/test_history_listing_index.py -- tracked in #39021
Sibling: test/test_ledger_sweep.py -- tracked in #39021
Sibling: test/test_managed_agent_shadow.py -- tracked in #39021
Sibling: test/test_markdown_agent_specs.py -- tracked in #39021
Sibling: test/test_mate_config_migration.py -- tracked in #39021
Sibling: test/test_mcp_apps_render_claim_recovery.py -- tracked in #39021
Sibling: test/test_mcp_command_casing_shared.py -- tracked in #39021
Sibling: test/test_mcp_gateway_command_casing.py -- tracked in #39021
Sibling: test/test_member_essential_context.py -- tracked in #39021
Sibling: test/test_member_essential_context_left_out_sources.py -- tracked in #39021
Sibling: test/test_member_essential_context_resource_admission.py -- tracked in #39021
Sibling: test/test_native_skill_projection_lease_scan.py -- tracked in #39021
Sibling: test/test_native_skill_projection_scan_bounds.py -- tracked in #39021
Sibling: test/test_owner_only_files.py -- tracked in #39021
Sibling: test/test_pre_fix_fork_carrier_repair.py -- tracked in #39021
Sibling: test/test_project_scan_walker.py -- tracked in #39021
Sibling: test/test_project_tree.py -- tracked in #39021
Sibling: test/test_registry_store_art_prewarm_rewarm.py -- tracked in #39021
Sibling: test/test_sandbox_governance_mask_model_loaders.py -- tracked in #39021
Sibling: test/test_session_storage_restore.py -- tracked in #39021
Sibling: test/test_side_readonly_spec_reap.py -- tracked in #39021
Sibling: test/test_skill_mapping_and_lazy_default.py -- tracked in #39021
Sibling: test/test_skill_pending_api.py -- tracked in #39021
Sibling: test/test_skill_trust_store_project_skill_loading.py -- tracked in #39021
Sibling: test/test_skills_cold_start.py -- tracked in #39021
Sibling: test/test_snapshot_unreadable_entries.py -- tracked in #39021
Sibling: test/test_spawn_parent_available_agents.py -- tracked in #39021
Sibling: test/test_subagent_panel_durable_replay.py -- tracked in #39021
Sibling: test/test_trash_pinned_sibling_removal.py -- tracked in #39021
Pattern harvest
Rule candidate: review-prompt
Pattern: counting or recording spy set through
<mod>.os.<name>patches the stdlib for the whole workerChecklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.