Skip to content

test(dashboard): give the slots-flush read budget its own SEL audit window - #29588

Closed
iamwhatever wants to merge 1 commit into
mainfrom
crew/issue-triage/KiroCrew-29534
Closed

iamwhatever wants to merge 1 commit into
mainfrom
crew/issue-triage/KiroCrew-29534

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: test_one_slots_flush_stays_under_its_per_slot_facade_read_ceiling measures only per-row facade reads, so it stops failing on Windows when nothing per-row changed.

On Backend Tests (Windows) shard 3 the test fails on unrelated PRs (#29516, #29524, #29528, #29368, #29430, #29537) with one slots flush costs 6.005 facade reads per slot (ceiling 6): __path__ is read 401 times for 200 extra slots instead of 400.

Why it matters

Every PR whose Windows shard 3 lands this test goes red for a reason it did not cause, and a rerun can fail again on the same head. Authors lose time and stop trusting the read budget.

Not a goal

  • Changing the ceiling (SLOTS_FLUSH_READS_PER_SLOT stays 6, still an integer bar).
  • Changing how the WS grant audit is deduplicated in production code.
  • A platform skip.

What changed (motivation → approach → change)

Cause. Each slots frame sent to a dashboard socket records one grant audit, deduplicated per 5-minute window in ws_event_scope._sel_last_audit. That dict is module state, shared by every test on the worker. When an earlier test opened the window, it can close in the middle of this test. The flush that crosses it runs from kiro_crew.sel import sel (a facade), which reads __path__ once. When that flush is the measured 400-slot one, the count is 401 instead of 400. The single warm call cannot absorb it, because the window was opened by another test. Windows hits it because its shard timing puts the expiry there.

Change. The test gets its own dedup dict and a window that never closes. The one audit lands in the first warm call, and the measured flushes count only per-row reads.

Backwards compatibility

Compatible: test-only change.

Removes nothing: the diff only adds two monkeypatch.setattr lines to one test.

Tests

test/test_facade_read_budget.py, both tests:

  • Forced expiry (the old window closes right before the measured large flush, by jumping ws_event_scope.time.monotonic 301 s): the test on main fails with 6.025 facade reads per slot (ceiling 6); with this change it passes. Clearing the dedup dict at that point on main reproduces the CI message exactly (6.005, __path__ 401).
  • Mutant: one extra facade read per row in serialize_slots still fails with this change: one slots flush costs 7.0 facade reads per slot (ceiling 6).
  • Determinism: 10 runs in file order and 10 in reversed order, all pass.

Proof: not provable -- test-only change

Manual verification

N/A — the failure is Windows-only and cannot run here (Linux host); the forced-expiry repro above shows the same count on Linux, and CI's Windows shard confirms.

Related Issues

Fixes #29534

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners November 4, 2026 11:26
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 3bcf57b7ef76ad5ddadafc9a63833acd7c63852c touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 3bcf57b7ef76ad5ddadafc9a63833acd7c63852c ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 3bcf57b7ef76ad5ddadafc9a63833acd7c63852c — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this is a test-only fix that isolates the module-global dedup state causing the cross-test leak.

Notes

  • Other tests that count reads or audits through ws_event_scope share the same _sel_last_audit leak. A conftest fixture that resets it for each test would fix all of them, so it may be worth a follow-up.

[DESIGN-REVIEWED] 3bcf57b

Description read

[DESCRIPTION-READ] 308cae60fa033ae52cfeb03bc2385bc8ba4da8eb2304b4b5468bfe733d6ef183

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 3bcf57b7ef76ad5ddadafc9a63833acd7c63852c — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 3bcf57b

Verdict parsed from the review's SHA-scoped output markers for commit 3bcf57b7ef76ad5ddadafc9a63833acd7c63852c.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 3bcf57b7ef76ad5ddadafc9a63833acd7c63852c false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 3bcf57b7ef76ad5ddadafc9a63833acd7c63852c and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 3bcf57b

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 3bcf57b7ef76ad5ddadafc9a63833acd7c63852c false-positive: <reason, 20 to 500 characters>

@iamwhatever

Copy link
Copy Markdown
Contributor Author

Superseded by #29549 (merge 82dc9d7), which counts facade reads only on the test thread. The extra __path__ read in #29534 came from the sel-writer thread: when the 300 s grant-audit dedup window expires mid-test, one SEL write runs and its lock path imports through a facade. A forced-expiry repro on new main (c3b4457, window jumped 301 s before the measured flush) passes without this change, so closing this PR. Branch kept.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: test_one_slots_flush_stays_under_its_per_slot_facade_read_ceiling (one extra __path__ read on Windows)

1 participant