Skip to content

test(skill-projection): let refusal-count walks finish on a slow runner - #28718

Merged
iamwhatever merged 1 commit into
mainfrom
crew/conductor/scanflake-fx1
Nov 3, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
crew/conductor/scanflake-fx1

Conversation

@RohanK6

@RohanK6 RohanK6 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The skill-projection refusal-count tests stop failing on a slow CI runner.

test_a_refused_lease_unlink_reports_through_the_same_throttle failed in run 41534699427 with 2 similar refusal(s) where it expects 3. The prune walk has a real-time budget, _PRUNE_MAX_SECONDS_PER_RUN = 0.4 s. A slow runner spends it after two of the three candidates, so one refusal is never counted.

Why it matters

A red that the diff did not cause costs every PR author a re-run and a diagnosis. Two sibling tests in the same file count refusals from the same walk and carry the same risk.

Not a goal

What changed (motivation → approach → change)

Symptom: a refusal count one short. Cause: the 0.4 s prune budget runs out mid-walk on a slow runner. Fix: a test helper, _ProjectionClock, gives the projection modules their own monotonic clock that only the test moves. No time passes during the walk, so the 0.4 s budget is never spent and stays as is. The clock starts at 1000.0, because the throttle reads a last-report time of 0.0 as "never reported". The interval step moves the same clock with advance(), not a real-time offset. Other time names fall through to the real module. The three tests:

  • test_a_refused_lease_unlink_reports_through_the_same_throttle (the CI red)
  • test_a_refused_unlink_is_reported_once_per_interval_with_its_count
  • test_a_refusal_that_is_not_a_permission_problem_is_worded_neutrally

Backwards compatibility

Compatible: test-only change; no product code is touched.
Removes nothing: no product path, record or key changes.

Tests

Proof: not provable -- test-only change

Forced condition (testing-conventions, proving a determinism fix): a scratch probe, not committed, made each refused unlink sleep 0.25 s of real time, which is the slow-runner path.

  • On the parent: the target test failed with 2 similar refusal(s), the exact CI message; the 5-candidate sibling failed with 4 similar refusal(s) expected.
  • On this change: all three passed.

Repeats: test/test_native_skill_projection_scan_bounds.py passed 5 runs with -n0, once with -n 4, and once in a shuffled order (seed 7193), 44 tests each. The sibling files test_native_skill_projection_alias_reclaim.py and test_native_skill_projection_lease_scan.py pass (85 tests).

Manual verification

N/A — unit coverage sufficient: the change is to tests only, and the forced slow path above reproduces the CI failure.

Related Issues

Fixes #28685

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@RohanK6
RohanK6 requested review from a team and KiroCrewCaptain as code owners November 2, 2026 23:25
@RohanK6
RohanK6 requested a review from chenmingwei23 November 2, 2026 23:25
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 2, 2026
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 2bc69d8580cb3a89a858c59ee647b507a58051db ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 2bc69d8580cb3a89a858c59ee647b507a58051db touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 2bc69d8580cb3a89a858c59ee647b507a58051db — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the test removes the real-time budget dependency, and no product code changes.

[DESIGN-REVIEWED] 2bc69d8

Description read

[DESCRIPTION-READ] 413de4cd49388376e0d1406eae731ee70dd3f15b94e0d27488fee1a7b652c44f

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 2bc69d8580cb3a89a858c59ee647b507a58051db and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 2bc69d8

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 2bc69d8580cb3a89a858c59ee647b507a58051db false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 2bc69d8580cb3a89a858c59ee647b507a58051db — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 2bc69d8

Verdict parsed from the review's SHA-scoped output markers for commit 2bc69d8580cb3a89a858c59ee647b507a58051db.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 2bc69d8580cb3a89a858c59ee647b507a58051db false-positive: <reason, 20 to 500 characters>

@RohanK6
RohanK6 force-pushed the crew/conductor/scanflake-fx1 branch from 370e950 to 2bc69d8 Compare November 3, 2026 02:27
@RohanK6

RohanK6 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=e02c5798e291 Widening the prune deadline leaves refusal counts timing-dependent

fixed in 2bc69d8: the budget widening is gone; _PRUNE_MAX_SECONDS_PER_RUN stays 0.4 s.
_ProjectionClock binds the projection modules own monotonic to a test-moved clock starting at 1000.0, so no time passes during the walk and the throttle still sees a nonzero last-report time.
The two interval transitions move that same clock with advance(), not a real-time offset.
Forced slow runner (0.25 s real sleep per refused unlink): the target and the 5-candidate sibling fail on main and pass here.
self-added: yes
mechanism: controlled projection-module clock replaces the widened prune budget

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 3, 2026
@RohanK6
RohanK6 enabled auto-merge (squash) November 3, 2026 04:27

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 2bc69d8.

The three refusal-count tests now run the projection modules on a test-moved monotonic clock, so the 0.4 s prune budget cannot run out mid-walk and drop a count. The facade is a mirrored module, so replacing projection.time reaches the part modules where the deadline and the throttle read time.monotonic():

deadline = time.monotonic() + _PRUNE_MAX_SECONDS_PER_RUN
reclaimed = 0
examined = 0
ledger = _ViewLedgerWrites(directory, crew_home_id)
for index, path in enumerate(candidates):
if reclaimed >= cap:
logger.debug(
"skill projection: reclaim cap reached (%d); the rest drains on later spawns",
cap,
)
break
if time.monotonic() >= deadline:
and
global _UNLINK_WARNING_LAST, _UNLINK_WARNING_SUPPRESSED
now = time.monotonic()
with _UNLINK_WARNING_LOCK:
if _UNLINK_WARNING_LAST and now - _UNLINK_WARNING_LAST < _UNLINK_WARNING_INTERVAL_SECS:
_UNLINK_WARNING_SUPPRESSED += 1
return
suppressed = _UNLINK_WARNING_SUPPRESSED
_UNLINK_WARNING_SUPPRESSED = 0
_UNLINK_WARNING_LAST = now
. The clock starting at 1000.0 keeps the throttle's 0.0 "never reported" sentinel meaningful. No product code changes.

The earlier GPT finding about widening the deadline was fixed by dropping the widening; current-head AI lanes have no findings, and checks plus PR Readiness are green.

— crew-pr-reviewer (r143)

@iamwhatever
iamwhatever merged commit 8f0c906 into main Nov 3, 2026
149 of 158 checks passed
@iamwhatever
iamwhatever deleted the crew/conductor/scanflake-fx1 branch November 3, 2026 05:22
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: test_a_refused_lease_unlink_reports_through_the_same_throttle reports the first-refusal message instead of the throttled summary

2 participants