Skip to content

test(host-git-repo-exec): make the nested-repo fixture fire without racy timing - #25571

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
crew/issue-triage/fix-25499
Oct 29, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
crew/issue-triage/fix-25499

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: test_probes_see_a_driver_declared_in_a_nested_repo passes every time, in a busy CI shard as well as alone.

The test failed once in Backend Tests (3.12, 5) with AssertionError: fixture is inert: bare git status --porcelain ran no marker, on a PR that changed only website/ files. Run alone, it passes.

Why it matters

A red that has nothing to do with the PR costs every author a rerun and teaches people to ignore red tests. The liveness check it trips is the guard that keeps the security assertions after it honest.

Not a goal

What changed (motivation → approach → change)

The fixture's planted filter did not always run, so the liveness check failed. No other test leaks state into this one. The cause is timing in the fixture itself.

Fixture step Before After
Dirty edit of x.txt "dirty\n", a new size "y\n", the same size
Git sees the edit by size alone, no hash a changed mtime, so it must hash
Clean filter runs on .gitattributes, only if its index entry is racily clean x.txt, every time

Git counts an index entry as "racily clean" when its file was written in the same moment as the index. Only then does it re-hash .gitattributes and run the filter. In a fast run that is the normal case. In a loaded shard, the gap between writing the file and git add grows past that moment, and the filter never runs.

A same-size edit with an old mtime leaves git no shortcut: it must hash x.txt through the filter. _with_nested_filter_repo makes that edit, and a comment there says why. test_a_nested_repo_with_its_config_deleted_is_refused checks the same helper's liveness and gets the same fix.

Backwards compatibility

Compatible: test-only change; no product code is touched.

Removes nothing: only a test fixture's file content changes, and a test is added.

Tests

New test_the_nested_fixture_fires_without_a_racily_clean_index moves the nested index's mtime 60 s past .gitattributes, the state a slow shard produces, then requires the filter to fire.

  • Forced condition: on the old fixture ("dirty\n") the new test fails with the CI error, fixture is inert: bare git status --porcelain ran no marker. With the fix it passes.
  • Slow-shard repro: a 0.3 s pause before the nested git add made the original test fail 1 of 3 runs on the old fixture and pass 3 of 3 with the fix.
  • Repeats: TestNestedRepoDrivers + TestNestedProbeFailures, 20 of 20 runs green.
  • Shuffled order: the whole file (52 tests) in 3 seeded shuffled orders, all green.
  • Related backend tests (scripts/local-gate.py): 271 passed. Flake ratchet: 64 passed. Semgrep on the diff: 0 findings.

Proof: not provable -- test-only change

Manual verification

Measured with a standalone script that builds the same fixture (git 2.50.1): the filter's stdin shows it ran on .gitattributes alone. A pause of 0.3 s or more before git add made bare git status inert every time.

Related Issues

Closes #25499

Sibling sweep

Swept: git grep -nE '_prove_planted_vector_fires(.*"status"' -- test/
Sibling: test/test_host_git_repo_exec_neutralization.py:1030 -- changed here, through the shared _with_nested_filter_repo fixture
Sibling: test/test_host_git_repo_exec_neutralization.py:600 -- tracked in #25469, which pins an mtime for the post-index-change probe
Sibling: test/test_host_git_repo_exec_neutralization.py:723 -- not affected -- fires through core.fsmonitor, which every status runs
Sibling: test/test_host_git_repo_exec_neutralization.py:868 -- not affected -- fires through core.fsmonitor, which every status runs
Sibling: test/test_host_git_repo_exec_neutralization.py:412 -- not affected -- fires through core.fsmonitor, which every status runs

Pattern harvest

Rule candidate: review-prompt
Pattern: a fixture that needs git to hash a file makes a same-size edit with a changed mtime; a size change lets git skip the hash, and "racily clean" timing decides the rest.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

🤖 AI-assisted PR 🤖

@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners October 29, 2026 02:33
@iamwhatever
iamwhatever requested a review from CrysisDeu October 29, 2026 02:33
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 47bc25010ddbce5be3b51225355e6260b329ad59 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 47bc25010ddbce5be3b51225355e6260b329ad59 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 47bc25010ddbce5be3b51225355e6260b329ad59 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this is a test-only fix for a timing flake in the fixture itself, and the new test pins the slow-shard case.

[DESIGN-REVIEWED] 47bc250

Description read

[DESCRIPTION-READ] 9b110da2d020e31d475d8af39aac47dd4d9cb048975da2dec5ef5bd24c4eadab

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 47bc25010ddbce5be3b51225355e6260b329ad59 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 47bc250

Verdict parsed from the review's SHA-scoped output markers for commit 47bc25010ddbce5be3b51225355e6260b329ad59.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 47bc25010ddbce5be3b51225355e6260b329ad59 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 47bc25010ddbce5be3b51225355e6260b329ad59 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 47bc250

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 47bc25010ddbce5be3b51225355e6260b329ad59 false-positive: <reason, 20 to 500 characters>

@iamwhatever
iamwhatever force-pushed the crew/issue-triage/fix-25499 branch from 21c7d6d to 47bc250 Compare October 29, 2026 02:45
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 29, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Review-ready at head 47bc250: PR Readiness passed (label readiness: passed), 104 of 105 checks green, GPT and Opus reviewed this head, Design PASS, no open threads, MERGEABLE. The one red is Dependency License Gate, the repo-wide setting that fails on every PR. Backend Tests shard 6 went red once on the known flake #25586 (fix in #25469); one rerun passed. Test-only change; waiting for a maintainer's review. Not merged or approved by the agent.

🤖 AI-assisted comment 🤖

@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 29, 2026 06:04

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 file). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change that makes the nested-repo filter fixture keep the file size so git must hash it, plus a test proving the fixture fires without a racily clean index.

@KiroCrewCaptain
KiroCrewCaptain merged commit 1a6cecf into main Oct 29, 2026
154 of 165 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the crew/issue-triage/fix-25499 branch October 29, 2026 06:04
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 29, 2026
KiroCrewCaptain pushed a commit that referenced this pull request Oct 31, 2026
…-deleted test (#26694)

The config-deleted case's self-check went inert on two macOS shards (#25374).
Both runs merged onto a main from before #25571, whose nested fixture wrote a
size-changed x.txt; on a non-racily-clean index that file never reaches the
clean filter. Pin that index shape here so the fixture keeps proving the
vector fires under slow-shard timing, as the driver test beside it already does.

Closes #25374
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: test_probes_see_a_driver_declared_in_a_nested_repo fails in a full shard, passes alone

2 participants