Skip to content

test(md-notebook): stub the hook scan in the staging-sync test - #39279

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
crew/unflaky/md-notebook-hook-scan-stub
Nov 19, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
crew/unflaky/md-notebook-hook-scan-stub

Conversation

@CrysisDeu

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: test_a_sync_while_the_temp_is_still_staging_never_commits_it stops failing on Windows CI when nothing is wrong with lock ordering.

On Windows the test fails at test/test_md_notebook_write_safety.py:849:

E   AssertionError: the released sync/save chain never completed; a starved shared to_thread executor holds its git work (flake class 2)

Seen on main at 0330200 (job 135089405882, Backend Tests (Windows) (4), gw3), and earlier on fix/resume-restored-queue c5aa0d8 (check run 119475800273, Backend Tests (Windows) (5)). Both hits I could retrieve are Windows. A scan of 800 failed CI runs back to 11-16 found only the main hit.

Why it matters

It turns main red and blocks unrelated PRs on the Windows shard.

Not a goal

  • Changing how often production runs the hook scan. That trade-off is written up as a proposal in perf(md-notebook): hook scan runs ~3 git processes per run_git call, ~120 starts per Sync #39275 for the module owners.
  • The test's other waits (to_thread(temp_written.wait, 10), _parked_behind). Neither spans any git process (see below).
  • The sibling test_a_symlink_landing_while_a_writer_waits_on_the_lock_is_refused. It runs a real Sync too, but awaits it with no wall-clock bound, so it is slow (32.2 s in the same job) but not exposed.

What changed (motivation → approach → change)

Symptom → cause. The failure text blames a starved shared executor. That is not possible: pytest-asyncio 0.21.2 gives each test its own event_loop, conftest_floor_restores.py shuts its default executor down after each test, and that executor has min(32, cpu+4) ≥ 8 threads (4 xdist workers). The released chain needs at most 2 to 3 at once. Every Sync and save hop is sequential.

What the 30 s _BARRIER_LOST_RUN_SEC drain actually times is real git work. md_notebook.git_ops.run_git runs config_hook_disable_args before every git command, and that scan starts 3 git processes (config, rev-parse --show-toplevel, ls-files -s -z). One whole-scope Sync makes about 30 run_git calls (branch, origin ×2, gitdir, driver ×4, status ~7, auto_commit ~10, fetch, rev-parse ×2, merge, push):

git starts per Sync
before (real scan) ~30 × 4 = ~120
after (scan stubbed) ~30 × 1 = ~30

Plus upload-pack and receive-pack for the real fetch and push, in both cases. On a loaded Windows runner, 120 starts do not fit in 30 s. In the failing job no worker finished a test from 20:16:26 to 20:17:03 (37 s), the sibling real-Sync test took 32.2 s ("elapsed":32.215722), and this test's F printed at 20:17:20.

Change (test only, one file):

  1. In this test only, git_ops.config_hook_disable_args is monkeypatched to return []. The test proves lock ordering between staging, Sync and publish, not hook handling. The hook scan in run_git stays covered in test/test_git_config_hooks.py:
    • test_md_notebook_run_git_carries_the_disable (the disable flags reach the real git argv)
    • test_md_notebook_scan_failure_is_a_git_error (a scan failure fails closed)
    • test_md_notebook_commit_does_not_fire (a planted config hook does not fire on commit)
  2. The real remote and the real fetch/merge/push are kept. _BARRIER_LOST_RUN_SEC and every assertion are unchanged.
  3. The drain's failure text and comment now say what is measured: the released chain's real git work did not finish within the lost-run ceiling.

Other waits checked, not changed. to_thread(temp_written.wait, 10) spans the save's read_vaults, one os.stat, one mkdir and the temp write: no git process. _parked_behind spans the Sync's require_vault (a vaults.json read) up to vault_write_lock, and resolve_auth runs inside the lock: no git process. Neither shares this cause.

Backwards compatibility

Compatible: test-only change. Production code is untouched.

Removes nothing: only a test stub and a failure message change.

Tests

  • test/test_md_notebook_write_safety.py::test_a_sync_while_the_temp_is_still_staging_never_commits_it: the same unconditional assertions (committed list, ls-files, remote ls-tree, save 200, note content, temp gone), with the hook scan stubbed.
  • CI on the current head: pending, link added once Backend Tests (Windows) is green.

Proof: not provable -- test-only change

Static gates run locally: scripts/check_black_formatting.py (black 26.10.1) passed, flake8 passed, isort --check-only passed, mypy on the file passed.

Manual verification

N/A -- test-only change; CI runs it on Linux, macOS and Windows.

Real-machine verification

N/A -- the diff changes a test stub and a message; no product code runs differently.

Related Issues

Related: #39275 (the production hook-scan cost; this PR only removes it from this one test)

Sibling sweep

Swept: git grep -n '"/api/sync"' -- 'test/test_md_notebook_*.py', then each hit's bounded waits checked for a real Sync inside them
Sibling: test/test_md_notebook_write_safety.py:92 -- not affected -- its lost-run waits bound lock parking; the real Sync is awaited unbounded
Sibling: test/test_md_notebook_write_safety.py:201 -- not affected -- awaits its real Sync with no wall-clock bound
Sibling: test/test_md_notebook_write_safety.py:675 -- not affected -- real Sync awaited unbounded; its lost-run waits bound staging and rename only
Sibling: test/test_md_notebook_notes.py:297 -- not affected -- plain awaited Sync, no lost-run bound
Sibling: test/test_md_notebook_git_trust.py:256 -- not affected -- plain awaited Sync, no lost-run bound

Pattern harvest

Rule candidate: review-prompt
Pattern: a wall-clock lost-run bound wrapped around a chain of many real subprocess starts measures the runner; stub the per-call overhead that the test does not prove

Checklist

  • A Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@CrysisDeu
CrysisDeu requested review from a team and KiroCrewCaptain as code owners November 18, 2026 21:30
@CrysisDeu
CrysisDeu requested a review from bolichen97 November 18, 2026 21:30
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 18, 2026
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 7dc11e671e9c8ce0ee73cf6bed5bf235a900ea4e ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 7dc11e671e9c8ce0ee73cf6bed5bf235a900ea4e touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 7dc11e671e9c8ce0ee73cf6bed5bf235a900ea4e — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Test-only. The stub replaces a module-global that run_git looks up on every call (git_ops.py:522), so it applies, and the hook-scan behaviour is still tested in test_git_config_hooks.py.

Notes

[DESIGN-REVIEWED] 7dc11e6

Description read

[DESCRIPTION-READ] e549101aa91f0ad6c93ad6be46c55ba559c206fd3945b3a83621568e151e5472

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 7dc11e671e9c8ce0ee73cf6bed5bf235a900ea4e — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 7dc11e6

Verdict parsed from the review's SHA-scoped output markers for commit 7dc11e671e9c8ce0ee73cf6bed5bf235a900ea4e.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 7dc11e671e9c8ce0ee73cf6bed5bf235a900ea4e false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 7dc11e671e9c8ce0ee73cf6bed5bf235a900ea4e and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 7dc11e6

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 7dc11e671e9c8ce0ee73cf6bed5bf235a900ea4e false-positive: <reason, 20 to 500 characters>

@CrysisDeu

Copy link
Copy Markdown
Contributor Author

Backend Tests (3.12, 8) red is inherited from main, not this diff: test_bare_session_handle_guard flags test/test_kas_token_refresh_recovery.py:245 (AcpSessionHandle.__new__), which this PR does not touch. Tracked in #39261; fixes open in #39262 / #39274. I will rebase once one merges.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 18, 2026
… drain times lock ordering, not ~120 git starts
@CrysisDeu
CrysisDeu force-pushed the crew/unflaky/md-notebook-hook-scan-stub branch from b30a6c9 to 7dc11e6 Compare November 19, 2026 02:10
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Nov 19, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider estimates a 5% chance that this change introduces a defect, within the 12% limit for approving on its own, but it is held at tier 2: reviewer-finding: Design Review.

Test-only change. In one lock-ordering test, it stubs the md_notebook hook scan (git_ops.config_hook_disable_args) to return [] and rewords the timeout's failure text. The stub takes effect: run_git looks the function up as a module global at git_ops.py:521-526, the test patches that same module, and the helpers say the module is never reloaded. The lambda accepts run_git's positional and keyword arguments. The stub sits inside an existing MonkeyPatch.context, so it is undone when the block ends. Every assertion and the 30 s _BARRIER_LOST_RUN_SEC limit are unchanged, and the real fetch, merge and push still run. The hook scan keeps its own coverage in test_git_config_hooks.py. All eight Windows backend shards passed on this head commit. No production behaviour changes, so there is no regression or trust-boundary concern.

Tier: 2, held for a person: reviewer-finding: Design Review.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 7dc11e671e · 21 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 19, 2026 10:20

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed head 7dc11e6. The stub takes effect: run_git looks up the module-global config_hook_disable_args on every call, and the fixture never reloads git_ops. The patch is scoped to the existing MonkeyPatch.context, and both tasks are drained before that context exits, so nothing leaks into other tests. _BARRIER_LOST_RUN_SEC and every assertion are unchanged. The three hook behaviours this test stops exercising are still pinned in test/test_git_config_hooks.py (lines 289, 310 and 322). All eight Backend Tests (Windows) shards passed on this head.

Done well: the PR removes work the test never needed to prove instead of raising the timeout. The new failure text names the bound it measures.

Optional:

  • Lines 780 to 788 of test/test_md_notebook_write_safety.py still blame shared executor starvation, which the body rules out. You could reword them to match the new drain comment.
  • The stub returns [], so it also drops the submodule recursion pins. Returning list(git_config_hooks._SUBMODULE_RECURSION_PINS) would keep the argv the same as production, or the comment could say that the pins are dropped too.
  • The body's Tests section still says CI is pending. You could link the green Windows run.

@KiroCrewCaptain
KiroCrewCaptain merged commit d553f28 into main Nov 19, 2026
156 of 203 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the crew/unflaky/md-notebook-hook-scan-stub branch November 19, 2026 10:20
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants