Skip to content

test(platform-log): give the short-writer serialize test a wide budget - #24150

Merged
KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
IngridMorstrad:test/platform-log-append-serialize-budget
Oct 26, 2026
Merged

KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
IngridMorstrad:test/platform-log-append-serialize-budget

Conversation

@IngridMorstrad

@IngridMorstrad IngridMorstrad commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Stop test_concurrent_short_writers_serialize failing on slow Windows runners with a deadline error that has nothing to do with what it tests.

test/test_platform_log_append.py::test_concurrent_short_writers_serialize failed on Backend Tests (Windows) (3) with TimeoutError: decision log append deadline expired, raised from the write loop in append_line (src/kiro_crew/platform_log_append.py:350), and passed on re-run (run 40100625828, PR #23697).

Why it matters

A red Windows shard on an unrelated PR costs a re-run and a look at the log. It also points the reader at the decision-log deadline, which is working as designed.

Not a goal

  • Changing _APPEND_TIMEOUT_SECONDS or anything else in platform_log_append.py. The 0.5 s budget is deliberate: it keeps an observation from occupying a caller on contention.
  • Changing the Windows lock poll in platform_lock_compat.py.

What changed (motivation → approach → change)

  • Symptom: the last of four writers runs out of its append budget partway through its own writes.
  • Root cause: the test forces every os.write down to 3 bytes, so each 111-byte row takes ~37 writes. append_line gives each append one 0.5 s budget, started after the open, that covers the lock wait and every write. The last writer to get the lock waits out the other three's ~37 writes each and then makes its own: about 148 serialized writes plus lock hand-offs, so ~3.4 ms per write. A loaded Windows runner, which also takes the lock by polling msvcrt.locking, can be slower than that. The test checks serialization. The deadline is covered by test_contended_append_times_out_without_writing.
  • Change: in this test only, raise _APPEND_TIMEOUT_SECONDS to SERIALIZE_APPEND_CEILING_SECS = 10.0, a named lost-run ceiling (testing-conventions D8). The comment beside it cites the measurement: the test's call phase takes at most 0.078 s (median 0.034 s) over 35 ci.yml runs of Windows shard 3 (windows-pytest-progress-3), and 0.72 s per run on Linux with 4 ms added to every write. 10 s is over 10x both and under half the suite's --timeout=120. test_decisions_points.py, test_decisions_tool_risk.py and the decisions end-to-end tests already raise this budget the same way. The bound is raised, not removed: a stuck lock still fails the test by name, with TimeoutError from append_line, well inside wait_until's own timeout. No retry, skip or sleep is added.

Backwards compatibility

Compatible: test-only change, no production code touched.

Removes nothing: the diff only raises a monkeypatched constant in one test.

Tests

Proof: not provable -- test-only change

Red/green reproduction (Linux, in-process harness that runs the test body N times; per-write latency is added under the test's 3-byte shim, and "flat poll" sets the POSIX lock back-off to Windows' flat _LOCK_POLL_SECS poll):

test version per-write latency lock poll CPU load runs result
origin/main 0 ms POSIX back-off none 200 200 pass
origin/main 0 ms POSIX back-off 3 / 8 / 16 hogs on the test's core (taskset -c 0) 100 each all pass
origin/main 2 ms POSIX back-off none 30 30/30 TimeoutError: decision log append deadline expired (from platform_log_append.py:350)
origin/main 3 ms flat (Windows-like) none 50 49 TimeoutError at :350, 1 lock-wait OSError
origin/main 3 ms flat 16 hogs, taskset -c 0 30 28 TimeoutError at :350, 2 pass
this PR 2 ms POSIX back-off none 300 300 pass
this PR 3 ms POSIX back-off none 200 200 pass
this PR 5 ms POSIX back-off none 200 200 pass
this PR 1 / 3 / 4 ms flat none 300 / 200 / 200 all pass
this PR 3 ms flat 16 hogs, taskset -c 0 150 150 pass
this PR (named ceiling, head 78b18e5) 4 ms flat none 100 100 pass

The red runs fail in the same frame as CI (_remaining(deadline) in the write loop). At 4 to 5 ms some runs fail earlier, at file_lock, with could not acquire exclusive file lock: still held after waiting 0.50s, which is the same budget running out during the lock wait. CPU contention alone did not reproduce it on Linux, where a write is far under 1 ms. The failure needs per-write latency near 0.5 s / 148, which fits a slow Windows runner.

Local gate: python3 scripts/local-gate.py passed (backend related 179 passed / 7 skipped; frontend related 403 files, 11,983 tests passed). black, isort, flake8, comment-history, brand, file-line-cap and the flake-pattern ratchet (compared with badd8f5aa849) all pass. mypy was not run: no src/ change.

Manual verification

N/A: test-only change. The reproduction above is the verification.

Related Issues

Closes #24149

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running labels Oct 25, 2026
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 78b18e5702a5f47a3ca64be2f1fcbeeebc8366e5 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ⏭️ skipped

Revision 78b18e5702a5f47a3ca64be2f1fcbeeebc8366e5 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 78b18e5702a5f47a3ca64be2f1fcbeeebc8366e5 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the change is test-only and widens one monkeypatched budget that append_line reads at call time. The deadline behaviour keeps its own dedicated test.

[DESIGN-REVIEWED] 78b18e5

Description read

[DESCRIPTION-READ] d12cee7c69216841f68b0480ff1b661657ea05024a48f34f5357c3d51695e4f6

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 78b18e5702a5f47a3ca64be2f1fcbeeebc8366e5 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 78b18e5

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 78b18e5702a5f47a3ca64be2f1fcbeeebc8366e5 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 78b18e5

test_concurrent_short_writers_serialize timed out on Backend Tests
(Windows) with "decision log append deadline expired" and passed on
re-run. Four writers are forced into 3-byte writes, so the last one to
get the lock waits out the other three's ~37 writes each and then makes
its own, all inside one 0.5s _APPEND_TIMEOUT_SECONDS budget: about
3.4ms per write, which a loaded Windows runner can exceed.

The test checks serialization, and the deadline has its own test, so
raise the budget in this test only, to a named lost-run ceiling of 10s
sized from the measured Windows CI durations. A stuck lock still fails
it.
@IngridMorstrad
IngridMorstrad force-pushed the test/platform-log-append-serialize-budget branch from 837520e to 78b18e5 Compare October 25, 2026 23:27
@IngridMorstrad

Copy link
Copy Markdown
Contributor Author

Addressed the GPT 6.1 D8 finding in 78b18e5. The bare 10.0 literal is now SERIALIZE_APPEND_CEILING_SECS, a named lost-run ceiling, and the comment beside it cites the measurement: the call phase takes at most 0.078 s (median 0.034 s) over 35 ci.yml Windows shard 3 runs (windows-pytest-progress-3), and 0.72 s per run on Linux with 4 ms added to every write. 10 s is over 10x both and under half of --timeout=120. It expires loudly with TimeoutError from append_line.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 25, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 26, 2026 02:03

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change that raises the append budget for one serialization test so a loaded runner does not fail it on the deadline.

@KiroCrewCaptain
KiroCrewCaptain merged commit d55e2ec into kirodotdev:main Oct 26, 2026
106 of 114 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: test_concurrent_short_writers_serialize times out on Windows

2 participants