Skip to content

CI: refresh shard timing weights from main run 41148135309 (#5364 shard-6 timeout) - #5415

Merged
realmarcin merged 1 commit into
mainfrom
codex/ci-timing-refresh-41148135309-20261031
Oct 31, 2026
Merged

realmarcin merged 1 commit into
mainfrom
codex/ci-timing-refresh-41148135309-20261031

Conversation

@realmarcin

Copy link
Copy Markdown
Collaborator

Refreshes utils/ci_test_durations.json from the successful main run 41148135309 at 4c07980399d47d77b64c9f2bff7a50bfc5cc6ddf, with the unchanged utils/update_ci_shard_timings.py, as utils/ci_sharding.md documents. It changes scheduling only: no test, lane, deadline or gate changes.

Why

  • The shard-6 timeout. PR Receipts v5 (A18): instrument version 5 from the 'Show me the receipts' slides, inactive by default #5364 times out at 45 minutes on shard 6. The old snapshot has no weight for tests/test_report_admission.py, so it gets the median fallback of about 3.14 s.
    • What the two heaviest files measure on main's CI: report admission 2,697 case-seconds over 22 cases; tests/test_download/test_api_runner.py 2,988, recorded at 329.61.
    • What goes wrong: v5's ten new test files move report admission onto shard 6, next to the API runner.
  • Other unweighted files: 51 files had no weight; together they measure 5,839 s.
  • Main is near the limit too: main's own shards 2 and 3 took 35 of their 45 minutes.

Estimated case-seconds per shard (old assignment vs new, both valued with the refreshed weights)

These are scheduling estimates, not wall-time predictions.

Collection, shards Old max shard New per shard Report admission / API runner
main, 4 7,980 6,787 2/3 → 2/1
main, 6 6,513 4,525 5/6 → 2/1
main + v5, 4 10,267 6,794 4/3 → 2/1
main + v5, 6 7,595 4,529 4/6 → 2/1

Verification

  • The snapshot can be reproduced exactly. An independent re-run of the unchanged updater on the same four pytest.xml files gives byte-identical output.
  • The inputs match the run's artifacts. The XML sha256 values and the run's archive digests are recorded in notes/ci_timing_refresh_2026-10-31/timing-refresh.json.
  • Collection is unchanged: 20,805 cases. The four ordinary shards hold exactly 20,801 occurrences between them, with no loss or duplication.
  • Tests: test_measured_partition_is_order_independent_and_reduces_estimated_imbalance passes. The nested-xdist sharding tests run on Linux CI. Locally, an environment without xdist fails them, and the snapshot doesn't cause that.

Not measured

The real CI wall time. This PR's own CI run is the first measurement.

🤖 Generated with Claude Code

@realmarcin
realmarcin merged commit 94d4c93 into main Oct 31, 2026
20 checks passed
@realmarcin
realmarcin deleted the codex/ci-timing-refresh-41148135309-20261031 branch October 31, 2026 06:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant