Repository navigation
Release 1.6.9: a faster test suite and a scratch folder per agent - #35
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 1.6.9: a faster test suite and a scratch folder per agent.
The suite had grown to 1,862 tests that
tests/run-tests.mjsran four files at a time, and tests inside one file run one after another, socleanup.test.mjsalone (384 s of tests) set a floor for the whole run. Now the five slowest files keep their tests intests/<name>.cases.mjsand are split across 15tests/<name>.shard<k>.test.mjsfiles bytests/shard.mjs; no test changed. The runner's default is the CPU count less 4, between 4 and 12, and stays 4 inside a parallel-lanes lane check (PL_PHASE=lane), where several lanes may run the suite at once.Measured on Windows (20 cores, Node 24, Python 3.15): 617 s and 692 s before at four at a time; after, 428 s at 8, 405 s at 12 and 418 s at 16 (one run each). Pointing the tests' temp folder at a folder the virus scanner skips measured 384 s against 405 s, inside the spread, so it is not part of this release.
Also: the scratch-folder rule the final reviewers got in 1.6.7 (their own
mktemp -dfolder, never the session's scratchpad) is now in the rules every agent gets. On r168 seven implementers wrotered.txtandgreen.txtinto the shared scratchpad and two overwrote each other's file.Full suite on Windows after the split: 1868 tests, 1852 pass, 16 skipped, 0 fail; the prompt, engine, adjudicator and build tests after the scratch rule. The full local suite was not rerun at the release head because another parallel-lanes run is using this machine; CI runs it.