Repository navigation
ci: shard full gap and compile-smoke work within job budgets #11750
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
896f390
ci: shard full gap and compile-smoke work within job budgets
proggeramlug dfdc852
Document full compile sharding in #11750
proggeramlug 9a973ea
Merge current main into full-suite CI sharding
proggeramlug 7e0a9c9
Merge landed provider preflight into full-suite CI sharding
proggeramlug File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| Distribute full-tier compile work across bounded jobs after current runs hit | ||
| their timeout limits. The auto-optimize gap suite now uses twelve shards. | ||
| Compile-smoke assigns the complete top-level TypeScript inventory exactly once | ||
| across four round-robin shards, with at most two smoke shards running at once. | ||
| The existing compiler invocation, platform exclusions, retry policy and failure | ||
| markers remain intact; every shard still contributes to the aggregate result. | ||
| Invalid or empty selections fail closed, and smoke error artifacts are keyed | ||
| by shard. Local selection checks cover all 2,067 current files; actual full-tier | ||
| CI must establish the resulting wall-time margin. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| #!/usr/bin/env python3 | ||
| """Select a stable slice of the complete top-level TypeScript smoke inventory.""" | ||
| import argparse | ||
| from pathlib import Path | ||
| import sys | ||
| import tempfile | ||
| import unittest | ||
|
|
||
|
|
||
| def select(root, shard): | ||
| try: | ||
| index, total = map(int, shard.split('/')) | ||
| except ValueError as error: | ||
| raise ValueError('shard must be INDEX/TOTAL') from error | ||
| if not 1 <= index <= total: | ||
| raise ValueError('shard must satisfy 1 <= INDEX <= TOTAL') | ||
| if not root.is_dir(): | ||
| raise ValueError('test directory does not exist') | ||
| files = sorted(path for path in root.glob('*.ts') if path.is_file()) | ||
| if not files or len(files) < total: | ||
| raise ValueError('every shard must contain at least one test') | ||
| return files[index - 1::total] | ||
|
|
||
|
|
||
| class CoverageTests(unittest.TestCase): | ||
| def test_all_files_run_once_with_balanced_nonempty_shards(self): | ||
| with tempfile.TemporaryDirectory() as directory: | ||
| root = Path(directory) | ||
| for name in ['a.ts', 'b space.ts', 'c\nnewline.ts', 'd.ts', 'e.ts']: | ||
| (root / name).touch() | ||
| (root / 'ignored.js').touch() | ||
| (root / 'directory.ts').mkdir() | ||
| (root / 'nested').mkdir() | ||
| (root / 'nested/fixture.ts').touch() | ||
| expected = {p for p in root.glob('*.ts') if p.is_file()} | ||
| slices = [select(root, f'{i}/4') for i in range(1, 5)] | ||
| assigned = [p for part in slices for p in part] | ||
| self.assertEqual(set(assigned), expected) | ||
| self.assertEqual(len(assigned), len(expected)) | ||
| self.assertLessEqual(max(map(len, slices)) - min(map(len, slices)), 1) | ||
| self.assertTrue(all(slices)) | ||
| self.assertEqual(select(root, '1/4'), slices[0]) | ||
|
|
||
| def test_invalid_or_empty_scope_cannot_succeed(self): | ||
| with tempfile.TemporaryDirectory() as directory: | ||
| root = Path(directory) | ||
| (root / 'a.ts').touch() | ||
| for shard in ['0/4', '5/4', '1/0', '-1/4', '1/2', 'bad', '1/2/3']: | ||
| with self.subTest(shard=shard), self.assertRaises(ValueError): | ||
| select(root, shard) | ||
| with self.assertRaises(ValueError): | ||
| select(root / 'missing', '1/1') | ||
|
|
||
|
|
||
| def main(): | ||
| parser = argparse.ArgumentParser(description=__doc__) | ||
| parser.add_argument('--root', type=Path, default=Path('test-files')) | ||
| parser.add_argument('--shard') | ||
| parser.add_argument('--self-test', action='store_true') | ||
| args = parser.parse_args() | ||
| if args.self_test: | ||
| result = unittest.TextTestRunner().run(unittest.defaultTestLoader.loadTestsFromTestCase(CoverageTests)) | ||
| return 0 if result.wasSuccessful() else 1 | ||
| if not args.shard: | ||
| parser.error('--shard is required') | ||
| try: | ||
| paths = select(args.root, args.shard) | ||
| except ValueError as error: | ||
| parser.error(str(error)) | ||
| # Compute and validate the entire selection before emitting any scope. | ||
| sys.stdout.buffer.write(b'\0'.join(str(path).encode() for path in paths) + b'\0') | ||
| return 0 | ||
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| raise SystemExit(main()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Give each GC evidence artifact a shard-specific name.
Each new matrix cell runs
run_memory_stability_tests.shwithPERRY_GC_EVIDENCE_DIRset. The script creates a log in that directory. The unchanged upload step names every artifactgc-evidence-Linux, so uploads after the first fail with an artifact-name conflict. Those failures make the compile-smoke matrix andfull-suite-gatefail even when the tests pass. Add${{ matrix.shard }}to the GC evidence artifact name, as the error-log upload already does. (raw.githubusercontent.com)🤖 Prompt for AI Agents