Repository navigation
ci: a pull request runs only the nightly-parity legs its files feed (#1387) - #1391
Merged
Merged
Conversation
…anl#1387) Every PR that touched one of nightly-parity's PR paths ran all of its legs, ~1.5 h on ~10 runners. The plan job now lists the PR's changed files and parity_checks/nightly/pr_legs.py maps them to the legs whose runners import or read them, measured from the runners' imports (a test measures them again); each leg and the corpus job are gated on it. Shared code runs every leg, files outside the triggers run none, and scheduled or dispatched runs run every leg. The PR paths now include each leg's own directories.
…l#1387) On CI the virtualenv is .venv/ in the repo, so numpy's and libsbml's directories read as repo directories a runner imports.
…y imports too (lanl#1387) The directory rules ran every leg that shares a directory for a file only one of them reads. lanl#1202 changed bng_ode_run.py and ssa_screen.py, and the rules ran seven parity legs (bng-stoch's nf hour, rr-ode, AMICI, fd_sens and differential among them) for the two that read those files. lanl#1170, lanl#1173, lanl#887 and lanl#1006 each changed one BNG runner and ran both BNG legs; lanl#987 changed amici_sens_run.py and ran fd_sens and differential too. Documentation would run legs as well: a README alone in rr_parity ran all seven parity legs and the corpus. So would the golden gate's trajectories (golden-verify.yml's data), though no leg reads them. - Documentation feeds no leg: the workflow excludes "**/*.md" and pr_legs.py ignores it beside other changes. - bng_parity/golden/ feeds none. - Each runner, and the modules only it loads, feeds its own leg: rr_run.py, rr_parity's ssa_* (the screen, its attribution and its candidate list), bng_ode_run.py, bng_stoch_run.py with net_gillespie and net_roadrunner, amici_sens_run.py with amici_dispositions, fd_sens_run.py. A directory's other files keep its default. - benchmarks/suites/biomodels/biomodels_provenance.json is a trigger feeding the corpus legs. The corpus job hashes it into its key and the mirror fetch checks against it. lanl#1085 changed it and ran no leg. - The plan step lists a renamed file's old path too, so a file moved out of a leg's directory still runs that leg. Replayed over the 445 PRs merged since 2026-09-01, the rules now ask for 265 leg runs, against 273 before this commit and 192 under the old paths. Most of that rise is the coverage lanl#1387 asked for. Each run is a push, so the per-push saving is a multiple of this. The drift guard now checks each file a runner loads, not its directory. That retires its special case for benchmarks/, which let the SBML runner import any other module there unchecked. It also reads the loaded modules' import statements, so an import inside a function counts: ssa_screen.py loads ssa_attribution.py only that way, and the old probe never saw it. A missing package still skips; any other import failure now fails. New checks: the runners the test measures are the scripts the leg jobs run; every file in the corpus cache key feeds every corpus leg; and every path the workflow excludes feeds no leg.
…n runs none (lanl#1387) The README listed parity_checks/_core among the shared code that runs every leg, and the changelog said shared code still runs every leg. _core and _rr_common feed every parity leg, not sbml. Both now also say a runner's own files feed only its leg, and that documentation and the golden trajectories feed none.
…SSA jobs (lanl#1387) Two holes came in with main (lanl#1310, lanl#1312), which the branch merged. differential-nf, the NF engine agreement job, had no `needs: plan` and no `if:`. So every PR that triggered the workflow ran it (a 180-minute cap), whatever it changed. It is now a leg of its own, gated on its flag. Its runner is differential_run.py, which feeds both differential jobs. differential's SSA arm runs parity_checks/rr_parity/ssa_jobs.json. The `ssa_` rule sent that file to rr-ssa, which never reads it (the screen reads ssa_candidates.json), and not to differential. A PR regenerating the SSA jobs therefore ran rr-ssa for nothing and skipped the arm that reads them. It now feeds differential. The drift guard missed both. check_nf puts bng_parity on sys.path and only then imports bng_stoch_run, so the probe could not resolve the name and dropped it. An unresolved name is now looked up among the legs' code (parity_checks, benchmarks, harness) by file name. With that, the probe finds bng_stoch_run, _bng_common, net_gillespie and net_roadrunner. They now feed both differential jobs as well as their own. A runner whose import fails on a repo module now fails instead of skipping. The test that pinned RUNNERS to the workflow pooled every job's scripts, so an ungated job running a known runner passed. It now checks job by job: each leg's job runs that leg's runners, and no other job runs one. The pull_request.paths parser also reads entries that are unquoted or single-quoted. amici_ode_jobs.json and AMICI_PIN.json, which only amici-sens reads, feed only it.
…anl#1387) The README said a runner that comes to load another file fails the test until RULES names it. That holds for imports only. A jobs list or pin a job reads shows in no import (rr_ssa's ssa_jobs.json reached differential that way), so those are pinned by hand in the test's cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1387.
What was wrong
A pull request that touched any of
nightly-parity.yml's PR paths ran every leg, about 1.5 hours on ~10 runners, whatever it changed. No leg had anif:on what changed, so the #1305-#1313 differential fixes each ran the stochastic, RoadRunner, BNG, AMICI, fd_sens and SBML legs too. In the other direction,parity_checks/amici_parity/**was not a PR path, so a change to the AMICI runner alone ran no leg.The fix
The four items of the issue:
parity_checks/nightly/pr_legs.py: the PR's changed paths → one flag per leg, pluscorpus.RULESare measured, not guessed. I loaded each leg's runner as its job runs it and listed the repo directories its modules come from. The one data dependency an import trace cannot see is added by hand:differentialruns fd_sens's models.parity_checks/nightly/**parity_checks/_core/,parity_checks/rr_parity/(_rr_common, which every parity runner andreference_archive.pyimport)parity_checks/bng_parity/parity_checks/amici_parity/(fd_sens_runanddifferential_runimport its scheduler)parity_checks/fd_sens/parity_checks/differential/benchmarks/suites/sbml_test_suite/,benchmarks/_effort.py,harness/parity_checks/nightly/baselines/,flaky.jsonOnly the paths the workflow is triggered on are read (
TRIGGERS). A PR's changelog fragment,python/and its tests run no leg here, as a PR with only those runs none. A triggering path no rule names runs every leg: when unsure, run the leg.corpusis true where rr-ode, rr-ssa, amici-sens or differential runs.gh api .../pulls/N/files, withpull-requests: readon that job only) and hands them to the script; onscheduleandworkflow_dispatchit asks for--all;corpusgetsneeds: planandif: needs.plan.outputs.<leg> == 'true';reportandpublishare unchanged.verdicts.py summarizealready counts askippedjob as fine, andpublishruns only inrecordmode._core,rr_parity,bng_parity,amici_parityand the SBML leg's directories.The map on this week's PRs
The plan step's own command (
gh api ... | pr_legs.py), run on open PRs:amici_sens_run.py,fd_sens_rr_common.scheduleTests (
parity_checks/tests/test_pr_legs.py, 27)_core,rr_parity, the workflow, a nightly script, harness, a baseline alone and beside code, files outside the nightly alone and beside a differential change. Plus--allruns every leg.TRIGGERSequalspull_request.paths;corpusis an output ofplanand gated on it;Mutations, all caught:
amici_parityfeeds only amici-sensrr_parityrulefd_sensdoes not feed differentialTRIGGERSnot consultedChecked
actionlintandpre-commit run --all-filesare clean.parity_checks/tests: 967 passed and 2 failed. The same 2 fail on main without this change (test_bngsim_provenance,test_bng_reference_archive, an import path in my local environment).pull_requestbranch with every flag true, not a subset. The first differential-only PR after it lands is the run that shows the gating.