Repository navigation
nightly: re-baseline differential from the scheduled run on main at 9a26f5b5 (#1302, #1304) - #1367
Merged
Merged
Conversation
…a26f5b (#1302, #1304) verdicts.json of scheduled run 40712357505 on main at 9a26f5b, byte for byte. Three cases move, and no others: - BIOMD0000000338|klu_vs_dense PASS -> EXCEPTION. Since #1302, the forced-dense jacobian="fd" leg fails at t = 0.72 (CV_CONV_FAILURE) on every Linux runner; the KLU leg completes (ajac_vs_fdjac, the same KLU run bit for bit, passes). The model is #1304's. - BIOMD0000000385|ajac_vs_fdjac and |klu_vs_dense TIMEOUT -> PASS. The old file is #1291's run, where the pool killed every row at the 900 s cap; with #1321's per-check deadlines and the 1500 s cap these two finish.
wshlavacek
added a commit
to Montana/bngsim
that referenced
this pull request
Oct 28, 2026
) verdicts.json of this PR's Path-agreement run 40721476951, on 18c677c merged with main at 9a26f5b. Every row main's checks give is the same as in main's scheduled night 40712357505 (lanl#1367 re-baselines main from it), plus this PR's 1,353 jacobian_values rows. Against the old file: - BIOMD0000000338|klu_vs_dense PASS -> EXCEPTION, from main (lanl#1302, lanl#1304); - BIOMD0000000385|ajac_vs_fdjac and |klu_vs_dense TIMEOUT -> PASS (lanl#1321); - MODEL1112050001|jacobian_values SKIP -> PASS (its base ran this time); - MODEL1006230049|jacobian_values TIMEOUT -> SKIP, no analytic Jacobian; - the seven models Model.jacobian() refuses stay SKIP (18c677c).
#1354 re-baselined BIOMD0000000338|klu_vs_dense to the EXCEPTION row of run 40712357505; this branch already carries that run's verdicts.json whole, so its file is kept.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-baselines
parity_checks/nightly/baselines/differential.json. Every Path-agreement run since 2026-10-27 19:00 UTC fails on one alert,BIOMD0000000338|klu_vs_densePASS → EXCEPTION. That includes main's own scheduled night, 40712357505, and the differential PRs #1315-#1323. The change comes from main, not from any of those PRs.The file is replaced byte for byte by the
verdicts.jsonof scheduled run 40712357505 on main at9a26f5b5. Re-extracting that run'sreport.jsonwithverdicts.py extractgives the same bytes.klu_vs_densejacobian="fd"leg fails on Linux (below)ajac_vs_fdjacklu_vs_denseverdicts.py diffof that night against this file gives no alerts.BIOMD0000000338: what changed it
It changed at a commit boundary, not with the runner. The case reads PASS on every run before 2026-10-27 18:31 UTC and EXCEPTION on every run from 19:00 onward. Runs on each side used AMD EPYC 7763 and 9V45 runners, and two Xeon runs after 19:00 fail too:
klu_vs_denseThe first failing run is #1302's. The first failing run was created at 19:00 UTC, when the only bngsim change new to main was #1302 (merged 17:31 UTC). #1321's harness change, a
timeout=on every run, is not it: on macOS the run is bitwise identical with and without a timeout, for KLU, dense and the default solver.The failing leg is the dense one.
klu_vs_denserunsforce_sparse_linear_solver=True, jacobian="fd", thenforce_dense_linear_solver=True, jacobian="fd".jacobian="fd"run is KLU, bit for bit the first leg. That run isajac_vs_fdjac, and it passes on the same night (1.0e-7). So the leg that raises at t = 0.72 (CV_CONV_FAILURE) is the dense one.cvode_dense_dq_past_jumps, plus the jump test.It does not reproduce on macOS arm64. At 9a26f5b the dense leg completes, and Fg at t = 2 is 0.734, which agrees with libRoadRunner. The KLU leg ends at 311: #1304's branch split. With
BNGSIM_JAC_DEBUG=1, neither leg's quotient has an entry the jump test mends.The model is #1304's. KLU and dense take different branches on it past a step-size collapse, so a bit-level change in either quotient can move a leg. The new failure is a loud one: an exception, not an answer. Recording it here stops it alerting every night, and #1304 carries the finding.
A baseline-only change: the nightly's sweeps are not re-run for it.