Repository navigation
Conversation
steady_vs_run and newton_vs_run compare each steady_state() method with a run to t=1e6 from the same state, judged only where the solve says it converged and the run's last quarter moves it by at most 1e3 rtol. Where they differ, a state a run leaves, one off the initial state's conserved totals, or an unstable root is a DIFF; another state a run rests at is SKIP, both named, for the baseline to hold.
… no finding (lanl#1311) The corpus run's 2 DIFFs and 6 EXCEPTIONs were the reference's. Four models' runs to t=1e6 fail (CVODE) far past their own window: SKIP, no reference. BIOMD0000000518 starts on an unstable root (an eigenvalue of 5.6e-4): the solve stays on it, as the trajectory does in exact arithmetic, and the run leaves it on rounding alone.
…-vs-run steady_vs_run and newton_vs_run run after every check main runs, the runs to equilibrium bounded by what is left of the cap. A reference run the cap stops is this check's TIMEOUT, not a missing reference; steady_state() takes no timeout, and a kill still leaves the other rows.
This was referenced Oct 27, 2026
… fix/1311-steady-vs-run
… fix/1311-steady-vs-run # Conflicts: # python/tests/test_differential_shapes.py
…n EXCEPTION (lanl#1311) Since lanl#1337, Model.jacobian() refuses a difference quotient whose step decides an entry. check_steady asked it for the spectrum that tells an unstable root from another stable state, and _guard turned the refusal into EXCEPTION, a finding against steady_state(). Without a spectrum the stability is not judged: SKIP, said why, with both states named.
Resolve the differential harness: run lanl#1308's jacobian_values before the steady-state rows, and describe both in the docstring.
Collaborator
|
Landed through #1384 (merge commit 29bfc85), which merged this pull request at 4dbccf1. GitHub did not mark it merged because the branch moved afterwards to 3f03ec8, a merge of main (e425546) pushed after #1384 was cut. That merge adds nothing main lacks: its conflict resolution keeps both sides, in the order main now runs the checks (jacobian_values, then this pull request's), and differs from main only in one comment line. Closing as landed; #1311 was closed by #1384. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1311.
What it adds
steady_state()has its own routes: the CVODE march, the KINSOL polish, the scaled criterion's run test (#1001), the stability certificate (#78). None of them was compared with the plain integrator's answer. Two checks now do it, one per method, against the same run:steady_vs_runsteady_state()(integration)run()to t = 1e6 (steady_state()'s ownmax_time) from the same initial statenewton_vs_runsteady_state(method="newton")Both are passed the job's
rtol/atol. Write-only accumulators (Model.is_pure_sink()) are masked from the solve and left out of the comparison, since they settle nowhere (#74).Judged only where both have arrived, as the issue asks:
converged;Otherwise the row is SKIP, saying why. The bound is
max(1e3·rtol, 10·max(settle, drift)), wheredriftis the run's end at a tenfold tighter tolerance, the harness's usual rule.Where the two differ, a run from the solve's state, and its spectrum, say which case it is:
convergedthat is notconservation_laws), which the run keepseigvals(model.jacobian(x)), independent of the certificate) that the solve moved toA refusal by name (an event,
time(), a conservation law the RHS breaks: #710, #758) is UNSUPPORTED. A reference run that fails is SKIP ("no reference"): the parity suites own run failures.CHECKS = ( ... "grid", + "steady_vs_run", + "newton_vs_run", "sparsity_covers_jacobian", "backend_as_asked", )Over the corpus
Local run,
--checks steady_vs_run,newton_vs_run, fd_sens's 53.netmodels and 1,300 BioModels (2,706 rows):No finding. The first full run read 2 DIFF and 6 EXCEPTION. All eight were the reference's, and the second commit reclassifies them. I reran those 5 models on the final code; the rest of the corpus is unchanged by it:
steady_state()'s documented contract, not a bug.Two states, recorded as SKIP: BIOMD0000000072, BIOMD0000000546, MODEL1911120001 (Newton). On 72 Newton returns L = 0 while the run holds L at 6.02e17. L is not conserved (RL degrades), so L = 0 is the true limit, about 1e14/k away. The run settles only relative to its own size, and integration rightly reports unconverged. Hence the note's "or a tail slower than the horizon".
Cost: 1,663 s of CPU over the 1,300 BioModels (median 0.43 s per model, worst 155 s, MODEL1006230053), base run included. That is about 7 minutes on the nightly's 4 workers, against the job's 52 of 180.
Tests (
python/tests/test_differential_shapes.py, appended)New models, each called through
check_steadydirectly rather than added toSHAPES, so they don't pay for every other check's compiles:A <-> B(one conservation law);criterion="residual", #1001's defectX() at 0 against 1e-09A <-> Bwith the total off (2 for 1)The four event and time-reading shapes are declared refused by name in
REFUSED.Mutations, each caught:
convergedgate;methodignored;Suites:
parity_checks/testsand the shapes give 1,127 passed, 28 skipped, 1 xfailed. ruff 0.14.0 is clean. The branch merges cleanly with lanlmain.Still to do
The baseline (
parity_checks/nightly/baselines/differential.json) gains these rows from this PR's own nightly run. I'll record them from itsnightly-differentialartifact, as #1291 did.