Skip to content

ode: warn when the default atol is above species that matter (#1361) - #1379

Open
Montana wants to merge 1 commit into
lanl:mainfrom
Montana:fix/1361-atol-above-scale-warning
Open

Montana wants to merge 1 commit into
lanl:mainfrom
Montana:fix/1361-atol-above-scale-warning

Conversation

@Montana

@Montana Montana commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Closes #1361.

What was wrong

The default ODE tolerances are rtol=1e-8 and a scalar atol=1e-8: BNG2.pl's, chosen for species held as molecule counts. A species that stays below atol for a whole run has its error controlled by nothing but the tolerance. Models in molar units (SBML, BioModels) are full of such species, and nothing said so. On main c0877fa:

  • The issue's binding model (species near 1e-9 M), at defaults: max relative error 2.2 against a tight reference, and A = -8.7e-10 at t = 5000 for 4.1e-10.
  • tests/data/BIOMD0000000002.xml (L at 1e-5 M; complexes at 3e-10 to 7e-9 M): errors of 1e-2. Its largest species is above atol, so a test of the whole run against atol passes it. The species that matter are below it.

The fix: a warning, not a new default

Simulator.run() raises the new bngsim.AtolScaleWarning (a UserWarning, exported) for an ODE run when both hold:

  • The caller gave no atol: neither run(atol=...) nor set_tolerances(atol=...), and no vector or tracking atol is in force.
  • Some species stays below atol for the whole run while reaching at least 1e-6 of the run's largest species. That floor is what keeps a species stuck at rounding in a count model (1e-17 of the rest) quiet, while it catches BIOMD0000000002, whose complexes reach 7e-4 of its ligand.

The message names up to five such species and their peak, and names the remedy: atol="auto" (#196), or a scalar atol well below them. docs/user-guide/solvers.md gets a section on the default tolerances and models in small units.

Why not change the default? atol="auto" as the default is the issue's first suggestion, and arguably the better end state. But it changes every default ODE trajectory, BNG2.pl parity at defaults included, so it is a maintainer's call. This PR makes the hazard visible without moving any number; a follow-up can change the default.

Measured, for false positives

corpus runs warned
every .net in the repo (tests/data, benchmarks, parity_checks; counts, by BNG2.pl) 176 2: nanomolar_dimer.net (species at 1.5e-9) and jac_selfcheck_amount_scaled.net (at 1.7e-10). Both are small-unit models, which is what the warning is for
tests/data/BIOMD*.xml 12 1: BIOMD0000000002
the BioModels corpus (1,323 SBML files, t = 0–100) 1,260 (63 did not load or run) 35 (2.8%)

Are the 35 right? For each warned BioModel, the default run was compared with a reference at rtol=1e-12, atol=1e-20. The error is the largest over species, relative to each species' peak:

  • At the default: above 1e-3 in all 35, median 1 (100%). Three run away entirely (BIOMD0000000362, 0086 and 0085, at 1e+275 and more).
  • With atol="auto", the remedy the warning names: median 6.1e-7.
  • Two exceptions: BIOMD0000000609 (1) and BIOMD0000000438 (1.6e+5) stay far off with auto as well, so something beyond the tolerance is wrong in them. They are not this PR's.

So no warned model was a false alarm.

Tests (test_atol_scale_warning.py, 8)

test on main
test_a_molar_model_at_the_default_atol_is_warned: ends negative, one warning naming 'A()' and atol='auto' fail
test_a_mixed_scale_biomodel_is_warned: BIOMD0000000002, naming 'BL' fail
test_the_remedy_it_names_is_right_and_quiet: control. atol='auto' is within 1e-5 of the reference, with no warning pass
test_an_atol_the_caller_chose_is_not_second_guessed[run/set_tolerances]: controls pass
test_a_count_model_is_not_warned: control pass
test_a_species_at_the_noise_of_a_count_model_is_not_warned: control. A species below atol but at 1e-15 of the largest pass
test_ssa_is_not_warned: control. A concentration of 1e-9 in a compartment of 1e10, ten molecules: SSA has no atol fail (no AtolScaleWarning on main)

Mutations

mutant fails
V1: no 1e-6 relevance floor the count-model noise control
V2: a set_tolerances atol is warned as well [set_tolerances]
V3: SSA is checked too test_ssa_is_not_warned
V4: warn only when the whole run is below atol (the first design) the BIOMD0000000002 test

Measured

Full suite (macOS arm64, main c0877fa + this branch): 16,424 passed, 0 failed, 137 skipped. Six existing tests now raise the warning, and each is a real case: five run a model whose only species is tiny (7.5e-14 to 3e-9; literal-parsing and rounding tests), and one runs a molar BioModel whose Bfa1P4_Tem1GDP peaks at 3.2e-10 while its largest species reaches 1.7e-4. None of them asserts on warnings. pre-commit run --all-files (new files staged) and sphinx -W with docs/requirements-ci.txt are clean.

The default scalar atol=1e-8 is BNG2.pl's, for molecule counts. A
species that stays below it all run has no error control but the
tolerance: the issue's binding model ended at A = -8.7e-10 for 4.1e-10,
and BIOMD0000000002's complexes were off by 1e-2, with no word.
run() now raises bngsim.AtolScaleWarning, naming the species, when the
caller gave no atol and a species stays below it while reaching at
least 1e-6 of the run's largest species. The default is unchanged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ODE default atol=1e-8 silently returns wrong (even negative) trajectories for models in molar units

1 participant