[Montana]

Closes #1371.

What this records

These are the four klu_vs_dense rows of parity_checks/nightly/baselines/differential.json that #1295's fix (238d4d3) changed, taken from the first scheduled nightly on main after it: run 40899240805, at c0877fa.

case	baseline	tonight
MODEL1912090001|klu_vs_dense	DIFF, 3.11e-2 of the peak	PASS, 1.47e-4 (bound 2.0e-3)
MODEL2002030001|klu_vs_dense	EXCEPTION (CV_ERR_FAILURE)	PASS, 1.65e-6 (bound 1.0e-3)
MODEL7743528808|klu_vs_dense	DIFF, 2.09e-3 (bound 2.1e-3)	PASS, 0.0
BIOMD0000000827|klu_vs_dense	EXCEPTION (CV_CONV_FAILURE)	SKIP: no reference (the base moves 0.25 at a tenfold tighter tolerance)

The run's own diff against the committed baseline reads no regressions. These are its 3 improvements and its 1 change between failing states.

Only these four rows, not the whole file

The issue's recipe is to copy the run's verdicts.json over the baseline. That would be wrong here.

• The committed baseline is differential: the analytic Jacobian's values against differences of the right-hand side, not only its pattern #1308's PR run, 40721476951. It holds the jacobian_values check: 11 checks, 14,883 cases.
• Tonight's scheduled run predates that check merging: 10 checks, 13,530 cases.
• Copied whole, the file would lose 1,353 jacobian_values rows.
• Outside jacobian_values, exactly these four rows changed outcome.

So the file is #1308's run with these four rows replaced. It is written with verdicts.dump_verdicts, so the diff is one line per row. source.rows_from names the run, sha, CPU and cases they came from.

Checked

• verdicts.py diff, tonight's run against the patched baseline: 0 improved (3 against main's baseline). Every remaining difference is jacobian_values: 1,279 regressed and 74 dropped, which are rows tonight's run did not run.
• The recorded rows are not on a knife edge: the closest is 14× under its bound.
• ci/changelog.py required passes for a baseline-only change, and pre-commit is clean.

Not changed

• MODEL1006230034|klu_vs_dense is still EXCEPTION on the Linux runner, though klu: step each colored FD column on the error weights' scale (#1254) #1295 measured it at 3.2e-6 on macOS arm64. It stays as recorded.
• BIOMD0000000338|klu_vs_dense is Nightly parity: regressions detected #1355's flaky row, already in flaky.json. Unchanged.

A baseline-only PR does not run the nightly: pull_request.paths excludes parity_checks/nightly/baselines/**. The next scheduled nightly is the one that should read no change on these rows.

[wshlavacek]

Superseded by #1423, which records these four klu_vs_dense rows (and the rest of the baseline) from nightly run 41076909610 with the same outcomes.