[wshlavacek]

Follow-up to #1215 (PR #1327), from that PR's review.

What was wrong on main

#1327 made each MIR context reserve 32 MB of address space instead of 128, so a cap on the address space (ulimit -v, RLIMIT_AS) holds more live MIR-compiled models. It left what happens at the cap: MIR ends the process.

• A failed code mapping comes back from get_last_code_holder as a NULL code address, and most of its ~20 callers write through it. On main, refusing the mapping for a generated function's code (with the reservation refused, as it is at the cap) is a SIGSEGV.
• A failed allocation goes to MIR's default error function, which calls exit(1), or, inside a growable array, to mir_varr_error, which calls exit(1) itself.
• c2mir's preprocessor allocates its macro, macro-call and #if records with plain malloc and writes through the result unchecked.

Nothing in bngsim could catch any of these.

The fix

MirJit (include/bngsim/mir_jit.hpp) hands each MIR context:

• an allocator (mir_alloc) that, while compile() has a call into MIR under way, reports a null instead of returning it;
• a code allocator (MirCodeMaps) that wraps MIR's default one and reports a failed mapping, except the reservation MIR_init2 asks for first, whose refusal MIR handles by mapping page by page;
• an error function (mir_error).

All three longjmp back to MirJit::mir_guarded (MIR.md asks only that an error function never return), and compile() throws a RuntimeError.

• ODE and steady-state runs pass it on; SSA falls back to interpreted propensities, as for any JIT failure.
• A refused mapping whose errno is not ENOMEM (a policy against executable memory, such as SELinux deny_execmem) gets its own message, without the cap advice.
• A failure outside a compile (none happens today) still does what MIR's default does.

A context MIR failed in is abandoned, not finished. The first version finished it, and an ASan sweep crashed MIR_finish in two places:

• a function new_func_arr had added to the module before filling in its fields;
• the inliner's item->data = (void *) 1 flags, which remove_item frees.

MIR does not keep its state whole through an error that returns, and patching each such spot is open-ended. So MirJit unmaps the context's code (nothing can reach it) and leaves its heap allocated. See "Left alone".

c2mir is finished as soon as it has made the module, instead of when the model is destroyed. Its parse of the source was most of what a live context held. Live heap per MIR-compiled simulator, main → this PR: simple_decay 1.94 → 0.96 MB, egfr_simple_seed 8.6 → 5.5 MB, fceri_fyn 894 → 421 MB (the cc backend's .so holds about 25 MB). The freeing moves from destruction to the end of the compile, so compile + run + destroy is unchanged (fceri_fyn 1,906 → 1,915 ms, median of 3), and the first run of a large model starts about 120 ms later.

Two local carries, recorded in VENDOR.json and scripts/MIR_VENDORING.md. Neither is reported upstream yet, and upstream master dc2c763f still has the old code.

• mir.c: a one-line _MIR_get_default_code_alloc, so MirJit wraps MIR's default code allocator rather than copying its platform code (MAP_JIT and the per-thread write protection on Apple Silicon, VirtualAlloc on Windows).
• c2mir/c2mir.c: NULL checks on the three plain mallocs, through c2mir's alloc_error, so c2mir_compile returns 0 and MirJit raises.

CMake now defines MIR_CODE_RESERVE_SIZE PUBLIC on bngsim_mir (0 on Windows and 32-bit), because the code allocator tells the reservation apart by its size.

Docs. third_party/mir/README.md.bngsim is now the one home of the reservation's rationale and numbers. The CMakeLists.txt comment, both test files' comments and the 1215.fixed.md fragment point at it, or keep only what they need. +mir-439a90a2.changed.md no longer says the tree is stock upstream, or that each context reserves 128 MB.

Evidence

A test-only binding, _bngsim_core._mir_jit_compile(source, fail_allocation, fail_mapping), fails one allocation or code mapping of a compile. The counting allocator it uses is a separate table, so a production compile pays nothing for it.

sweep (final code)	failure points	result
every allocation of simple_decay's compile, ASan build	35,028	all RuntimeError, 0 ASan errors, and the thread compiles again after each
every 11th allocation of egfr_simple_seed, ASan	14,840 of 163,228	all RuntimeError, 0 ASan errors
every code mapping, with the reservation refused: simple_decay, egfr_simple_seed, fceri_fyn	1, 2, 122	all RuntimeError
a full fceri_fyn compile, ASan	—	clean

• Address space after a failed compile (VSZ): +0.00 MB each. A reservation left mapped would add 32.

• Heap a failed compile keeps (mstats bytes in use): see "Left alone".

• Compile time, MIR, macOS arm64, median of 3, main → this PR, compile + run + destroy: simple_decay 4.4 → 4.5 ms, egfr_simple_seed 25.0 → 25.1 ms, fceri_fyn 1,906 → 1,915 ms.

• Full suite on a MIR build (macOS arm64, -n 6): 16,296 passed, 453 skipped, 1 failed: test_run_tests_sh.py::test_the_script_itself_runs_walk_up_dependent_tests_green, which fails in any /private/tmp worktree, main's too. (An earlier run also failed test_cache_cli.py::TestPrune::test_an_entry_is_never_selected_twice once under load. That test reads file access times, touches nothing here, and passed 3 of 3 alone and under -n 6.)

• mir.yml's test list, run with BNGSIM_CODEGEN_JIT=mir and BNGSIM_SKIP_AUDIT=strict: 1,667 passed, 2 skipped (both declared).

• Mutations. Each of these fails exactly one of the three runtime tests:

  • finishing the failed context instead of abandoning it;
  • handing MIR the failed mapping;
  • keeping the mappings.

  Dropping a c2mir check fails the new static test and the checksum anchor.

Review

Round 1 found no CRITICAL or HIGH defects, and confirmed real ODE, steady-state and SSA runs end to end: a mapping refusal raises, a model that was already live is unaffected, and a retry once mappings are allowed again gives bit-identical results. It found:

• MEDIUM: heap left behind by a failed compile. Measured up to 850 MB on fceri_fyn, and the sweep test's own child process piled up 3.7 GB. Changed: c2mir is finished early (above), which halves the code-mapping case to ~390 MB, and the sweep runs in children of at most 200 failures. The rest is under "Left alone".
• MEDIUM: Windows. 1 << 40 overflowed the binding's 32-bit long under MSVC. The hook arguments and counters are now long long.
• LOW: a counter that could overflow on Windows. The production path incremented a long counter that could wrap to -1 and inject a spurious failure. Counting now runs only under the test hook.
• LOW: overhead and wording.
  • mir_allocated read a thread-local before its null check; it now reads it after.
  • A refusal other than ENOMEM was reported as running out; it now has its own message.

Round 2 (the changes above) found no CRITICAL or HIGH defects.

• Early c2mir_finish holds up.

  • MIR copies every name, string and data item it takes from c2mir.
  • 215 dumped sources (ODE, sensitivity, steady-state and SSA, for every tests/data net plus fceri_fyn) compile clean under ASan, followed by symbol lookups, module output and moves.
  • Forced-MIR results are bit-identical to round 1's across 214 arrays and 8 seeded SSA runs.
  • The memory and time numbers above reproduce: 850 → 386 MB standalone, 889 → 423 MB per Simulator, total time unchanged.

• The new errno classification holds up. A DYLD mmap interposer gives:

  • the policy message for EACCES, EINVAL, EPERM and EAGAIN, across the binding, ODE and steady state, with SSA falling back to interpreted propensities;
  • the cap message for ENOMEM.

  macOS's hardened runtime without the allow-jit entitlement refuses MAP_JIT with EINVAL.

It found four LOW items, all fixed:

• An out-of-memory in the three c2mir mallocs surfaced as "c2mir failed to compile … no memory", without the cap message. The carry now reports through MIR's error function as an allocation error, as c2mir_calloc does. Failing each such malloc under ASan now reports out of memory: 173 of 173 on simple_decay, 486 of 486 on egfr_simple_seed.
• The README's "the cc backend's .so holds about 25 MB" was wrong. It's removed.
• The README's "compiled MIR, not the reservation, is most of what a live model holds" is true only of large models. It's now scoped.
• Nothing tested the non-ENOMEM branch. _mir_jit_compile takes a fail_mapping_errno, and a test asserts EACCES gets the policy message and ENOMEM the cap message. Making the branch always ENOMEM fails it.

Round 2 also saw a clean compile's allocation count move by one with heap layout under ASan. So the sweep probe now asserts only that the thread compiles again, not that it does so in the same number of allocations.

Round 3 (the round-2 fixes) was clean.

• Failing each c2mir malloc under ASan reported out of memory 173/173 and 486/486, and left the thread's failure target and locale as they were.
• The next compile on the same thread gave a bit-identical RHS.
• fail_mapping_errno is portable and resets after each call.
• Three mutations of the c2mir check each fail test_mir_vendoring.py.

Left alone

• The heap of an abandoned compile. mstats bytes in use per failed compile on fceri_fyn: about 390 MB for a refused code mapping (the cap case), and 225–483 MB for a failure while c2mir is still parsing. simple_decay keeps 0.2–1.7 MB and egfr_simple_seed 4–6.5 MB. Reclaiming it needs one of two things, and both are a design call:
  • an allocator that tracks every block: a header on each, a link and unlink on every allocation and free, and c2mir's six plain frees of context-allocated blocks routed through it. Every compile would pay for that in time and memory.
  • carries that keep MIR's state finishable through any error, which the sweep showed is open-ended.
• A failed mapping of the init-time wrapper-end thunk now raises. On main, MIR got NULL and carried on, since eager generation never calls that thunk. It fails only when not one page of address space is left, at which point the next mapping would fail anyway.
• Not verified, no Windows here: upstream's Windows default_mem_unmap calls VirtualFree(addr, len, MEM_RELEASE) with a non-zero length, which Windows rejects. If so, code pages are never freed on Windows, on main as on this branch.
• vm.max_map_count: _MIR_set_code ignores mprotect's result. A Linux process past its map count would write to a read-only page. That is a different limit, and a cap on the address space comes first in practice.

Tests

• python/tests/test_mir_equivalence.py (MIR builds; run by mir.yml):
  • each allocation (the first 300 and every 23rd after) and each code mapping of a compile, failed in turn in child processes;
  • a refusal other than ENOMEM reported as such;
  • address space returned after a failed compile;
  • on Linux only, a compile under a real RLIMIT_AS a little above the process's size, with 0 to 64 MB of headroom.
• python/tests/test_mir_vendoring.py:
  • every plain malloc of a struct in c2mir.c is checked;
  • MIR_CODE_RESERVE_SIZE is PUBLIC;
  • the carries are recorded.
• ci/test_floors.env: the skip ceiling rises from 225 to 230, by the five new tests that a build without MIR skips. ubuntu skips 218 on main and 223 with them.