Skip to content

mir: reserve 32 MB of address space per context for its code, not 128 (#1215) - #1327

Open
elyfmiller wants to merge 3 commits into
lanl:mainfrom
elyfmiller:fix/1215-mir-code-reserve
Open

elyfmiller wants to merge 3 commits into
lanl:mainfrom
elyfmiller:fix/1215-mir-code-reserve

Conversation

@elyfmiller

@elyfmiller elyfmiller commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1215.

Problem

Since #1208 moved the vendored MIR to upstream 439a90a2, each MIR context reserves 128 MB of address space for its generated code at MIR_init, on 64-bit hosts other than Windows (third_party/mir/mir.c, MIR_CODE_RESERVE_SIZE). That keeps the context's code within its own direct-branch reach (±128 MB on aarch64). It holds no memory, but a cap on the address space (ulimit -v, RLIMIT_AS) counts it. With about 60 live MIR-JIT'd models, a cap of 8 GB is full.

Measured on main (aedadd8), MIR build, macOS arm64, 200 live Simulator(..., codegen=True) of simple_decay.net under BNGSIM_CODEGEN_JIT=mir:

address space per model
main +25,606 MB 128.0 MB

Fix

The fix is a CMake cache option, BNGSIM_MIR_CODE_RESERVE_MB (default 32). It sets MIR_CODE_RESERVE_SIZE for bngsim_mir on 64-bit hosts other than Windows. A value that isn't a whole number is refused at configure time. 0 reserves nothing, and Windows keeps MIR's own 0. No vendored file is edited; the define overrides MIR's #ifndef default.

Why 32 MB. It holds the largest committed model's code in one piece. That matters for locality only, not correctness. bngsim generates code eagerly (MIR_link with MIR_set_gen_interface). Its calls go through a register on aarch64, and the thunk redirects check their reach on both targets, so code mapped on its own runs the same. Measured with vmmap as the resident size of each context's reservation, BNGSIM_CODEGEN_JIT=mir at the automatic optimization level:

model generated code
simple_decay.net 16 KB
fceri_fyn.net 7.4 MB
fceri_gamma.net (the largest .net in benchmarks/models/net/ode) 27.9 MB

Code that outgrows the reservation is mapped on its own, which is what MIR does once a reservation is full. Before #1208 every context did that.

Results

The same 200 live models, on this branch's MIR build with the option set three ways:

BNGSIM_MIR_CODE_RESERVE_MB address space per model RSS growth
128 (upstream's) +25,610 MB 128.1 MB +384 MB
32 (default) +6,410 MB 32.1 MB +384 MB
0 +14 MB 0.1 MB +382 MB
  • The RSS doesn't change with the reservation: only address space does.
  • Under an 8 GB cap that is about 250 live models at 32 MB, against about 60 at 128.
  • With 32 MB, fceri_gamma's code (27.9 MB resident) and fceri_fyn's (7.4 MB) each sit in their own 32 MB reservation.

Tests

Two tests, both in files mir.yml already runs:

  • test_cmake_sizes_the_mir_code_reservation, in test_mir_vendoring.py: CMake sets the option and the define, and the vendored default it overrides is still upstream's 128 MB, so a refresh that moves it is seen.
  • test_a_live_mir_model_holds_far_less_than_upstreams_reservation, in test_mir_equivalence.py: 20 live MIR-JIT'd models in a subprocess, each holding at most 40 MB of address space. That catches a fall back to upstream's size, not the exact setting. It reads /proc/self/status on Linux and ps -o vsz elsewhere.
    • On builds without MIR, the module's own declared skip covers it. On Windows it skips with the declared "POSIX-specific".
    • It lives in test_mir_equivalence.py because test_mir_vendoring.py is part of run_tests.sh's walk-up probe (-k vendoring), which allows no skip. In its first place, it failed test_run_tests_sh on the three Python legs (5bfc698 moves it).
test 128 MB (as main) 32 MB
test_a_live_mir_model_holds_far_less_than_upstreams_reservation fails, assert 128.0 <= 40.0 passes
test_cmake_sizes_the_mir_code_reservation fails on main (no option) passes

Test audit

Checked against the test-writing rules Bill sent: each expected value's oracle, each tolerance's reason, and each test shown to fail against broken code. Commit 64936e8.

  • The runtime test's bound, at most 40 MB a model, now carries its reason. It is the 32 MB reserved plus about 0.1 MB of other mappings (measured at 0), with 8 MB to spare. Upstream's 128 fails it by a factor of three, and it fails on main (128.0).
  • The static test is an exact text oracle, and it fails on main.

Other checks:

  • mir.yml's test list, all 35 files, on this branch's MIR build under BNGSIM_CODEGEN_JIT=mir and BNGSIM_SKIP_AUDIT=strict: 1,318 passed, 1 xfailed.
  • test_mir_vendoring.py, test_skip_audit.py, test_ci_run_list_coverage.py and test_changelog_fragments.py under the strict audit: 116 passed on the MIR build. On a build without MIR, the first two give 44 passed and 1 skipped, and the skip is a declared one.
  • scripts/vendor_mir.py --check: OK (third_party/mir matches VENDOR.json).
  • ci/changelog.py check and ruff 0.14.0: clean.

The change is CMake inside if(BNGSIM_ENABLE_MIR) plus tests and docs, so a default build is untouched. perf_ab doesn't apply: it measures the default build.

Review

No second session was available, so a separate agent reviewed the diff, read-only:

  • One finding would have failed CI: both new skip reasons were undeclared under the strict skip audit. They are fixed: the skips now use the declared "without the MIR backend" and "POSIX-specific" wording.
  • The others are wording: the test's name now matches its bound, the docs say the 27.9 MB was measured on macOS arm64, and the README notes that the value is a CMake cache entry, so a build directory keeps the last one passed.
  • What it confirmed: for bngsim's eager generation, branch reach does not depend on the reservation.

Not tested here

  • Linux under ulimit -v. macOS can't set RLIMIT_AS, and Docker wasn't running. The Linux MIR leg of mir.yml runs the new test, which measures the same reservation.
  • Windows, which is unchanged.
  • The configure-time refusal of a BNGSIM_MIR_CODE_RESERVE_MB that isn't a whole number. It runs in CMake, and no test configures the project.
  • x86-64 code sizes. They were measured on arm64 only. A larger encoding only means a model's code is mapped in two pieces.

…lanl#1215)

Since lanl#1208, each MIR context reserves MIR_CODE_RESERVE_SIZE, upstream's
128 MB on 64-bit hosts other than Windows, so that its generated code's
direct branches reach each other. It holds no memory, but a cap on the
address space (ulimit -v, RLIMIT_AS) counts it: about 60 live MIR-JIT'd
models filled 8 GB.

The new CMake option BNGSIM_MIR_CODE_RESERVE_MB (default 32) sets it for
bngsim_mir. 32 MB holds the largest committed model's code in one piece
(fceri_gamma, 27.9 MB; fceri_fyn, 7.4 MB). Code that outgrows it is mapped
on its own, as MIR does once a reservation is full. 0 reserves nothing;
Windows keeps MIR's own 0. No vendored file changes.

Measured on macOS arm64, 200 live MIR Simulators: 128.1 MB of address
space each at 128, 32.1 at 32, 0.1 at 0, with RSS the same at all three.
…nl#1215)

test_mir_vendoring.py's modules are part of run_tests.sh's walk-up probe
(its -k "vendoring"), which allows no skip it cannot excuse. The live-model
test skipped on every build without MIR, so test_run_tests_sh failed on all
three Python legs. It now lives in test_mir_equivalence.py, which mir.yml
runs and which skips as a whole, with its declared reason, without MIR. The
static check of the CMake option stays in test_mir_vendoring.py.
40 MB a model is the default 32 plus the 0.1 MB of other mappings measured at
0, with 8 MB to spare, and upstream's 128 fails it by a factor of three.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MIR JIT: each context now reserves 128 MB of address space for code; under RLIMIT_AS (ulimit -v) that caps a process at ~60 live JIT'd models per 8 GB

1 participant