From 51bc7e8d643a68c6feb85714967b27c049cb7553 Mon Sep 17 00:00:00 2001 From: MauroFab Date: Tue, 8 Sep 2026 03:08:25 -0300 Subject: [PATCH] prover: run the lib test harness on the shipped allocator Every host-memory number in the campaign's record was taken from `cargo test -p lambda-vm-prover --lib` runs, and that binary had no `#[global_allocator]`, so it ran on glibc malloc while the only shipped binary, `bin/cli`, runs jemalloc. The two return freed memory to the kernel differently, and it showed: at the wrap's q=41 rung the harness read 98.6 GiB of max RSS at the tip against 86.7 GiB at the parent of #956, a difference that came entirely from 12 GiB of freed column buffers glibc kept resident in an arena, not from anything the prover held. The regression attributed to #956 was a measurement of the allocator. Install jemalloc as the lib test harness's global allocator, the same one `bin/cli/src/main.rs` installs, so a harness measurement is a production-allocator measurement. `tikv-jemallocator` was already a dev-dependency; `tests/calibration.rs` already installs it for its own crate. Integration tests remain separate crates. What the control runs established, with `MALLOC_MMAP_THRESHOLD_=1048576` standing in for the allocator change (box A, one run per cell). MEMORY: the saving is large, reproducible and controlled. From a high-retention start it is about 13 GiB at q=41 (98.6 to 85.5 GiB, tip) and about 6 GiB at q=20 (51.0 to 43.9 GiB); from a low-retention start it is about 1.2 GiB (86.7 to 85.5 GiB at the parent of #956, 45.1 to 43.9 GiB at q=20). The parent and the tip read 604 kB apart under the knob, which is the control. TIME: measured within plus or minus 3 percent with no consistent sign across the four cells, one run each, three of them inside what a single run resolves; no time effect is claimed in either direction, and establishing one needs paired repeats per shape. One observation, not a mechanism: under the knob the runs agree closely with each other (140.1 to 140.2 s at q=20, 275 to 278 s at q=41) while the runs without it scatter. --- prover/src/lib.rs | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/prover/src/lib.rs b/prover/src/lib.rs index 38b39ecab..8a9f12893 100644 --- a/prover/src/lib.rs +++ b/prover/src/lib.rs @@ -29,6 +29,17 @@ pub mod test_utils; #[cfg(test)] pub mod tests; +// The lib's test harness runs the allocator the shipped binary runs +// (`bin/cli/src/main.rs` installs the same one), so every host-memory number a +// `cargo test --lib` measurement produces is a production-allocator number. +// Under the platform allocator the same proves read up to 13 GiB higher at the +// wrap's q=41 rung: glibc kept freed arena chunks resident, and the run +// measured the allocator, not the prover. Integration tests are separate +// crates and install their own (`tests/calibration.rs` already does). +#[cfg(test)] +#[global_allocator] +static TEST_ALLOC: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc; + use std::fmt; use crypto::fiat_shamir::is_transcript::IsTranscript;