avg_interval: regrid-created levels index time_avg out of bounds and skip Average_Type

[WeiqunZhang]

Severity: high/medium/low · Category: correctness, memory-ub · Fix order: 6 of 21 — fix this 6th.

_{Filenames are numbered in reverse fix order: 001 = fix last, 021 = fix first. This file is 016.}

Locations: Source/NS_derive.cpp:27, Source/NavierStokesBase.cpp:2633, Source/NavierStokesBase.cpp:885, Source/NavierStokesBase.cpp:1796

Based on commit 9bf664bf (line numbers refer to that tree).

All 13 findings are one missing piece: the ns.avg_interval feature assumes a hierarchy fixed at initialization, so nothing teaches it about a level created later (there is no pre-port Fortran ancestor to compare against — NS_average.cpp was written directly in C++). The three sub-fixes below — vector sizing, Average_Type fill, per-level checkpoint — are separate edits that should land together, since each alone leaves another's symptom reachable.

The defect

Source/NavierStokesBase.cpp:2633 — Static vectors time_avg, time_avg_fluct and dt_avg are sized only in post_init/post_restart to the then-current finest_level+1 and never resized on regrid, so post_timestep writes out of bounds through the Real& arguments when a deeper level appears later. _{Reported independently at this same line by F001, F025, F027, F034, F040; each reviewer's own wording and evidence is under Verification evidence below.}

Source/NavierStokesBase.cpp:885 — checkPoint writes the single shared file /TimeAverage once per level with std::ofstream::trunc, so only the finest level's time_avg/time_avg_fluct survive; post_restart then loads that one pair into every level, and dt_avg is never checkpointed (reset to 0 at line 2519). _{Reported independently at this same line by F066; each reviewer's own wording and evidence is under Verification evidence below.}

Source/NavierStokesBase.cpp:1796 — NavierStokesBase::init() (the overload used when a brand-new AMR level appears) never initializes Average_Type, unlike init(AmrLevel& old) which FillPatches it and initData() which zeroes it, so the new level's average state is read before it is ever written. _{Reported independently at this same line by F018, F026, F039, F053; each reviewer's own wording and evidence is under Verification evidence below.}

Why it matters

F013: ns.avg_interval>0, run starts (or restarts) with finest_level < max_level; error tagging later creates a new finer level. That level's post_timestep calls time_average(time_avg[level],...) with level >= time_avg.size(): out-of-bounds heap write (a_dt_avg accumulation) each step, plus OOB read in der_vel_avg (NS_derive.cpp:27) — heap corruption/UB in release builds.

F054: Multi-level run with ns.avg_interval>0: level 0's time_avg differs from finer levels (it alone gets the initial sample in NavierStokes::post_init:1296), but after restart every level gets the finest level's value, mis-normalizing velocity_average (NS_derive divides by time_avg[level]). Even single-level, a checkpoint between averaging steps loses accumulated dt_avg, so restarted averages differ from an uninterrupted run.

F017: ns.avg_interval>0 with amr.max_level>=1 and a level that first appears at a later regrid (tagging criterion met mid-run). Amr::regrid calls init() for that level; get_new_data(Average_Type) is left uninitialized. advance_setup then allocOldData()/setVal(0)/swapTimeLevels, making the uninitialized buffer the old data, and NS_average.cpp:45 computes S_avg = S_avg_old + dt*S_state, propagating garbage (or snan in debug builds) into velocity_average/x_vel_rms in every plotfile and checkpoint from then on.

How to reach it

• Tutorials/HotSpot/inputs.2d.average_hotspot (ns.avg_interval=1, amr.check_int=1, amr.regrid_int=2); restart from any checkpoint with amr.max_level=2. Next regrid creates level 2; its post_timestep hits time_avg[2] on size-2 vectors.
• Tutorials/HotSpot/inputs.2d.average_hotspot (ns.avg_interval=1, amr.max_level=1, amr.regrid_int=2) with amr.hi_temp.start_time/amr.gradT.start_time>0 (NS_error.cpp:38 supports it): level 1 first appears mid-run. Also restart with raised max_level.

Note from the audit

F054: Apply together with F013 (§A, Source/NavierStokesBase.cpp:2633), or this fix introduces F013's bug at level 0. The diff below writes time_avg[lev], time_avg_fluct[lev] and dt_avg[lev] for lev = 0 .. parent->finestLevel(), but F013 establishes that those static vectors are sized only in post_init/post_restart and are shorter than finestLevel()+1 once a regrid has created a level. Applied alone, this turns every checkpoint into an out-of-bounds read of the missing levels. Either take F013's post_regrid resize as well, or guard the loop with lev < static_cast<int>(time_avg.size()).

F017: The Average_Type BC dependency named in the fix note below is reported in this same report as F038 (Source/NS_setup.cpp:403, §B; see also F016 in §F). That matters here because this fix introduces the operation those findings are about — a FillCoarsePatch of Average_Type whose coarse-patch out-of-domain ghosts are int_dir/quiet_NaN. Apply F038's NS_BC.H diff together with this one, or every regrid-created level will traverse that path.

Suggested fix

Make the three statics level-lifecycle-aware with one grow-only helper — resize(finestLevel()+1, 0.) preserves existing accumulations — and call it from init() as well as post_regrid: AMReX runs post_regrid on every level only after the new level's init() (AMReX_Amr.cpp:2759), so resizing only there is too late for anything init() seeds. In init(), fill Average_Type like the twin init(AmrLevel&) (lines 1734-1737); the descriptor has 0 ghosts and null_bf, so nothing else ever writes it. That FillCoarsePatch needs F038's average_bc fix (NS_BC.H:57 is all int_dir), or the coarse patch's boundary ghosts stay quiet_NaN and cell_cons_interp spreads NaN inward. Judgement call: the interpolated data is ∫u dt over the coarse window, so either seed time_avg[level] from level-1 or zero MultiFab and scalars together — interpolating with zero scalars leaves der_vel_avg dividing by 1. Apply F054's per-level checkpoint loop only with F013's resize, and decide whether old single-value TimeAverage files must still restart.

For Source/NavierStokesBase.cpp:2633 (F013):

--- a/Source/NavierStokesBase.cpp
+++ b/Source/NavierStokesBase.cpp
@@ -2445,6 +2445,21 @@
 NavierStokesBase::post_regrid (int lbase,
                                int /*new_finest*/)
 {
+    //
+    // A regrid may have created levels that did not exist when the on-the-fly
+    // averaging data were sized in post_init/post_restart.
+    //
+    if (avg_interval > 0)
+    {
+        const int   finest_level = parent->finestLevel();
+        if (NavierStokesBase::time_avg.size() < finest_level+1)
+        {
+            NavierStokesBase::time_avg.resize(finest_level+1);
+            NavierStokesBase::time_avg_fluct.resize(finest_level+1);
+            NavierStokesBase::dt_avg.resize(finest_level+1);
+        }
+    }
+
 #ifdef AMREX_PARTICLES
     if (NSPC && level == lbase)
     {

_{Root fix lands outside the target file: amrex Amr::regrid calls post_regrid on levels 0..new_finest before any post_timestep or derive, so growing the three static vectors there makes both NS_derive.cpp:27 and post_timestep in bounds (resize zero-fills). Maintainer must decide whether a brand-new level should instead inherit the coarser time_avg, since init() never fills Average_Type. (sketch written against F001)}

For Source/NavierStokesBase.cpp:885 (F054):

--- a/Source/NavierStokesBase.cpp
+++ b/Source/NavierStokesBase.cpp
@@ -860,7 +860,7 @@
 {
     AmrLevel::checkPoint(dir, os, how, dump_old);
 
-    if (avg_interval > 0)
+    if (avg_interval > 0 && level == 0)
     {
         VisMF::IO_Buffer io_buffer(VisMF::IO_Buffer_Size);
 
@@ -882,8 +882,15 @@
             // write out title line
             TImeAverageFile << "Writing time_average to checkpoint\n";
 
-            TImeAverageFile << NavierStokesBase::time_avg[level] << "\n";
-            TImeAverageFile << NavierStokesBase::time_avg_fluct[level] << "\n";
+            //
+            // Write one (time_avg, time_avg_fluct, dt_avg) triple per level.
+            //
+            for (int lev = 0; lev <= parent->finestLevel(); lev++)
+            {
+                TImeAverageFile << NavierStokesBase::time_avg[lev] << "\n";
+                TImeAverageFile << NavierStokesBase::time_avg_fluct[lev] << "\n";
+                TImeAverageFile << NavierStokesBase::dt_avg[lev] << "\n";
+            }
         }
     }
 
@@ -2511,12 +2518,19 @@
       std::string fileCharPtrString(fileCharPtr.dataPtr());
       std::istringstream isp(fileCharPtrString, std::istringstream::in);
 
       // read in title line
       std::getline(isp, line);
 
-      isp >> NavierStokesBase::time_avg[level];
-      isp >> NavierStokesBase::time_avg_fluct[level];
-      NavierStokesBase::dt_avg[level]   = 0;
+      //
+      // The file holds one (time_avg, time_avg_fluct, dt_avg) triple per
+      // level; read forward to this level's triple.
+      //
+      for (int lev = 0; lev <= level; lev++)
+      {
+          isp >> NavierStokesBase::time_avg[level];
+          isp >> NavierStokesBase::time_avg_fluct[level];
+          isp >> NavierStokesBase::dt_avg[level];
+      }
 
     }
   }

_{Level 0 alone writes TimeAverage, with one (time_avg, time_avg_fluct, dt_avg) triple per level; post_restart reads forward to its level's triple and restores dt_avg instead of zeroing. Maintainer must bless the format change: old-format checkpoints restart with zeros beyond level 0's pair (failed extractions yield 0 since C++11). Per-level files are the alternative.}

For Source/NavierStokesBase.cpp:1796 (F017):

--- a/Source/NavierStokesBase.cpp
+++ b/Source/NavierStokesBase.cpp
@@ -1794,6 +1794,11 @@
     FillCoarsePatch(S_new,0,cur_time,State_Type,0,NUM_STATE);
     FillCoarsePatch(P_new,0,cur_pres_time,Press_Type,0,1);
     FillCoarsePatch(Gp_new,0,cur_pres_time,Gradp_Type,0,AMREX_SPACEDIM,Gp_new.nGrow());
+
+    if (avg_interval > 0){
+      MultiFab& Save_new = get_new_data(Average_Type);
+      FillCoarsePatch(Save_new,0,cur_time,Average_Type,0,AMREX_SPACEDIM*2);
+    }
     //
     // Get best coarse divU and dSdt data.
     //

_{Mirrors the correct twin init(AmrLevel& old), which FillPatches Average_Type: the brand-new-level init() now FillCoarsePatches all AMREX_SPACEDIM*2 components from the coarser level, so advance_setup/time_average never read an uninitialized buffer. Depends on Average_Type's registered BCs for physical-boundary fill (separate NS_setup.cpp finding, if reported).}

_{Diff(s) are against 9bf664bf, written from the current source and verified only with git apply --check — never compiled, never run, never applied to the tree. Treat them as precise intent, not tested patches.}

Verification evidence

F001 — confirmed (two independent verifier lenses)

Reported as: NavierStokesBase::time_avg (and time_avg_fluct, dt_avg) are sized only in post_init/post_restart, never on regrid; for a level created after initialization, der_vel_avg reads time_avg[level] out of bounds, and post_timestep (NavierStokesBase.cpp:2633) writes through out-of-bounds references dt_avg[level]/time_avg[level] into time_average.

Failure scenario: ns.avg_interval>0, amr.max_level=2, initial tagging creates only level 1; a later regrid adds level 2. Its first post_timestep calls time_average(time_avg[2],...) on size-2 vectors: heap read/write past the end (NS_average.cpp lines 23/59), corrupting memory; plotting velocity_average also reads time_avg[2] OOB here. Debug builds assert in amrex::Vector.

Lens 1 (refutation attempt): Only resize sites: NavierStokes.cpp:1289-1291 (post_init) and NavierStokesBase.cpp:2470-2472 (post_restart), both sized to finestLevel()+1 at that moment; post_regrid (NavierStokesBase.cpp:2445) never resizes. post_timestep:2633 calls time_average(time_avg[level],...) and NS_average.cpp:23 writes 'a_dt_avg = a_dt_avg + dt_level' through the OOB reference; NS_derive.cpp:27 reads 'NavierStokesBase::time_avg[level]' OOB. amrex::Vector::operator[] asserts in debug. git -S shows no fix landed.

Lens 2 (reachability/intent): post_regrid (NavierStokesBase.cpp:2445-2456) never resizes time_avg/time_avg_fluct/dt_avg; only post_init (NavierStokes.cpp:1289, level-0 only) and post_restart (NavierStokesBase.cpp:2470) do. post_timestep:2633 indexes time_avg[level] unconditionally when avg_interval>0, and NS_average.cpp:23 writes a_dt_avg through the OOB reference every step. No assert/abort limits level growth; commit 7c97555 ("major bug fix for AMR") and Average_Type FillPatch in init(AmrLevel&) show multi-level use is intended, not excluded.

F013 — confirmed (two independent verifier lenses)

Lens 1 (refutation attempt): post_timestep line 2633: time_average(time_avg[level],...) unconditionally when avg_interval>0. Only resizes are NavierStokes.cpp:1289-1291 (post_init, level-0 only, finest_level+1 at t=0) and NavierStokesBase.cpp:2470-2472 (post_restart). post_regrid (2444-2456) touches nothing. A regrid-created deeper level indexes past the end and time_average writes a_dt_avg through the Real& (NS_average.cpp:23). OOB read also in NS_derive.cpp:27.

Lens 2 (reachability/intent): Resize only in post_init (NavierStokes.cpp:1289-91) and post_restart (NavierStokesBase.cpp:2470-72); post_regrid (2445-56) never resizes. post_timestep:2633 binds time_avg[level]/dt_avg[level] for any level; NS_average.cpp:23 writes a_dt_avg unconditionally each step. amrex::Vector::operator[] asserts only in debug (AMReX_Vector.H:35-47) — release builds get raw heap OOB. Commit 7c97555 ("major bug fix for AMR") shows multi-level use is intended, not excluded.

F025 — confirmed (one verifier lens)

Reported as: The static Vectors time_avg/time_avg_fluct/dt_avg are sized only once (post_init / post_restart) to parent->finestLevel()+1 and are never resized when a regrid later creates a finer level, so time_avg[level] is an out-of-bounds std::vector access on the new level.

Failure scenario: ns.avg_interval=1, amr.max_level=1, initial data tags nothing so bldFineLevels leaves finest_level=0 and post_init (NavierStokes.cpp:1289) resizes the vectors to size 1. A later regrid creates level 1; level 1's post_timestep/checkPoint/der_vel_avg then read-write time_avg[1] past the end of a size-1 vector -> heap corruption or garbage averaging time (assert in DEBUG builds).

Lens 1 (refutation attempt): checkPoint line 885: 'TImeAverageFile << NavierStokesBase::time_avg[level]' — static vectors sized only at post_init (NavierStokes.cpp:1289, level-0 only) and post_restart (2470). No resize on regrid; a level created later indexes past the end here, in post_timestep:2633 (written through), and NS_derive.cpp:27. amrex::Vector asserts in debug, UB in release.

F054 — confirmed (one verifier lens)

Lens 1 (refutation attempt): Lines 871-874: every level opens the same '

/TimeAverage' with std::ofstream::trunc; Amr::checkPoint iterates levels 0..finest so only the finest level's pair survives. post_restart 2508-2518 reads that single file into every level's time_avg[level]; dt_avg[level]=0 at 2519 (and 2495), never checkpointed. Level 0 genuinely differs (extra post_init flush, NavierStokes.cpp:1296), and mid-window dt_avg accumulation is lost, so restarted averages differ.

F017 — confirmed (one verifier lens)

Lens 1 (refutation attempt): Confirmed all three contrasts: init(AmrLevel&) FillPatches Average_Type (1734-1737), initData zeroes it (NavierStokes.cpp:351-352 'Save_new.setVal(0.)'), init() (1759-1806) does nothing. advance_setup's allocOldData/setVal(0)/swapTimeLevels sequence makes the uninitialized buffer the old data; NS_average.cpp:45 propagates it into every subsequent S_avg.

F018 — confirmed (one verifier lens)

Reported as: NavierStokesBase::init() (fill a totally new level) never initializes Average_Type, unlike its twin init(AmrLevel& old) which FillPatches it at lines 1734-1737; the new level's Average_Type data is read before ever being filled by time_average.

Failure scenario: ns.avg_interval>0, regrid creates a new deepest level mid-run. init() fills State/Press/Gradp/Divu only; Average_Type stays uninitialized. advance_setup swaps it into old data, then time_average computes S_avg = S_avg_old(garbage) + dt*u (NS_average.cpp:45), so velocity_average and its RMS on that level are permanently garbage/NaN (sqrt of negative garbage in der_vel_avg).

Lens 1 (refutation attempt): init() (1759-1806) FillCoarsePatches only State/Press/Gradp/Divu/Dsdt; no Average_Type, unlike init(AmrLevel&) which FillPatches it at 1734-1737. post_regrid does nothing. advance_setup (691-701) zeroes only the newly-alloc'd old data then swapTimeLevels makes the uninitialized new-data buffer the old data; NS_average.cpp:45 computes S_avg = S_avg_old(garbage) + dt*u and persists it (line 46).

F026 — confirmed (one verifier lens)

Reported as: NavierStokesBase::init() (brand-new level interpolated from coarser) fills State/Press/Gradp/Divu/Dsdt but never fills Average_Type, unlike its twin init(AmrLevel&) which does (lines 1734-1737); the new level's Average_Type data is therefore read before ever being written.

Failure scenario: ns.avg_interval>0 with max_level>0: a regrid creates a new fine level via init(). Average_Type new_data is uninitialized; advance_setup zeroes only old_data then swapTimeLevels makes the garbage buffer the old data, so time_average() computes S_avg = S_avg_old + dt*u from garbage. The velocity_average plotfile variable on that level is garbage/NaN for the rest of the run.

Lens 1 (refutation attempt): Verified: init() fills S_new/P_new/Gp_new/Divu/Dsdt via FillCoarsePatch (1794-1805) and never touches Average_Type; twin at 1734-1737 does. advance_setup 693-700: old_data zeroed only when freshly allocated, then swapTimeLevels — the garbage new-data becomes old_data; time_average reads S_avg_old (NS_average.cpp:39,45). Garbage persists for the run.

F039 — confirmed (one verifier lens)

Reported as: NavierStokesBase::init() (brand-new level) fills State/Press/Gradp/Divu/Dsdt but never initializes Average_Type, so a level created by regrid carries uninitialized Average_Type data when averaging is active.

Failure scenario: ns.avg_interval>0, run starts with fewer levels than amr.max_level, regrid later creates a new level: its Average_Type new-data is never set (post_regrid does nothing either); after swapTimeLevels, time_average (NS_average.cpp:45) reads S_avg_old garbage and accumulates it, so velocity_average/rms on that level (plotfile and checkpoint) is undefined garbage forever.

Lens 1 (refutation attempt): Same defect as F018: init() at 1759-1806 has no Average_Type fill; post_regrid (2444-2456) is particles-only. Average_Type new_data of a regrid-created level is the raw AmrLevel-constructor allocation (uninitialized/snan); time_average accumulates it into S_avg permanently, contaminating velocity_average in plotfiles and checkpoints.

F053 — confirmed (one verifier lens)

Reported as: init() (fills a totally new level from the coarser one) never initializes Average_Type, unlike its twin init(AmrLevel&) which FillPatches it (line 1734-1737), so a regrid-created level carries uninitialized on-the-fly-average data forever.

Failure scenario: ns.avg_interval>0, regrid creates a level that did not exist before. Its Average_Type new_data is never filled; the first advance_setup swaps it into old_data, and time_average then accumulates S_avg = garbage + dt*S. velocity_average/RMS in plotfiles and the checkpointed Average_Type on that level are garbage/NaN from then on.

Lens 1 (refutation attempt): Same confirmed defect: brand-new-level init() (1759-1806) omits Average_Type, twin FillPatch at 1734-1737 exists, no other fill (post_regrid empty). First advance swaps the uninitialized buffer into old data; NS_average.cpp:45-46 accumulates garbage into both S_avg and S_avg_old, so plotfile/checkpoint Average_Type on that level is garbage/NaN thereafter.

F027 — confirmed (one verifier lens)

Reported as: time_avg / time_avg_fluct / dt_avg are static Vectors sized to finestLevel()+1 only in post_init (NavierStokes.cpp:1289) and post_restart (line 2470); nothing resizes them when a later regrid raises finestLevel, so post_timestep indexes them out of bounds.

Failure scenario: ns.avg_interval>0, amr.max_level=2, but only level 0-1 exist at t=0 (tagging criterion not yet met). time_avg has size 2. Later a regrid creates level 2; post_timestep at level 2 evaluates time_avg[2]/time_avg_fluct[2]/dt_avg[2] -> heap out-of-bounds read and write (also checkPoint line 885 and NS_derive.cpp:27).

Lens 1 (refutation attempt): Grep confirms resize only at NavierStokes.cpp:1289-1291 and NavierStokesBase.cpp:2470-2472; post_timestep 2630-2634 indexes time_avg[level]/time_avg_fluct[level]/dt_avg[level] unconditionally for avg_interval>0. A regrid-created level >= vector size gives OOB read/write; checkPoint:885 and NS_derive.cpp:27 also index [level].

F034 — confirmed (one verifier lens)

Reported as: The static Vectors time_avg/time_avg_fluct/dt_avg are sized only in post_init (NavierStokes.cpp:1289) and post_restart (NavierStokesBase.cpp:2470) to the then-current finest_level+1, but post_timestep dereferences time_avg[level] on every level, so a level created later by regrid writes past the end of the vectors.

Failure scenario: ns.avg_interval>0 with amr.max_level>=1 and initial data that tags no cells (finest_level=0 at post_init, vectors sized 1); a later regrid creates level 1; level 1's post_timestep calls time_average(time_avg[1],...), which writes through Real& references to out-of-bounds heap memory (time_average assigns a_dt_avg/a_time_avg) -> heap corruption/UB; same after restarting with a larger max_level.

Lens 1 (refutation attempt): Confirmed: 'time_average(time_avg[level], time_avg_fluct[level], dt_avg[level], dt_level)' at 2633; NS_average.cpp:23 'a_dt_avg = a_dt_avg + dt_level' and :59 'a_time_avg = a_time_avg + a_dt_avg' write through the OOB Real& references. Sized only in post_init/post_restart; restart with larger max_level plus later regrid hits the same OOB.

F040 — confirmed (one verifier lens)

Reported as: time_avg/time_avg_fluct/dt_avg are sized only in post_init and post_restart (finest_level+1 at that moment); post_timestep calls time_average(time_avg[level],...) for a level created by a later regrid, an out-of-bounds std::vector access that is also written through.

Failure scenario: ns.avg_interval>0, initial finest_level < amr.max_level, regrid creates a new finest level: its first post_timestep evaluates time_avg[level] with level == time_avg.size(), and time_average writes through the reference (a_time_avg = a_time_avg + a_dt_avg) -> out-of-bounds heap read/write; debug builds assert, release builds silently corrupt memory. der_vel_avg (NS_derive.cpp:27) performs the same OOB read at plotfile time.

Lens 1 (refutation attempt): Same confirmed OOB: vectors sized to finest_level+1 at post_init/post_restart only; new finest level's first post_timestep evaluates time_avg[level] with level==size() and time_average writes through the reference. amrex::Vector::operator[] asserts in debug, silent heap corruption in release; der_vel_avg (NS_derive.cpp:27-36) performs the OOB read at plot time.

F066 — confirmed (one verifier lens)

Reported as: checkPoint writes per-level time-average bookkeeping (time_avg[level], time_avg_fluct[level]) into one level-independent file <chkdir>/TimeAverage opened with std::ofstream::trunc, so every level overwrites the previous one and only the finest level's values survive.

Failure scenario: ns.avg_interval>0 with amr.max_level>=1: Amr::checkPoint calls each level's checkPoint in order 0..finest, so the file ends up holding time_avg[finest]. post_restart (line 2508-2518) reads that same file for every level, so level 0 gets the fine level's accumulation time. Because level 0 gets one extra flush at post_init (NavierStokes.cpp:1296, levelSteps(0)==0), time_avg[0] is permanently larger than time_avg[lev>0], so after restart the derived velocity_average at level 0 is normalized by the wrong time and reports wrong values.

Lens 1 (refutation attempt): Confirmed: single '/TimeAverage' opened with trunc per level (871-874), last (finest) writer wins; post_restart 2508-2518 loads it into every level. Level 0 alone gets the post_init flush (NavierStokes.cpp:1296 with levelSteps(0)==0), so time_avg[0] = time_avg[l>0] + dt_init permanently; after restart level 0 is normalized by the finest level's value in der_vel_avg. Error magnitude is one initial dt — low severity is right.

---

_{Based on commit 9bf664bf, which is also the tree the audit verified against. From an automated audit of Source/, Tutorials/ and Util/. Audit finding ids: F001, F013, F025, F054, F017, F018, F026, F039, F053, F027, F034, F040, F066. Reviewer unit(s): Core-Driver+BaseHeader, Derive+Error+Utils, NSB-1 setup/dt/umac_grown, NSB-2 init/sync/post-step, NavierStokes-1, Setup+BCfill+OutFlow, theme:restart-regrid. Nothing here was compiled or run — the failure scenarios are code reasoning, so the reaching configuration above is the cheapest way to confirm or refute it.}