Skip to content
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **benchmarks**: new **`benchmarks/bench_crash_recovery.cpp`** (`BM_WriterDurabilityTier`) quantifies the writer hot-path cost of the recovery defaults across the three durability tiers (journal off / journal + flush-only / journal + fsync). Reference numbers on an NVMe dev machine, 200 small frames per iteration: **~19 us/frame** with no journal, **~30 us/frame** flush-only (process-crash safe -- effectively free), **~289 us/frame** with the default per-operation fsync (power-loss safe; ~1.7% of a 60 fps frame budget on NVMe -- on spinning disks prefer `durable_writes = false`). The byte-identity tests now compare from the end of the header on (the header embeds a second-granularity recording timestamp, so two separately recorded sessions may legitimately differ there; repair never rewrites the header)
- **docs**: **`docs/SDK_API.md`** "Writing Replays" gains a **"Crash recovery"** section -- the `.recovery` sidecar model, the user-driven `ReplayNeedsRecovery` / `RepairReplayFile` / `RecoveryJournalPath` flow with `RepairResult` semantics, the exact-times guarantee, the refusal cases, and the sink durability knob table (`durable_writes` / `enable_recovery_journal` / `journal_compact_threshold_bytes`). **`README.md`** feature list gains a "Crash-safe recording" bullet. `DurableFile` hardening: `Seek`/`SeekEnd` failures now latch `Good()` false (a write after an unnoticed seek failure would land at the wrong offset), and the dead `Truncate` primitive left over from the pre-append-only journal design was removed

- **reader/api**: side-effect-free frame peek -- **`IVtxReaderFacade::GetResidentFrame(frame_index)`** (`ReplayReader::GetResidentFramePtr`) returns the frame ONLY if its chunk is already resident, **without** moving the cache window, triggering loads, or cancelling in-flight ones. Intended for incidental reads (e.g. drawing a stale frame while the target frame streams in); `GetFrame` remains the sole window-driving read. The inspector's stale-frame fallback now uses it (see Fixed)
- **reader/api**: new **`ReplayReaderEvents::OnChunkLoadCancelled(int32_t)`** -- fired when a load that already emitted `OnChunkLoadStarted` ends **without** the chunk becoming resident (cancelled by a window shift, or the worker failed). `ReaderChunkState` handles it (clears the chunk from the loading set without marking it loaded) and `OpenReplayFile` wires it automatically, so the started/finished pair always balances

### Changed

- **reader/perf**: chunk teardown moved off the caller thread. Evicting a chunk destroys up to ~1000 frames x hundreds of entities (measured hundreds of ms to >1s for real captures); `UpdateCacheWindow` now only unlinks evicted chunks and hands the owned data to a detached background task, so cross-range jumps no longer freeze the calling (UI) thread. The reader destructor does the same with the resident cache (closing a large replay returns at once) -- it still waits for in-flight load workers (they touch the reader), but not for the frees (chunk data is self-contained). Cancelled prefetch futures are parked and reaped via `wait_for(0)` instead of being destructed inline (a `std::async` future blocks in its destructor until the worker unwinds)
- **sdk-wide**: error/warning handling unified onto the diagnostics model -- there is now a single `Severity` enum, a single `VtxErrorCode` vocabulary, and a single diagnostic/result type family across the SDK:
- The schema validator's `SchemaIssueSeverity` is now an alias of `Severity`, and `SchemaValidationResult::ToReport()` is the single canonical bridge from `SchemaIssue` to `VtxDiagnostic` (used by `ValidateSchema`).
- The writer's frame-rejection reason (previously a writer-private `FrameRejectReason` enum) was removed; `RecordResult` now carries a `VtxError`.
Expand All @@ -61,6 +65,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- **reader/async**: cancelled or failed async chunk loads corrupted the chunk-state tracking. `AsyncLoadTask` fired `OnChunkLoadFinished` for ANY surviving worker -- including cancelled ones whose data was discarded -- so `ReaderChunkState` marked chunks "loaded" that were never resident; eviction (which iterates the cache) never fired for them and the loaded set grew without bound. A *failed* (non-cancelled) load also inserted an **empty** `CachedChunk` into the cache, permanently blocking retries for that chunk (reads returned null forever). Now a chunk is inserted only when the load actually succeeded, and non-resident outcomes emit the new `OnChunkLoadCancelled`. Regression test: `AsyncRandomJumpsDoNotLeakLoadedSet`
- **reader/async**: the async view path could wedge on "loading" forever. `trigger(current_idx)` ran subject to the `max_concurrent_loads` cap and AFTER the range-equality short-circuit -- a cross-range jump with the slots saturated by not-yet-reaped cancelled loads skipped the viewed chunk, committed the new range, and every subsequent (unchanged-range) call returned early, so the chunk was never queued. The viewed chunk now triggers first, uncapped (`priority=true`), and before the short-circuit, making the path self-healing. Regression test: `AsyncViewPathResolvesAfterCrossRangeJump`
- **tools/inspector**: scrubbing could ping-pong the cache window and never finish loading. While the target frame streamed in, the stale-frame fallback called `GetFrame(last_drawn)` -- a window-driving read that moved the window back to the stale chunk and cancelled the target's load; the next tick reversed it, indefinitely. The fallback now uses the side-effect-free `GetResidentFrame`. Regression tests: `ResidentFramePeekDoesNotCancelTargetLoad`, `ResidentFramePeekNeverTriggersLoad`
- **common/hash**: `Helpers::CalculateContainerHash` used a single shared `thread_local` XXH3 state while **recursing** into nested structs / struct-arrays / maps -- each recursive `reset` wiped the parent's in-progress accumulation, so once a container had any nested/map field its scalars / strings / vectors no longer affected the hash. Now each call uses its own state. Latent until the arena gained nested/map fields; it made distinct entities collide (e.g. two players sharing a `content_hash`), which in turn made the content-hash-based diff short-circuit treat changed frames as identical. Regression tests added in `tests/common/test_content_hash_edges.cpp` (`NestedStruct/Map/StructArrayDoesNotMaskPreRecursionFields`)
- **tools/cli**: the interactive `diff <a> <b>` command always reported "frames identical" (0 ops) -- `CliSession::DiffFrames` computed the patch but **discarded it** (returned `{}`) and passed the possibly-evicted `span_a` instead of the `bytes_a` copy it had made. Now returns the real patch computed over the copy (compounded by the `content_hash` bug above; both fixed)
- **common/loaders**: `GenericFlatBufferLoader::LoadArray` could not handle `vector<string>` -- the pointer-element branch assumed table structs and tried to `Load` a `flatbuffers::String` (no `FlatBufferBinding`, a hard compile error if instantiated). It now detects the string element (via `s->str()`) and fills the scalar string array, removing the manual workaround in the arena FlatBuffers binding
Expand Down
8 changes: 8 additions & 0 deletions docs/SDK_API.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,12 @@ int32_t total = reader->GetTotalFrames();
// Synchronous read (blocks until chunk is loaded)
const VTX::Frame* frame = reader->GetFrameSync(42);

// Side-effect-free peek: returns the frame ONLY if its chunk is already
// resident. Does NOT move the cache window, trigger loads, or cancel
// in-flight ones — use for incidental reads (e.g. drawing a stale frame
// while another frame streams in). GetFrame is the window-driving read.
const VTX::Frame* resident = reader->GetResidentFrame(42); // nullptr if not resident

// Copy-based read
VTX::Frame frame_copy;
bool ok = reader->GetFrame(42, frame_copy);
Expand Down Expand Up @@ -109,6 +115,8 @@ VTX::ReaderChunkSnapshot snapshot = ctx.chunk_state->GetSnapshot();
// snapshot.loading_chunks — chunk indices being loaded asynchronously
```

Custom consumers registering their own `ReplayReaderEvents` should handle all four chunk signals: `OnChunkLoadStarted`, then exactly one of `OnChunkLoadFinished` (chunk became resident) or `OnChunkLoadCancelled` (load was cancelled by a window shift, or failed — the chunk is NOT resident), and later `OnChunkEvicted` for resident chunks leaving the cache. Treating `Started` as "will finish" leaks the loading set: cancelled loads never emit `Finished`.

### Cache Window Control

```cpp
Expand Down
Loading
Loading