diff --git a/CHANGELOG.md b/CHANGELOG.md index 87d8ae8..a2d2c5d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,6 +37,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **common/schema**: the schema's top-level **`"buckets"` array is now parsed and is the source of truth for the frame bucket layout** -- `SchemaRegistry::GetBucketNames()` returns the declared names in order (`"buckets"[i]` names Frame bucket index `i`), and `PropertyAddressCache` carries them (`bucket_names`) so the reader can use them too. A new `Buckets` validation rule rejects a malformed key when present (non-array, non-string entries, empty or duplicate names); schemas without the key stay valid (legacy) - **writer/api**: **schema-driven bucket normalization** -- before finalization the writer rearranges every frame's buckets into the schema-declared layout: buckets are reordered to schema order, declared buckets missing from the frame are created empty, and a bucket the schema does not declare rejects the frame with the new **`VtxErrorCode::BucketUnresolved`** (observable via `TryRecordFrame`). Frames built positionally (no `bucket_map`) adopt the schema layout as long as they do not exceed the declared bucket count. Schemas without a `"buckets"` array skip the normalization entirely - **reader/api**: **bucket names restored on read** -- bucket names never hit the wire (both formats serialize buckets positionally), so deserialized frames used to come back with an empty `bucket_map`. The reader now stamps `bucket_map` from the embedded schema's `"buckets"` array when chunks are deserialized, so by-name lookups (`Frame::GetBucket(name)` const) and `bucket_map` iteration work on read frames. Replays written without a `"buckets"` array keep the old positional-only behavior +- **common/schema**: **array & map pre-sizing** -- extends the existing scalar pre-sizing (`type_max_indices`) so declared *array* and *map* fields are also materialized on load. `SchemaStruct` / `StructSchemaCache` gain `array_max_indices` (max Array-container index per `FieldType`) and `map_max_index` (count of Struct-valued map fields); `Helpers::ResizeContainerToMaxIndices` now pre-creates one empty subarray per declared array field in the matching `FlatArray` (via new `FlatArray::EnsureSubArrayCount`) and pre-sizes `map_properties`. A declared-but-unpopulated array/map field is now present-and-empty instead of absent, symmetric with scalars. Applies wherever scalar pre-sizing already ran (the four frame loaders + `schema_dynamic_loader`); the loader CRTP hook `GetTypeMaxIndices` became `GetStructSizing` (returns the `StructSchemaCache`) +- **reader/api**: **declared-empty arrays restored on read** -- an array with no data is not serialized (nothing to store), so deserialized entities used to come back missing those subarrays even for schema-declared array fields (maps already round-trip their slot count). The reader now re-creates the declared-but-empty subarrays from the embedded schema (`Helpers::EnsureDeclaredArrays`, grow-only, recursing into nested structs / struct-array elements / map values) so a read frame mirrors the array layout of an ingest-loaded frame. Arrays that carry data round-trip unchanged (their `offsets` already encode empty subarrays); scalars and maps are untouched ### Changed @@ -60,7 +62,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **writer/reader (Protobuff -> Protobuf rename)** (#28): the misspelled "Protobuff" was removed from the public API -- **breaking, no aliases kept**. `SerializationFormat::Protobuffs` -> **`SerializationFormat::Protobuf`**; `CreateProtobuffWriterFacade` -> **`CreateProtobufWriterFacade`**; `CreateProtobuffNetworkWriterFacade` -> **`CreateProtobufNetworkWriterFacade`**; reader `CreateProtobuffFacade` -> **`CreateProtobufFacade`** (internal `ProtobuffFacadeImpl` -> `ProtobufFacadeImpl`). Migrate call sites with a literal `Protobuff` -> `Protobuf` rename - **common/types**: `FlatArray::GetSubArray` & `FlatArray::GetMutableSubArray` did not have a check for StartIndex being greater or equal to EndIndex. In exceptional cases, like an empty array this can cause a crash or undefined behaviour in the best case scenario. - **writer/flatbuffers**: `FlatBuffersVtxPolicy::FromNative` hardcoded exactly two bucket slots -- `buckets[0]` was renamed to `"data"`, `buckets[1]` to `"bone_data"`, and **any bucket at index >= 2 was silently dropped** from the file. It now iterates all buckets generically (same shape as the Protobuf policy, shared `Serialization::SortBucketByTypeId` helper in `bucket_type_sort.h`), preserving every bucket and the frame's `bucket_map`. Bucket naming is schema-driven (see the `"buckets"` entries above), not serializer-driven -- **reader/schema**: the reader-side `PropertyAddressCache` built from the embedded schema (`PopulateCacheFromJsonString`) hand-copied the registry cache and **omitted `type_max_indices`**, so `ValidateEntity` on read frames treated every per-type max as 0 (any populated property would flag `FieldIndexOutOfRange` once frame validation ran). It now reuses `SchemaRegistry::GetPropertyCache()` verbatim. Also makes `ValidateReplay`'s per-frame pass effective: it iterates `frame.bucket_map`, which was always empty on read frames before bucket-name restoration +- **reader/schema**: the reader-side `PropertyAddressCache` built from the embedded schema (`PopulateCacheFromJsonString`) hand-copied the registry cache and **omitted `type_max_indices`**, so `ValidateEntity` on read frames treated every per-type max as 0 (any populated property would flag `FieldIndexOutOfRange` once frame validation ran). It now reuses `SchemaRegistry::GetPropertyCache()` verbatim. This also makes `ValidateReplay`'s per-frame pass effective **for replays whose schema declares a `"buckets"` array** -- it iterates `frame.bucket_map`, which was always empty on read frames before bucket-name restoration. Replays whose schema has no `"buckets"` array still read back with an empty `bucket_map`, so their per-frame pass remains a no-op (unchanged from before) - **samples/benchmarks**: `basic_write.cpp` and `bench_writer.cpp` created a `"Players"` bucket while writing against the arena schema, which declares `"buckets": ["entity"]` -- under schema-driven normalization those frames would now be rejected. Both use `"entity"` diff --git a/sdk/include/vtx/reader/core/vtx_reader.h b/sdk/include/vtx/reader/core/vtx_reader.h index 279121e..69da043 100644 --- a/sdk/include/vtx/reader/core/vtx_reader.h +++ b/sdk/include/vtx/reader/core/vtx_reader.h @@ -667,6 +667,7 @@ namespace VTX { SerializerPolicy::ProcessChunkData(idx, compressed_blob, stop_token, cc.native_frames, cc.decompressed_blob, cc.raw_frames_spans); RestoreBucketNames(cc.native_frames); + RestoreDeclaredArrays(cc.native_frames); return cc; } catch (const std::exception& e) { VTX_ERROR("[READER] Chunk {} deserialization failed: {}", idx, e.what()); @@ -696,6 +697,43 @@ namespace VTX { } } + // A declared array field with no data is not serialized (an empty array has + // nothing to store), so deserialized entities come back missing those + // subarrays. Re-create the declared-but-empty subarrays from the schema so a + // read frame mirrors the same array layout an ingest-loaded frame has (maps + // already round-trip their slot count, so only arrays need this). Recurses + // into nested structs / struct-array elements / map values, matching the + // loader's recursive PrepareContainer. Grow-only: never touches populated + // arrays, scalars, or maps. + void RestoreDeclaredArrays(std::vector& frames) const { + if (property_address_cache_.structs.empty()) + return; + for (auto& frame : frames) { + for (auto& bucket : frame.GetMutableBuckets()) { + for (auto& entity : bucket.entities) { + PreSizeDeclaredArraysRecursive(entity); + } + } + } + } + + void PreSizeDeclaredArraysRecursive(VTX::PropertyContainer& container) const { + auto it = property_address_cache_.structs.find(container.entity_type_id); + if (it != property_address_cache_.structs.end()) { + Helpers::EnsureDeclaredArrays(container, it->second.array_max_indices); + } + for (auto& nested : container.any_struct_properties) + PreSizeDeclaredArraysRecursive(nested); + for (auto& nested : container.any_struct_arrays.data) + PreSizeDeclaredArraysRecursive(nested); + for (auto& map_item : container.map_properties) + for (auto& value : map_item.values) + PreSizeDeclaredArraysRecursive(value); + for (auto& map_item : container.map_arrays.data) + for (auto& value : map_item.values) + PreSizeDeclaredArraysRecursive(value); + } + private: std::string filepath_; diff --git a/sdk/src/vtx_writer/src/vtx/writer/formatters/flatbuffers_vtx_policy.cpp b/sdk/src/vtx_writer/src/vtx/writer/formatters/flatbuffers_vtx_policy.cpp index 0cfab45..85fec1b 100644 --- a/sdk/src/vtx_writer/src/vtx/writer/formatters/flatbuffers_vtx_policy.cpp +++ b/sdk/src/vtx_writer/src/vtx/writer/formatters/flatbuffers_vtx_policy.cpp @@ -21,14 +21,7 @@ std::unique_ptr VTX::FlatBuffersVtxPolicy: sorted_frame->GetMutableBuckets().resize(native_buckets.size()); for (size_t b_idx = 0; b_idx < native_buckets.size(); ++b_idx) { - const auto& src_bucket = native_buckets[b_idx]; - auto& dst_bucket = sorted_frame->GetBucket(static_cast(b_idx)); - - if (b_idx == 0) { - Serialization::SortBucketByTypeId(src_bucket, dst_bucket); - } else { - dst_bucket = src_bucket; - } + Serialization::SortBucketByTypeId(native_buckets[b_idx], sorted_frame->GetBucket(static_cast(b_idx))); } return sorted_frame; diff --git a/sdk/src/vtx_writer/src/vtx/writer/formatters/protobuff_vtx_policy.cpp b/sdk/src/vtx_writer/src/vtx/writer/formatters/protobuff_vtx_policy.cpp index 0f00b69..0e44c6c 100644 --- a/sdk/src/vtx_writer/src/vtx/writer/formatters/protobuff_vtx_policy.cpp +++ b/sdk/src/vtx_writer/src/vtx/writer/formatters/protobuff_vtx_policy.cpp @@ -16,14 +16,7 @@ std::unique_ptr VTX::ProtobufVtxPolicy::FromN sorted_native.GetMutableBuckets().resize(native_buckets.size()); for (size_t b_idx = 0; b_idx < native_buckets.size(); ++b_idx) { - const auto& src_bucket = native_buckets[b_idx]; - auto& dst_bucket = sorted_native.GetBucket(static_cast(b_idx)); - - if (b_idx == 0) { - Serialization::SortBucketByTypeId(src_bucket, dst_bucket); - } else { - dst_bucket = src_bucket; - } + Serialization::SortBucketByTypeId(native_buckets[b_idx], sorted_native.GetBucket(static_cast(b_idx))); } auto proto = std::make_unique(); diff --git a/tests/common/test_schema_registry.cpp b/tests/common/test_schema_registry.cpp index d60e4b5..56982ca 100644 --- a/tests/common/test_schema_registry.cpp +++ b/tests/common/test_schema_registry.cpp @@ -184,3 +184,92 @@ TEST(SchemaRegistry, ReloadReplacesBucketNames) { ASSERT_TRUE(schema.LoadFromRawString(raw)); EXPECT_TRUE(schema.GetBucketNames().empty()); } + +// --------------------------------------------------------------------------- +// Array / Map pre-sizing (symmetric with scalar pre-sizing) +// --------------------------------------------------------------------------- + +namespace { + // Player declares: 2 int32 scalars, 1 float array, 2 string arrays, + // 1 struct array (Inventory of Item) and 1 struct map (AmmoByWeapon of Ammo). + constexpr const char* kArrayMapSchema = R"({ + "version": "1.0.0", + "buckets": ["entity"], + "property_mapping": [ + { "struct": "Item", "values": [ + {"name":"Id","typeId":"Int32","containerType":"None","keyId":"None","structType":"","meta":{"defaultValue":"0","fixedArrayDim":1}} + ]}, + { "struct": "Ammo", "values": [ + {"name":"Count","typeId":"Int32","containerType":"None","keyId":"None","structType":"","meta":{"defaultValue":"0","fixedArrayDim":1}} + ]}, + { "struct": "Player", "values": [ + {"name":"Score","typeId":"Int32","containerType":"None","keyId":"None","structType":"","meta":{"defaultValue":"0","fixedArrayDim":1}}, + {"name":"Level","typeId":"Int32","containerType":"None","keyId":"None","structType":"","meta":{"defaultValue":"0","fixedArrayDim":1}}, + {"name":"Cooldowns","typeId":"Float","containerType":"Array","keyId":"None","structType":"","meta":{"defaultValue":"","fixedArrayDim":0}}, + {"name":"Tags","typeId":"String","containerType":"Array","keyId":"None","structType":"","meta":{"defaultValue":"","fixedArrayDim":0}}, + {"name":"Names","typeId":"String","containerType":"Array","keyId":"None","structType":"","meta":{"defaultValue":"","fixedArrayDim":0}}, + {"name":"Inventory","typeId":"Struct","containerType":"Array","keyId":"None","structType":"Item","meta":{"defaultValue":"","fixedArrayDim":0}}, + {"name":"AmmoByWeapon","typeId":"Struct","containerType":"Map","keyId":"String","structType":"Ammo","meta":{"defaultValue":"","fixedArrayDim":1}} + ]} + ] + })"; +} // namespace + +TEST(SchemaRegistry, PreSizesArraysAndMapsFromSchema) { + VTX::SchemaRegistry schema; + ASSERT_TRUE(schema.LoadFromRawString(kArrayMapSchema)); + + const VTX::SchemaStruct* player = schema.GetStruct("Player"); + ASSERT_NE(player, nullptr); + + VTX::PropertyContainer c; + VTX::Helpers::PreparePropertyContainer(c, *player); + + // Scalars: unchanged (two int32 fields). + EXPECT_EQ(c.int32_properties.size(), 2u); + + // Arrays: one empty subarray per declared array field, grouped per type. + EXPECT_EQ(c.float_arrays.SubArrayCount(), 1u); + EXPECT_EQ(c.string_arrays.SubArrayCount(), 2u); + EXPECT_EQ(c.any_struct_arrays.SubArrayCount(), 1u); + EXPECT_TRUE(c.float_arrays.GetSubArray(0).empty()); + EXPECT_TRUE(c.string_arrays.GetSubArray(1).empty()); + + // Array types with no declared fields stay untouched. + EXPECT_EQ(c.int32_arrays.SubArrayCount(), 0u); + EXPECT_EQ(c.vector_arrays.SubArrayCount(), 0u); + + // Map: one empty, Struct-valued map slot. + ASSERT_EQ(c.map_properties.size(), 1u); + EXPECT_TRUE(c.map_properties[0].keys.empty()); + EXPECT_TRUE(c.map_properties[0].values.empty()); +} + +TEST(SchemaRegistry, ArrayAndMapSizingLandInPropertyCache) { + VTX::SchemaRegistry schema; + ASSERT_TRUE(schema.LoadFromRawString(kArrayMapSchema)); + + const int32_t player_id = schema.GetStructTypeId("Player"); + ASSERT_GE(player_id, 0); + + const auto& sc = schema.GetPropertyCache().structs.at(player_id); + const auto string_idx = static_cast(VTX::FieldType::String); + const auto float_idx = static_cast(VTX::FieldType::Float); + ASSERT_GT(sc.array_max_indices.size(), string_idx); + EXPECT_EQ(sc.array_max_indices[string_idx], 2); + EXPECT_EQ(sc.array_max_indices[float_idx], 1); + EXPECT_EQ(sc.map_max_index, 1); +} + +// A schema without array/map fields leaves those sizes empty/zero. +TEST(SchemaRegistry, ScalarOnlySchemaHasNoArrayOrMapSizing) { + const std::string raw = std::string(R"({ "version": "1.0.0", )") + kTinyMapping + "}"; + VTX::SchemaRegistry schema; + ASSERT_TRUE(schema.LoadFromRawString(raw)); + + const int32_t id = schema.GetStructTypeId("Tiny"); + ASSERT_GE(id, 0); + const auto& sc = schema.GetPropertyCache().structs.at(id); + EXPECT_TRUE(sc.array_max_indices.empty()); + EXPECT_EQ(sc.map_max_index, 0); +} diff --git a/tests/writer/test_roundtrip.cpp b/tests/writer/test_roundtrip.cpp index 9c8170e..7b8da63 100644 --- a/tests/writer/test_roundtrip.cpp +++ b/tests/writer/test_roundtrip.cpp @@ -257,9 +257,97 @@ TEST_P(RoundtripTest, MultiBucketFramesSurviveRoundtrip) { ASSERT_EQ(buckets[2].entities.size(), 1u); EXPECT_EQ(buckets[2].unique_ids[0], "economy_0"); EXPECT_EQ(buckets[2].entities[0].int32_properties[1], 2); + + // type_ranges must be rebuilt for EVERY bucket, not just index 0, so + // typed access works on non-first buckets too (regression: SortBucketByTypeId + // used to run only for bucket 0, leaving others with empty type_ranges). + EXPECT_EQ(buckets[0].GetEntitiesOfType(0).size(), 1u); + EXPECT_EQ(buckets[1].GetEntitiesOfType(0).size(), 1u); + EXPECT_EQ(buckets[2].GetEntitiesOfType(0).size(), 1u); } } +// --------------------------------------------------------------------------- +// A frame built positionally (raw GetMutableBuckets().push_back, empty +// bucket_map) is adopted into the schema layout instead of rejected. +// --------------------------------------------------------------------------- + +TEST_P(RoundtripTest, AdoptsPositionallyBuiltFrameToSchemaLayout) { + auto cfg = MakeConfig("positional", "uuid-rt-positional"); + cfg.schema_json_path.clear(); + cfg.schema_json_content = kMultiBucketSchema; // declares ["entity","bone_data","economy"] + { + auto writer = CreateWriter(cfg); + ASSERT_TRUE(writer); + for (int i = 0; i < 4; ++i) { + VTX::Frame f; + // Build positionally: no CreateBucket, so bucket_map stays empty. + auto& raw = f.GetMutableBuckets(); + raw.emplace_back(); // will map to schema bucket 0 ("entity") + VTX::PropertyContainer pc; + pc.entity_type_id = 0; + pc.int32_properties = {i}; + raw[0].unique_ids.push_back("e_0"); + raw[0].entities.push_back(std::move(pc)); + + VTX::GameTime::GameTimeRegister t; + t.game_time = float(i) / kFps; + const auto r = writer->TryRecordFrame(f, t); + EXPECT_TRUE(r.IsWritten()) << "positional frame " << i << " rejected: " << r.error.message; + } + writer->Stop(); + } + + auto ctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(ctx) << ctx.error; + ASSERT_EQ(ctx.reader->GetTotalFrames(), 4); + + const VTX::Frame* f = ctx.reader->GetFrameSync(0); + ASSERT_NE(f, nullptr); + // The single positional bucket was adopted as schema bucket 0 and the two + // remaining declared buckets were materialized empty. + ASSERT_EQ(f->GetBuckets().size(), 3u); + EXPECT_EQ(f->bucket_map.at("entity"), 0u); + ASSERT_EQ(f->GetBuckets()[0].entities.size(), 1u); + EXPECT_EQ(f->GetBuckets()[0].unique_ids[0], "e_0"); + EXPECT_TRUE(f->GetBuckets()[1].entities.empty()); + EXPECT_TRUE(f->GetBuckets()[2].entities.empty()); +} + +// --------------------------------------------------------------------------- +// A schema without a "buckets" array (legacy) may record zero-bucket frames +// (idle/gap ticks). They must round-trip without crashing on load. +// Regression: the reader policies indexed bucket 0 unconditionally. +// --------------------------------------------------------------------------- + +TEST_P(RoundtripTest, ZeroBucketFramesFromSchemalessWriterRoundtrip) { + auto cfg = MakeConfig("zero_bucket", "uuid-rt-zerobucket"); + cfg.schema_json_path.clear(); + cfg.schema_json_content = + R"({"version":"1.0.0","property_mapping":[{"struct":"S","values":[)" + R"({"name":"X","typeId":"Int32","containerType":"None","keyId":"None","structType":"","meta":{"defaultValue":"0","fixedArrayDim":1}})" + R"(]}]})"; + { + auto writer = CreateWriter(cfg); + ASSERT_TRUE(writer); + for (int i = 0; i < 3; ++i) { + VTX::Frame frame; // zero buckets + VTX::GameTime::GameTimeRegister t; + t.game_time = float(i) / kFps; + writer->RecordFrame(frame, t); + } + writer->Stop(); + } + + auto ctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(ctx) << ctx.error; + ASSERT_EQ(ctx.reader->GetTotalFrames(), 3); + + const VTX::Frame* f = ctx.reader->GetFrameSync(0); // must not crash + ASSERT_NE(f, nullptr); + EXPECT_TRUE(f->GetBuckets().empty()); +} + // --------------------------------------------------------------------------- // Bucket names restored on read for the single-bucket fixture schema too. // --------------------------------------------------------------------------- @@ -291,6 +379,161 @@ TEST_P(RoundtripTest, RestoresBucketNamesFromSchemaOnRead) { EXPECT_EQ(frame_ref.GetBucket("entity").entities.size(), 1u); } +// --------------------------------------------------------------------------- +// Populated array and map field VALUES survive write -> read on both backends. +// (The existing scalar roundtrip does not cover FlatArray / map serialization.) +// Note: pre-sizing creates EMPTY declared slots in memory, but an array with no +// data is intentionally not serialized, so this covers the case that matters on +// disk -- populated arrays/maps. +// --------------------------------------------------------------------------- + +TEST_P(RoundtripTest, ArrayAndMapFieldValuesRoundtrip) { + auto cfg = MakeConfig("arraymap", "uuid-rt-arraymap"); + { + auto writer = CreateWriter(cfg); + ASSERT_TRUE(writer); + + VTX::Frame f; + auto& bucket = f.CreateBucket("entity"); + + VTX::PropertyContainer e; + e.entity_type_id = 0; + e.float_arrays.AppendSubArray({1.5f, 2.5f, 3.5f}); + e.string_arrays.AppendSubArray({std::string("alpha"), std::string("beta")}); + + VTX::MapContainer m; + m.keys = {"rifle", "pistol"}; + VTX::PropertyContainer v0; + v0.entity_type_id = 0; + v0.int32_properties = {30}; + VTX::PropertyContainer v1; + v1.entity_type_id = 0; + v1.int32_properties = {12}; + m.values.push_back(std::move(v0)); + m.values.push_back(std::move(v1)); + e.map_properties.push_back(std::move(m)); + + bucket.unique_ids.push_back("e0"); + bucket.entities.push_back(std::move(e)); + + VTX::GameTime::GameTimeRegister t; + t.game_time = 0.0f; + writer->RecordFrame(f, t); + writer->Stop(); + } + + auto ctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(ctx) << ctx.error; + + const VTX::Frame* f = ctx.reader->GetFrameSync(0); + ASSERT_NE(f, nullptr); + ASSERT_EQ(f->GetBuckets().size(), 1u); + ASSERT_EQ(f->GetBuckets()[0].entities.size(), 1u); + const auto& e = f->GetBuckets()[0].entities[0]; + + // Float array survives with its values, in order. + ASSERT_EQ(e.float_arrays.SubArrayCount(), 1u); + const auto fa = e.float_arrays.GetSubArray(0); + ASSERT_EQ(fa.size(), 3u); + EXPECT_FLOAT_EQ(fa[0], 1.5f); + EXPECT_FLOAT_EQ(fa[1], 2.5f); + EXPECT_FLOAT_EQ(fa[2], 3.5f); + + // String array survives. + ASSERT_EQ(e.string_arrays.SubArrayCount(), 1u); + const auto sa = e.string_arrays.GetSubArray(0); + ASSERT_EQ(sa.size(), 2u); + EXPECT_EQ(sa[0], "alpha"); + EXPECT_EQ(sa[1], "beta"); + + // Map survives with keys and per-key values. + ASSERT_EQ(e.map_properties.size(), 1u); + ASSERT_EQ(e.map_properties[0].keys.size(), 2u); + EXPECT_EQ(e.map_properties[0].keys[0], "rifle"); + EXPECT_EQ(e.map_properties[0].keys[1], "pistol"); + ASSERT_EQ(e.map_properties[0].values.size(), 2u); + ASSERT_FALSE(e.map_properties[0].values[0].int32_properties.empty()); + EXPECT_EQ(e.map_properties[0].values[0].int32_properties[0], 30); + EXPECT_EQ(e.map_properties[0].values[1].int32_properties[0], 12); +} + +// --------------------------------------------------------------------------- +// Declared-but-empty array fields are restored on read. An array with no data +// is not serialized, so a reader must re-create the declared empty subarrays +// from the schema for the read frame to mirror an ingest-loaded frame's layout. +// (Maps already round-trip their slot count, so this covers only arrays.) +// --------------------------------------------------------------------------- + +namespace { + // Hero declares three array-typed fields (one float array, two string arrays, + // one struct array); an entity that leaves them empty exercises read-side + // restoration. + constexpr const char* kHeroArraySchema = R"({ + "version": "1.0.0", + "buckets": ["entity"], + "property_mapping": [ + { "struct": "Item", "values": [ + {"name":"Id","typeId":"Int32","containerType":"None","keyId":"None","structType":"","meta":{"defaultValue":"0","fixedArrayDim":1}} + ]}, + { "struct": "Hero", "values": [ + {"name":"UniqueID","typeId":"String","containerType":"None","keyId":"None","structType":"","meta":{"defaultValue":"","fixedArrayDim":1}}, + {"name":"Score","typeId":"Int32","containerType":"None","keyId":"None","structType":"","meta":{"defaultValue":"0","fixedArrayDim":1}}, + {"name":"Cooldowns","typeId":"Float","containerType":"Array","keyId":"None","structType":"","meta":{"defaultValue":"","fixedArrayDim":0}}, + {"name":"Tags","typeId":"String","containerType":"Array","keyId":"None","structType":"","meta":{"defaultValue":"","fixedArrayDim":0}}, + {"name":"Names","typeId":"String","containerType":"Array","keyId":"None","structType":"","meta":{"defaultValue":"","fixedArrayDim":0}}, + {"name":"Inventory","typeId":"Struct","containerType":"Array","keyId":"None","structType":"Item","meta":{"defaultValue":"","fixedArrayDim":0}} + ]} + ] + })"; +} // namespace + +TEST_P(RoundtripTest, ReaderRestoresDeclaredEmptyArrays) { + VTX::SchemaRegistry reg; + ASSERT_TRUE(reg.LoadFromRawString(kHeroArraySchema)); + const int32_t heroTypeId = reg.GetStructTypeId("Hero"); + ASSERT_GE(heroTypeId, 0); + + auto cfg = MakeConfig("declared_empty_arrays", "uuid-rt-declared-empty"); + cfg.schema_json_path.clear(); + cfg.schema_json_content = kHeroArraySchema; + { + auto writer = CreateWriter(cfg); + ASSERT_TRUE(writer); + + VTX::Frame f; + auto& bucket = f.CreateBucket("entity"); + VTX::PropertyContainer e; + e.entity_type_id = heroTypeId; + e.string_properties = {"hero_1"}; // only a scalar; all array fields left empty + bucket.unique_ids.push_back("hero_1"); + bucket.entities.push_back(std::move(e)); + + VTX::GameTime::GameTimeRegister t; + t.game_time = 0.0f; + writer->RecordFrame(f, t); + writer->Stop(); + } + + auto ctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(ctx) << ctx.error; + + const VTX::Frame* f = ctx.reader->GetFrameSync(0); + ASSERT_NE(f, nullptr); + ASSERT_EQ(f->GetBuckets()[0].entities.size(), 1u); + const auto& e = f->GetBuckets()[0].entities[0]; + + // The declared array fields come back present-and-empty even though nothing + // was serialized for them (without read-side restoration these would be 0). + EXPECT_EQ(e.float_arrays.SubArrayCount(), 1u); // Cooldowns + EXPECT_EQ(e.string_arrays.SubArrayCount(), 2u); // Tags, Names + EXPECT_EQ(e.any_struct_arrays.SubArrayCount(), 1u); // Inventory + EXPECT_TRUE(e.float_arrays.GetSubArray(0).empty()); + EXPECT_TRUE(e.string_arrays.GetSubArray(0).empty()); + + // A type with no declared array field is untouched. + EXPECT_EQ(e.int32_arrays.SubArrayCount(), 0u); +} + // --------------------------------------------------------------------------- // Backend instantiation -- produces: // BothBackends/RoundtripTest.PreservesFrameData/FlatBuffers