BsonJsonConverter fails for certain large json objects

[zupfnet]

Package version

5.1

Affected package

BLite.Bson (BSON library)

.NET version

10.0.401

Description

BsonJsonConverter.FromJson fails with certain long json input.
It turned out, the size of the largest the object is an issue.

Minimal reproduction

var json = 
"""
{"streams":[{"index":0,"codec_type":"video","codec_tag_string":"brhq","codec_tag":"0x71687262","width":2720,"height":1528,"has_b_frames":0,"level":-99,"id":"0x1","r_frame_rate":"25/1","avg_frame_rate":"25/1","time_base":"1/25","start_pts":0,"start_time":"0.000000","duration_ts":328,"duration":"13.120000","bit_rate":"256326868","nb_frames":"328","tags":{"creation_time":"2021-02-02T03:49:49.000000Z","language":"eng","handler_name":"\u0019Apple Video Media Handler","vendor_id":"[0][0][0][25]","timecode":"05:41:49:16"}},{"index":1,"codec_name":"pcm_s24le","codec_long_name":"PCM signed 24-bit little-endian","codec_type":"audio","codec_tag_string":"in24","codec_tag":"0x34326e69","sample_fmt":"s32","sample_rate":"48000","channels":2,"bits_per_sample":24,"initial_padding":0,"id":"0x2","r_frame_rate":"0/0","avg_frame_rate":"0/0","time_base":"1/48000","start_pts":0,"start_time":"0.000000","duration_ts":629760,"duration":"13.120000","bit_rate":"2304000","bits_per_raw_sample":"24","nb_frames":"629760","tags":{"creation_time":"2021-02-02T03:49:49.000000Z","language":"eng","handler_name":"\u0019Apple Sound Media Handler","vendor_id":"[0][0][0][0]","timecode":"05:41:49:16"}},{"index":2,"codec_type":"data","codec_tag_string":"tmcd","codec_tag":"0x64636d74","id":"0x3","r_frame_rate":"0/0","avg_frame_rate":"25/1","time_base":"1/25","start_pts":0,"start_time":"0.000000","duration_ts":328,"duration":"13.120000","bit_rate":"2","nb_frames":"1","extradata_size":18,"tags":{"creation_time":"2021-02-02T03:49:49.000000Z","language":"eng","handler_name":"Time Code Media Handler","timecode":"05:41:49:16"}}]}
""";
var engine = BLiteEngine.CreateInMemory();
var result = BsonJsonConverter.FromJson(json, engine.GetKeyMap(), engine.GetReverseMap());

Expected behavior

return a BsonDocument object

Actual behavior

{System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
at BLite.Bson.BsonSpanWriter.WriteString(String name, String value)
at BLite.Bson.BsonValue.WriteTo(BsonSpanWriter& writer, String fieldName)
at BLite.Bson.BsonDocument.WriteFieldsTo(BsonSpanWriter& writer)
at BLite.Bson.BsonValue.WriteTo(BsonSpanWriter& writer, String fieldName)
at BLite.Bson.BsonDocument.WriteFieldsTo(BsonSpanWriter& writer)
at BLite.Bson.BsonValue.WriteToArray(BsonSpanWriter& writer, Int32 index)
at BLite.Bson.BsonValue.WriteTo(BsonSpanWriter& writer, String fieldName)
at BLite.Bson.BsonDocumentBuilder.Add(String name, BsonValue value)
at BLite.Bson.BsonJsonConverter.<>c__DisplayClass4_0.b__0(BsonDocumentBuilder builder)
at BLite.Bson.BsonDocument.Create(IReadOnlyDictionary2 keyMap, ConcurrentDictionary2 reverseKeyMap, Action1 buildAction) at BLite.Bson.BsonJsonConverter.ObjectElementToDocument(JsonElement obj, ConcurrentDictionary2 keyMap, ConcurrentDictionary2 reverseKeyMap) at BLite.Bson.BsonJsonConverter.FromJson(String json, ConcurrentDictionary2 keyMap, ConcurrentDictionary`2 reverseKeyMap)

Additional context

If any Json object or array results in a BsonValue > 1024 Bytes, the reseved capacity is insufficient:

public BsonDocumentBuilder Add(string name, BsonValue value)
    {
        EnsureCapacity(1024);   // <-- fixed pad, sufficient for simple values but any json object will be converted to a value
        var writer = new BsonSpanWriter(_buffer.AsSpan(_position..), _keyMap);
        value.WriteTo(ref writer, name);
        _position += writer.Position;
        return this;
    }

With nested object trees this limit is easily reached.

[github-actions]

👋 Thanks for the well-formed bug report! It has been added to the triage queue.

[fskalkan]

Hi, I’d like to take a look at this one if it’s available.