diff --git a/CHANGELOG.md b/CHANGELOG.md index 4ae3cd1..b8566aa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,12 +37,23 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) relocates every cache key, so existing entries are rewritten under the new spelling. - **`CacheKeyComparer`** exposes cached `Sensitive` / `Insensitive` equality comparers over `CacheKey`, in the `StringComparer` shape. -- **`ISerializerProxy`** is a new serialization seam, defaulting to - `SystemJsonByteSerializerProxy`: byte payloads pass through raw (no base64, no JSON) and everything - else is UTF-8 JSON, with the requested type argument deciding. The existing - `ISerializerProxy` registration and every existing wire format are untouched. Swapping - in a binary serializer such as MessagePack is one class and one registration — see - [how-to/extending.md](docs/how-to/extending.md). +- **`ISerializerProxy`** is now the library's only serialization seam, replacing + `ISerializerProxy` everywhere — see **Removed** below. The serialization contract no + longer names a Redis type, so a custom serializer needs no dependency on StackExchange.Redis, and + one registration covers `ICache`, `IHashCache` and `ISetCache`. Swapping in a binary serializer + such as MessagePack is one class and one registration — see + [how-to/extending.md](docs/how-to/extending.md). Two implementations ship: + - `SystemJsonByteSerializerProxy` (the default) is UTF-8 JSON for every value, which is + byte-for-byte the format `SystemJsonSerializerProxy` wrote — **no cache entry needs migrating**, + byte payloads and nulls included. + - `RawByteSerializerProxy` stores byte payloads verbatim — no base64, no JSON, no encoding layer — + and JSON for every other type. `AddDistributedCache` constructs this for the provider it builds + rather than resolving one from DI, because an `IDistributedCache` payload is caller bytes the + caller has already serialized and re-encoding them would be both wasteful and wrong; that also + means replacing the app-wide serializer does not change what the adapter stores. Registering it + app-wide is available as an opt-in, and *is* a wire-format change: it returns stored bytes as-is + rather than base64-decoding them, so relocate the keyspace (a version segment in `AppShortName` + or the cache-key strategy) when switching an existing deployment over. - **`RedisCacheOptions.AwaitRefresh`** (default `false`) waits for the server to apply a refresh instead of sending `KEYEXPIRE`/`PERSIST` fire-and-forget. Off by default, which keeps the round trip off the sliding-expiration path — it runs on every read of a sliding entry — and preserves existing behavior. @@ -118,6 +129,25 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - **`Microsoft.Extensions.*` dependency floor for `net10.0` raised to 10.0.11** (from 10.0.10). The `net8.0` floor is unchanged at 8.0.x. - **OpenTelemetry packages moved to 1.18.0** (`OpenTelemetry.Instrumentation.StackExchangeRedis` to 1.18.0-beta.1). +- **BREAKING (source, not data):** every cache now serializes through `ISerializerProxy`. + `RedisCache`, `RedisHashCache`, `RedisSetCache`, the memory set tier and all four providers take + `ISerializerProxy` in place of `ISerializerProxy`, and the broadcast change + token is built as `ChangeTokenFactory`. **No stored entry changes format and nothing needs + migrating** — the default serializer emits exactly what `SystemJsonSerializerProxy` emitted. + Consumers who pass these constructors a serializer directly, or who register a custom one, need a + one-line type change; `AddCaching` now throws at startup if a leftover + `ISerializerProxy` registration is present, rather than ignoring it and silently + falling back to JSON. + +### Removed + +- **BREAKING:** `SystemJsonSerializerProxy` and the `ISerializerProxy` registration. + `SystemJsonByteSerializerProxy` (in `UiPath.Caching.Abstractions`) is the single default and writes + the same bytes. The generic `ISerializerProxy` interface itself is unchanged. +- **BREAKING:** `IEventFormatterProxy.Decode(string)` and `EncodeAsString(T)`, both obsolete since + the broadcast path moved to `ReadOnlyMemory` and both unused — nothing in the library called + them and no implementation overrode the default bodies. Use `Decode(ReadOnlyMemory)` and + `Encode(T)`. ### Fixed diff --git a/docs/how-to/extending.md b/docs/how-to/extending.md index fcd74cc..2aef8cb 100644 --- a/docs/how-to/extending.md +++ b/docs/how-to/extending.md @@ -11,7 +11,7 @@ Triage what you actually need before reaching for the seams below. | Change how keys look on Redis (prefixing, sharding, namespacing) | `ICacheKeyStrategy` — see [telemetry-and-strategies.md](telemetry-and-strategies.md#cache-key-strategies) | | Use a different Redis instance, custom multiplexer, or OTel hookup | `IConnectionMultiplexerFactory` — see [recipes/opentelemetry-multiplexer-factory.md](../recipes/opentelemetry-multiplexer-factory.md) | | Route telemetry events to a non-OTel surface | `ICachingTelemetryProvider` — see [recipes/custom-telemetry-provider.md](../recipes/custom-telemetry-provider.md) | -| Change how cache values are serialized to Redis | `ISerializerProxy` — see [Serializers](#custom-serializer) below | +| Change how cache values are serialized to Redis | `ISerializerProxy` — see [Serializers](#custom-serializer) below | | Add a brand-new storage backend (Memcached, S3, local file, in-memory test fake) | `ICacheProvider` — see [Cache providers](#custom-cache-provider) below | | Add a brand-new cross-node broadcast transport (Kafka, NATS, RabbitMQ) | `ITopicProvider` — see [Topic providers](#custom-topic-provider) below | | Override the lock backend for cross-node single-flight | `IDistributedLock` — register a custom impl via `services.AddSingleton()` | @@ -208,7 +208,33 @@ Select via configuration: ## Custom serializer -`ISerializerProxy` is the seam for cache-value serialization. The library ships exactly one implementation — `SystemJsonSerializerProxy` (which implements `ISerializerProxy`) — and consumers swap it to use MessagePack, ProtoBuf, MemoryPack, or any other format. +`ISerializerProxy` is the seam for cache-value serialization — one seam for every cache the +library creates, including the `IDistributedCache` adapter. Consumers swap it to use MessagePack, +ProtoBuf, MemoryPack, or any other format. + +The seam lives in `UiPath.Caching.Abstractions` and names no Redis type, so a custom serializer needs +no dependency on StackExchange.Redis. + +Two implementations ship: + +| | Byte payloads | Everything else | Used by | +|---|---|---|---| +| `SystemJsonByteSerializerProxy` (default) | base64 inside a JSON string | UTF-8 JSON | every cache, unless you replace it | +| `RawByteSerializerProxy` | stored verbatim | UTF-8 JSON | the provider `AddDistributedCache` builds | + +The default is precisely the format the library has always written, so upgrading moves no cache entry. +(A null value is the one exception: it now serializes to a null payload rather than the four-byte +JSON `null` literal, following the contract note below. Both read back as `default`.) +`RawByteSerializerProxy` exists because an `IDistributedCache` +payload is caller bytes that the caller has already serialized: base64-ing them would cost a third +again in size for no benefit, and a consumer who swaps in MessagePack should not have their session +bytes re-encoded. Neither implementation sniffs the payload — the requested type argument decides. + +You can register `RawByteSerializerProxy` app-wide to get raw byte storage everywhere, but that *is* +a wire-format change: it returns stored bytes as-is rather than base64-decoding them, so a 1.x +`byte[]` entry would come back as the base64 ASCII, and without throwing — nothing degrades it to a +cache miss. Relocate the keyspace (a version segment in `AppShortName` or in the cache-key strategy) +so reads cannot land on entries written the other way. ### Interface @@ -228,41 +254,50 @@ The two `TryDeserialize` overloads exist so callers can attempt a deserializatio ### Skeleton implementation +Most binary serializers natively produce `byte[]`, so an implementation is mostly delegation: + ```csharp using MessagePack; -using StackExchange.Redis; using UiPath.Caching; -public sealed class MessagePackSerializerProxy : ISerializerProxy +public sealed class MessagePackByteSerializerProxy(MessagePackSerializerOptions? options = null) + : ISerializerProxy { - private readonly MessagePackSerializerOptions _options = MessagePackSerializerOptions.Standard; - - public RedisValue Serialize(object? value) => - value is null ? RedisValue.Null : MessagePackSerializer.Serialize(value.GetType(), value, _options); + public byte[]? Serialize(object? value) => + value is null ? null : MessagePackSerializer.Serialize(value.GetType(), value, options); - public T? Deserialize(RedisValue value) => - value.IsNull ? default : MessagePackSerializer.Deserialize((byte[])value!, _options); + public T? Deserialize(byte[]? value) => + value is null or { Length: 0 } ? default : MessagePackSerializer.Deserialize(value, options); public bool TryDeserialize(string? value, out T? result) { - // MessagePack is a binary format; string inputs aren't expected from RedisValue - // (which stores byte[] directly). Return false here so callers route to the - // RedisValue overload via the TryDeserialize(object?) entry point. result = default; - return false; + if (string.IsNullOrWhiteSpace(value)) + { + return false; + } + try + { + result = MessagePackSerializer.Deserialize(Convert.FromBase64String(value), options); + return true; + } + catch + { + return false; + } } - public bool TryDeserialize(object? value, out T? result) => - value is RedisValue rv ? TryDeserializeRedis(rv, out result) : TryDeserialize(value?.ToString(), out result); - - private bool TryDeserializeRedis(RedisValue value, out T? result) + public bool TryDeserialize(object? value, out T? result) { result = default; - if (value.IsNull) return false; try { - result = MessagePackSerializer.Deserialize((byte[])value!, _options); - return true; + if (value is byte[] bytes) + { + result = MessagePackSerializer.Deserialize(bytes, options); + return true; + } + return TryDeserialize(value?.ToString(), out result); } catch { @@ -275,23 +310,22 @@ public sealed class MessagePackSerializerProxy : ISerializerProxy ### Registration ```csharp -services.AddSingleton, MessagePackSerializerProxy>(); +services.AddSingleton>(new MessagePackByteSerializerProxy()); ``` -That single registration replaces the default JSON serializer for every cache the library creates. +That single registration replaces the default JSON serializer for the caches the library resolves +from DI — `ICache`, `IHashCache` and `ISetCache`. It does **not** reach the `IDistributedCache` +adapter, which constructs its own `RawByteSerializerProxy`; see the contract note below. ### Contract notes -- **`Serialize(null)` must round-trip.** Producing a `RedisValue.Null` on input null is the convention; `Deserialize` should return `default` for `RedisValue.Null` input. If you persist null sentinels differently, the cache's `CacheNullValues` flag and the `NullCache` write-path will not behave correctly. -- **Schema evolution is your problem.** The library does not version cached payloads. If your serializer can't deserialize an older payload after a deploy, `TryDeserialize` should return `false` and the cache will treat that as a miss; the generator will run and write a fresh entry. -- **Consider `RedisValue` directly.** `RedisValue` can hold strings, bytes, or numbers without conversion. A binary serializer (MessagePack, ProtoBuf, MemoryPack) should write `byte[]` directly via `RedisValue` implicit conversion rather than going through `string`/Base64 — JSON-style proxies can stay string-based. - -### Byte-oriented serializers (`ISerializerProxy`) +- **`Serialize(null)` must return a non-null payload.** A null return reaches StackExchange.Redis as `RedisValue.Null`, which throws `ArgumentException` on `SADD` and on the multi-field `HSET`, and silently stores nothing on the single-value paths. Encode null as a sentinel your own `Deserialize` maps back to `default` — the default proxy uses JSON's four-byte `null` literal. `Deserialize` should also return `default` for a null or empty buffer, except where a raw type argument makes the empty buffer meaningful: `RawByteSerializerProxy.Deserialize` returns the empty array, because an empty payload is a legitimate value there rather than an absent one. +- **Schema evolution is your problem.** The library does not version cached payloads. If your serializer can't deserialize an older payload after a deploy, `TryDeserialize` should return `false` and the cache will treat that as a miss; the generator will run and write a fresh entry. Note that `Deserialize` is the hot path for reads, and a throw there is caught by the caches and reported as a miss — but a payload your serializer *misreads* without throwing is returned to the caller as-is. +- **Round-trip `byte[]` symmetrically, and decide deliberately.** Passthrough avoids a per-entry encoding layer; base64 (what the default does) keeps the format the library has always written. Either works, but the two are mutually unreadable and neither can detect the other, so a serializer swap that changes this needs a keyspace relocation, not a compatibility shim. +- **Only replace the default globally.** `AddDistributedCache` gives its own provider `RawByteSerializerProxy` explicitly rather than resolving one from DI, so replacing the app-wide registration does not change what the `IDistributedCache` adapter stores in its `data` field. That is deliberate: those bytes belong to the caller. +- **Set members are compared by their serialized bytes.** `ISetCache`'s local tier keys its snapshot on `Serialize` output using structural byte equality, so a serializer must be deterministic: two equal values must serialize identically, or set membership and de-duplication will disagree between the memory and Redis tiers. -The distributed cache path (`AddDistributedCache`) serializes through `ISerializerProxy` -instead of `ISerializerProxy`. The default, `SystemJsonByteSerializerProxy`, passes -`byte[]` payloads through raw — no base64, no JSON, no extra encoding layer — and JSON-encodes -every other type; the requested type argument decides, there is no format sniffing. +### Distributed cache keyspace The distributed cache stores each entry as a Redis hash with a `data` field plus `absexp`/`sldexp` expiration metadata, the conventional layout for this contract, so @@ -325,67 +359,6 @@ o.RedisKeyStrategyFactory = new MyRedisKeyStrategyFactory(); // gets "dh" (or Left null it inherits the application's own `RedisCacheOptions.RedisKeyStrategyFactory`, so `AppShortName`, the separator and sharding behave as they do everywhere else. -Swapping it follows the same pattern as the `RedisValue` proxy — and is simpler, because most -binary serializers natively produce `byte[]`: - -```csharp -public sealed class MessagePackByteSerializerProxy(MessagePackSerializerOptions? options = null) - : ISerializerProxy -{ - public byte[]? Serialize(object? value) => - value is null ? null : MessagePackSerializer.Serialize(value.GetType(), value, options); - - public T? Deserialize(byte[]? value) => - value is null or { Length: 0 } ? default : MessagePackSerializer.Deserialize(value, options); - - public bool TryDeserialize(string? value, out T? result) - { - result = default; - if (string.IsNullOrWhiteSpace(value)) - { - return false; - } - try - { - result = MessagePackSerializer.Deserialize(Convert.FromBase64String(value), options); - return true; - } - catch - { - return false; - } - } - - public bool TryDeserialize(object? value, out T? result) - { - result = default; - try - { - if (value is byte[] bytes) - { - result = MessagePackSerializer.Deserialize(bytes, options); - return true; - } - return TryDeserialize(value?.ToString(), out result); - } - catch - { - return false; - } - } -} - -services.AddSingleton>(new MessagePackByteSerializerProxy()); -``` - -Two rules for custom implementations: - -1. **Round-trip `byte[]` symmetrically.** The distributed cache stores the caller's payload directly - in the hash's `data` field; raw passthrough (recommended) avoids per-entry encoding overhead, but - any symmetric encoding also works. -2. **This does not change the main caches' wire format** — `ICache`/`IHashCache` still serialize - through `ISerializerProxy`. Swap both registrations if you want one format everywhere. - ## Swapping the default factories `ICacheFactory` and `ICachePolicyFactory` each have a default DI registration set up by `AddCaching` — the concrete `CacheFactory` and `DefaultCachePolicyFactory` respectively. When you need to substitute either, use the fluent `Use*Factory` extensions on `ICachingBuilder`. They internally call `Services.Replace(...)` so the swap survives the rest of the `AddCaching` pipeline (which uses `TryAddSingleton` and would otherwise lose to the default registration). diff --git a/src/UiPath.Caching.Abstractions/Broadcast/IEventFormatterProxy.cs b/src/UiPath.Caching.Abstractions/Broadcast/IEventFormatterProxy.cs index 19c7319..172386e 100644 --- a/src/UiPath.Caching.Abstractions/Broadcast/IEventFormatterProxy.cs +++ b/src/UiPath.Caching.Abstractions/Broadcast/IEventFormatterProxy.cs @@ -3,15 +3,7 @@ namespace UiPath.Caching.Broadcast; public interface IEventFormatterProxy where T : IEvent { - [Obsolete("Use Decode(ReadOnlyMemory); decoding from the RedisValue payload bytes avoids a UTF-16 transcode.")] - T? Decode(string body) => - Decode(new ReadOnlyMemory(Encoding.UTF8.GetBytes(body))); - T? Decode(ReadOnlyMemory body); ReadOnlyMemory Encode(T @event); - - [Obsolete("Use Encode(T); publishing the encoded bytes directly avoids a UTF-16 transcode.")] - string? EncodeAsString(T @event) => - Encoding.UTF8.GetString(Encode(@event).Span); } diff --git a/src/UiPath.Caching.Abstractions/PublicAPI.Unshipped.txt b/src/UiPath.Caching.Abstractions/PublicAPI.Unshipped.txt index 12654ad..32cc417 100644 --- a/src/UiPath.Caching.Abstractions/PublicAPI.Unshipped.txt +++ b/src/UiPath.Caching.Abstractions/PublicAPI.Unshipped.txt @@ -28,11 +28,17 @@ UiPath.Caching.NullCache.TryAddAsync(UiPath.Caching.CacheKey cacheKey, T? val UiPath.Caching.NullCache.TryAddAsync(UiPath.Caching.CacheKey cacheKey, T? value, System.TimeSpan? expiration = null, UiPath.Caching.CachePolicy? policy = null, System.Threading.CancellationToken token = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.ValueTask UiPath.Caching.NullCache.TryAddAsync(UiPath.Caching.CacheKey cacheKey, T? value, UiPath.Caching.CachePolicy? policy = null, System.Threading.CancellationToken token = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.ValueTask UiPath.Caching.SystemJsonByteSerializerProxy -UiPath.Caching.SystemJsonByteSerializerProxy.Deserialize(byte[]? value) -> T? -UiPath.Caching.SystemJsonByteSerializerProxy.Serialize(object? value) -> byte[]? UiPath.Caching.SystemJsonByteSerializerProxy.SystemJsonByteSerializerProxy(System.Text.Json.JsonSerializerOptions? options = null) -> void -UiPath.Caching.SystemJsonByteSerializerProxy.TryDeserialize(object? value, out T? result) -> bool +virtual UiPath.Caching.SystemJsonByteSerializerProxy.Serialize(object? value) -> byte[]? +virtual UiPath.Caching.SystemJsonByteSerializerProxy.Deserialize(byte[]? value) -> T? UiPath.Caching.SystemJsonByteSerializerProxy.TryDeserialize(string? value, out T? result) -> bool +UiPath.Caching.SystemJsonByteSerializerProxy.TryDeserialize(object? value, out T? result) -> bool +UiPath.Caching.RawByteSerializerProxy +UiPath.Caching.RawByteSerializerProxy.RawByteSerializerProxy(System.Text.Json.JsonSerializerOptions? options = null) -> void +override UiPath.Caching.RawByteSerializerProxy.Serialize(object? value) -> byte[]? +override UiPath.Caching.RawByteSerializerProxy.Deserialize(byte[]? value) -> T +*REMOVED*UiPath.Caching.Broadcast.IEventFormatterProxy.Decode(string! body) -> T? +*REMOVED*UiPath.Caching.Broadcast.IEventFormatterProxy.EncodeAsString(T event) -> string? static UiPath.Caching.CacheKey.DefaultCasing.get -> UiPath.Caching.CacheKeyCasing static UiPath.Caching.CacheKey.DefaultCasing.set -> void static UiPath.Caching.CacheKeyComparer.Insensitive.get -> UiPath.Caching.CacheKeyComparer! diff --git a/src/UiPath.Caching.Abstractions/RawByteSerializerProxy.cs b/src/UiPath.Caching.Abstractions/RawByteSerializerProxy.cs new file mode 100644 index 0000000..5beb745 --- /dev/null +++ b/src/UiPath.Caching.Abstractions/RawByteSerializerProxy.cs @@ -0,0 +1,51 @@ +using System.Text.Json; + +namespace UiPath.Caching; + +/// +/// Stores byte payloads verbatim and UTF-8 JSON for everything else, decided by the type argument. +/// AddDistributedCache gives this to its own provider, whose payload is caller bytes the +/// caller has already serialized. +/// +/// +/// Registering this app-wide is a wire-format change: it returns stored bytes as-is rather than +/// base64-decoding what wrote, and cannot detect the +/// difference. Relocate the keyspace when switching an existing deployment over. +/// +public class RawByteSerializerProxy(JsonSerializerOptions? options = null) + : SystemJsonByteSerializerProxy(options) +{ + public override byte[]? Serialize(object? value) => value switch + { + byte[] bytes => bytes, + ReadOnlyMemory memory => memory.ToArray(), + Memory memory => memory.ToArray(), + _ => base.Serialize(value), + }; + + /// + /// Declared as T, not T?: an override cannot restate that annotation on an + /// unconstrained type parameter, so the maybe-null contract comes from the base and the + /// suppressions below are what that costs. + /// + public override T Deserialize(byte[]? value) + { + if (value is null) + { + return default!; + } + if (typeof(T) == typeof(byte[])) + { + return (T)(object)value; + } + if (typeof(T) == typeof(ReadOnlyMemory)) + { + return (T)(object)new ReadOnlyMemory(value); + } + if (typeof(T) == typeof(Memory)) + { + return (T)(object)new Memory(value); + } + return base.Deserialize(value)!; + } +} diff --git a/src/UiPath.Caching.Abstractions/SystemJsonByteSerializerProxy.cs b/src/UiPath.Caching.Abstractions/SystemJsonByteSerializerProxy.cs index a2ba826..5e49151 100644 --- a/src/UiPath.Caching.Abstractions/SystemJsonByteSerializerProxy.cs +++ b/src/UiPath.Caching.Abstractions/SystemJsonByteSerializerProxy.cs @@ -2,42 +2,23 @@ namespace UiPath.Caching; -/// JSON serializer over byte[]: byte payloads pass through raw, everything else is UTF-8 JSON; the type argument decides, no format sniffing. +/// +/// The default serializer: UTF-8 JSON for every value, byte payloads included, which means base64 +/// inside a JSON string. That is the wire format the library has always written, so entries survive +/// an upgrade untouched. Use to store byte payloads verbatim. +/// public class SystemJsonByteSerializerProxy(JsonSerializerOptions? options = null) : ISerializerProxy { - public byte[]? Serialize(object? value) => value switch - { - null => null, - byte[] bytes => bytes, - ReadOnlyMemory memory => memory.ToArray(), - Memory memory => memory.ToArray(), - _ => JsonSerializer.SerializeToUtf8Bytes(value, options), - }; + /// + /// Null goes through JSON like everything else, producing the four-byte null literal. A + /// null payload would reach StackExchange.Redis as RedisValue.Null, which throws on SADD + /// and on the multi-field HSET, and silently stores nothing on the single-value paths. + /// + public virtual byte[]? Serialize(object? value) => + JsonSerializer.SerializeToUtf8Bytes(value, options); - public T? Deserialize(byte[]? value) - { - if (value is null) - { - return default; - } - if (typeof(T) == typeof(byte[])) - { - return (T)(object)value; - } - if (typeof(T) == typeof(ReadOnlyMemory)) - { - return (T)(object)new ReadOnlyMemory(value); - } - if (typeof(T) == typeof(Memory)) - { - return (T)(object)new Memory(value); - } - if (value.Length == 0) - { - return default; - } - return JsonSerializer.Deserialize(value, options); - } + public virtual T? Deserialize(byte[]? value) => + value is null or { Length: 0 } ? default : JsonSerializer.Deserialize(value, options); public bool TryDeserialize(string? value, out T? result) { diff --git a/src/UiPath.Caching.Azure/AzureEntraConnectionConfigurator.cs b/src/UiPath.Caching.Azure/AzureEntraConnectionConfigurator.cs index 3fe9500..8b83b98 100644 --- a/src/UiPath.Caching.Azure/AzureEntraConnectionConfigurator.cs +++ b/src/UiPath.Caching.Azure/AzureEntraConnectionConfigurator.cs @@ -1,3 +1,5 @@ +using StackExchange.Redis; + namespace UiPath.Caching.Azure; /// Configures a Redis connection to authenticate with Microsoft Entra ID. diff --git a/src/UiPath.Caching.Azure/GlobalUsings.cs b/src/UiPath.Caching.Azure/GlobalUsings.cs index 3037633..82671cf 100644 --- a/src/UiPath.Caching.Azure/GlobalUsings.cs +++ b/src/UiPath.Caching.Azure/GlobalUsings.cs @@ -5,7 +5,6 @@ global using Microsoft.Extensions.DependencyInjection; global using Microsoft.Extensions.DependencyInjection.Extensions; global using Microsoft.Extensions.Options; -global using StackExchange.Redis; global using UiPath.Caching.Config; global using UiPath.Caching.Redis; diff --git a/src/UiPath.Caching.Queue/ByteArrayEqualityComparer.cs b/src/UiPath.Caching.Queue/ByteArrayEqualityComparer.cs new file mode 100644 index 0000000..dda497e --- /dev/null +++ b/src/UiPath.Caching.Queue/ByteArrayEqualityComparer.cs @@ -0,0 +1,30 @@ +namespace UiPath.Caching; + +/// +/// keys its snapshot on serialized members, and byte[] compares +/// by reference. Without structural equality every lookup misses, and +/// reports that as an authoritative answer +/// rather than falling through to the backing tier. +/// +internal sealed class ByteArrayEqualityComparer : IEqualityComparer +{ + public static readonly ByteArrayEqualityComparer Instance = new(); + + private ByteArrayEqualityComparer() + { + } + + public bool Equals(byte[]? x, byte[]? y) => + ReferenceEquals(x, y) || (x is not null && y is not null && x.AsSpan().SequenceEqual(y)); + + public int GetHashCode(byte[]? obj) + { + if (obj is null) + { + return 0; + } + var hash = new HashCode(); + hash.AddBytes(obj); + return hash.ToHashCode(); + } +} diff --git a/src/UiPath.Caching.Queue/Config/QueueCacheCollectionExtensions.cs b/src/UiPath.Caching.Queue/Config/QueueCacheCollectionExtensions.cs index 6cbdf84..52adbe7 100644 --- a/src/UiPath.Caching.Queue/Config/QueueCacheCollectionExtensions.cs +++ b/src/UiPath.Caching.Queue/Config/QueueCacheCollectionExtensions.cs @@ -57,7 +57,7 @@ public static ICachingBuilder AddQueueRedis(this ICachingBuilder builder, string /// /// Prerequisite: the core caching and Redis services must already be registered (via /// AddCaching(... builder.AddRedis())). The provider resolves , - /// , , + /// , , /// , , the core cache options /// and from the container — the same set as the core /// RedisCacheProvider; if AddCaching() has not run, resolving diff --git a/src/UiPath.Caching.Queue/GlobalUsings.cs b/src/UiPath.Caching.Queue/GlobalUsings.cs index 3434b1f..740c0db 100644 --- a/src/UiPath.Caching.Queue/GlobalUsings.cs +++ b/src/UiPath.Caching.Queue/GlobalUsings.cs @@ -5,7 +5,6 @@ global using Microsoft.Extensions.Logging; global using Microsoft.Extensions.Logging.Abstractions; global using Microsoft.Extensions.Options; -global using StackExchange.Redis; global using UiPath.Caching.Policies; global using UiPath.Caching.Redis; global using UiPath.Caching.Telemetry; diff --git a/src/UiPath.Caching.Queue/InMemoryQueueCacheProvider.cs b/src/UiPath.Caching.Queue/InMemoryQueueCacheProvider.cs index 0efb06b..bf21b24 100644 --- a/src/UiPath.Caching.Queue/InMemoryQueueCacheProvider.cs +++ b/src/UiPath.Caching.Queue/InMemoryQueueCacheProvider.cs @@ -13,7 +13,7 @@ public sealed class InMemoryQueueCacheProvider : IQueueCacheProvider { private readonly InMemoryQueueCacheOptions _options; private readonly IMemoryCacheFactory _memoryCacheFactory; - private readonly ISerializerProxy _serializer; + private readonly ISerializerProxy _serializer; private readonly ILocalLock _localLock; private readonly Lazy _setCache; @@ -24,7 +24,7 @@ public sealed class InMemoryQueueCacheProvider : IQueueCacheProvider public InMemoryQueueCacheProvider( IOptions optionsAccessor, IMemoryCacheFactory memoryCacheFactory, - ISerializerProxy serializer, + ISerializerProxy serializer, ILocalLock localLock) { _options = optionsAccessor.Value; diff --git a/src/UiPath.Caching.Queue/InMemoryRedisQueueCacheProvider.cs b/src/UiPath.Caching.Queue/InMemoryRedisQueueCacheProvider.cs index b7ba577..fb948e3 100644 --- a/src/UiPath.Caching.Queue/InMemoryRedisQueueCacheProvider.cs +++ b/src/UiPath.Caching.Queue/InMemoryRedisQueueCacheProvider.cs @@ -15,7 +15,7 @@ public sealed class InMemoryRedisQueueCacheProvider : IQueueCacheProvider private readonly InMemoryRedisQueueCacheOptions _options; private readonly IMemoryCacheFactory _memoryCacheFactory; private readonly Lazy _queueCacheFactory; - private readonly ISerializerProxy _serializer; + private readonly ISerializerProxy _serializer; private readonly ILocalLock _localLock; private readonly Lazy _setCache; @@ -27,7 +27,7 @@ public InMemoryRedisQueueCacheProvider( IOptions optionsAccessor, IMemoryCacheFactory memoryCacheFactory, Func queueCacheFactoryAccessor, - ISerializerProxy serializer, + ISerializerProxy serializer, ILocalLock localLock) { _options = optionsAccessor.Value; diff --git a/src/UiPath.Caching.Queue/MemorySetCache.cs b/src/UiPath.Caching.Queue/MemorySetCache.cs index 0f3b380..1e1b4ec 100644 --- a/src/UiPath.Caching.Queue/MemorySetCache.cs +++ b/src/UiPath.Caching.Queue/MemorySetCache.cs @@ -6,14 +6,25 @@ namespace UiPath.Caching; internal sealed class MemorySetCache( string cacheName, IMemoryCache memoryCache, - ISerializerProxy serializer, + ISerializerProxy serializer, ILocalLock localLock, IMemoryCacheOptions memoryCacheOptions) { private readonly bool _trackSize = memoryCacheOptions.SizeLimit.HasValue; private readonly string _localLockKeyPrefix = cacheName + ":"; - private sealed record Snapshot(ImmutableHashSet Members, DateTimeOffset? Expiration); + private static readonly ImmutableHashSet EmptyMembers = + ImmutableHashSet.Create(ByteArrayEqualityComparer.Instance); + + private sealed record Snapshot(ImmutableHashSet Members, DateTimeOffset? Expiration); + + /// + /// A passthrough serializer such as hands back the caller's + /// own array. The snapshot hashes its members, so a caller mutating that array afterwards would + /// change an element's hash while it sits in the set. Only the paths that store need this; the + /// lookup paths may compare against the caller's array directly. + /// + private static byte[] Owned(byte[] value) => value.Length == 0 ? value : (byte[])value.Clone(); public bool TryGetMembers(string key, [NotNullWhen(true)] out IReadOnlyCollection? members) { @@ -44,7 +55,7 @@ public bool TryContainsItem(string key, T item, out bool contains) contains = false; return false; } - contains = snapshot.Members.Contains(serializer.Serialize(item)); + contains = snapshot.Members.Contains(serializer.Serialize(item)!); return true; } @@ -54,7 +65,7 @@ public bool TryContainsItem(string key, T item, out bool contains) public async ValueTask ReplaceAsync(string key, IEnumerable members, DateTimeOffset? expiration, CancellationToken token) { - var set = members.Select(m => serializer.Serialize(m)).ToImmutableHashSet(); + var set = members.Select(m => Owned(serializer.Serialize(m)!)).ToImmutableHashSet(ByteArrayEqualityComparer.Instance); using (await localLock.AcquireAsync(_localLockKeyPrefix + key, token).ConfigureAwait(false)) { StoreSnapshot(key, new Snapshot(set, expiration)); @@ -63,7 +74,7 @@ public async ValueTask ReplaceAsync(string key, IEnumerable members, DateT public async ValueTask AddAsync(string key, IEnumerable items, CancellationToken token) { - var values = items.Select(i => serializer.Serialize(i)).ToArray(); + var values = items.Select(i => Owned(serializer.Serialize(i)!)).ToArray(); if (values.Length == 0) { return; @@ -79,7 +90,7 @@ public async ValueTask AddAsync(string key, IEnumerable items, Cancellatio public async ValueTask AddAsync(string key, IEnumerable items, DateTimeOffset? expiration, CancellationToken token) { - var values = items.Select(i => serializer.Serialize(i)).ToArray(); + var values = items.Select(i => Owned(serializer.Serialize(i)!)).ToArray(); if (values.Length == 0) { return 0; @@ -91,7 +102,7 @@ public async ValueTask AddAsync(string key, IEnumerable items, DateT memoryCache.Remove(key); return 0; } - var members = TryGetSnapshot(key, out var snapshot) ? snapshot.Members : ImmutableHashSet.Empty; + var members = TryGetSnapshot(key, out var snapshot) ? snapshot.Members : EmptyMembers; var updated = members.Union(values); StoreSnapshot(key, new Snapshot(updated, expiration)); return updated.Count - members.Count; @@ -100,7 +111,7 @@ public async ValueTask AddAsync(string key, IEnumerable items, DateT public async ValueTask RemoveAsync(string key, IEnumerable items, CancellationToken token) { - var values = items.Select(i => serializer.Serialize(i)).ToArray(); + var values = items.Select(i => serializer.Serialize(i)!).ToArray(); if (values.Length == 0) { return 0; @@ -144,7 +155,7 @@ public async ValueTask RemoveKeyAsync(string key, CancellationToken token) } var pool = snapshot.Members.ToArray(); var take = (int)Math.Min(count, pool.Length); - var picked = new RedisValue[take]; + var picked = new byte[take][]; for (var i = 0; i < take; i++) { var j = Random.Shared.Next(i, pool.Length); @@ -178,7 +189,7 @@ private void StoreSnapshot(string key, Snapshot snapshot) memoryCache.Set(key, snapshot, options); } - private IReadOnlyCollection Deserialize(IReadOnlyCollection values) + private List Deserialize(IReadOnlyCollection values) { if (values.Count == 0) { @@ -187,7 +198,7 @@ private void StoreSnapshot(string key, Snapshot snapshot) var list = new List(values.Count); foreach (var value in values) { - if (value.IsNull) + if (value is null) { continue; } diff --git a/src/UiPath.Caching.Queue/MultilayerSetCache.cs b/src/UiPath.Caching.Queue/MultilayerSetCache.cs index 35355e7..3bd0085 100644 --- a/src/UiPath.Caching.Queue/MultilayerSetCache.cs +++ b/src/UiPath.Caching.Queue/MultilayerSetCache.cs @@ -18,7 +18,7 @@ public MultilayerSetCache( string name, ISetCache inner, IMemoryCacheFactory memoryCacheFactory, - ISerializerProxy serializer, + ISerializerProxy serializer, IMemoryCacheOptions memoryOptions, ILocalLock localLock, TimeSpan? localMaxExpiration, diff --git a/src/UiPath.Caching.Queue/PublicAPI.Unshipped.txt b/src/UiPath.Caching.Queue/PublicAPI.Unshipped.txt index 7dc5c58..874e42e 100644 --- a/src/UiPath.Caching.Queue/PublicAPI.Unshipped.txt +++ b/src/UiPath.Caching.Queue/PublicAPI.Unshipped.txt @@ -1 +1,9 @@ #nullable enable +UiPath.Caching.InMemoryQueueCacheProvider.InMemoryQueueCacheProvider(Microsoft.Extensions.Options.IOptions! optionsAccessor, UiPath.Caching.IMemoryCacheFactory! memoryCacheFactory, UiPath.Caching.ISerializerProxy! serializer, UiPath.Caching.Locking.ILocalLock! localLock) -> void +*REMOVED*UiPath.Caching.InMemoryQueueCacheProvider.InMemoryQueueCacheProvider(Microsoft.Extensions.Options.IOptions! optionsAccessor, UiPath.Caching.IMemoryCacheFactory! memoryCacheFactory, UiPath.Caching.ISerializerProxy! serializer, UiPath.Caching.Locking.ILocalLock! localLock) -> void +UiPath.Caching.InMemoryRedisQueueCacheProvider.InMemoryRedisQueueCacheProvider(Microsoft.Extensions.Options.IOptions! optionsAccessor, UiPath.Caching.IMemoryCacheFactory! memoryCacheFactory, System.Func! queueCacheFactoryAccessor, UiPath.Caching.ISerializerProxy! serializer, UiPath.Caching.Locking.ILocalLock! localLock) -> void +*REMOVED*UiPath.Caching.InMemoryRedisQueueCacheProvider.InMemoryRedisQueueCacheProvider(Microsoft.Extensions.Options.IOptions! optionsAccessor, UiPath.Caching.IMemoryCacheFactory! memoryCacheFactory, System.Func! queueCacheFactoryAccessor, UiPath.Caching.ISerializerProxy! serializer, UiPath.Caching.Locking.ILocalLock! localLock) -> void +UiPath.Caching.Redis.RedisSetCache.RedisSetCache(UiPath.Caching.Redis.IRedisConnector! redis, UiPath.Caching.ISerializerProxy! serializer, UiPath.Caching.Policies.IResiliencePipelineProvider! resiliencePipelineProvider, UiPath.Caching.Telemetry.ICachingTelemetryProvider! telemetryProvider, UiPath.Caching.Redis.RedisCacheOptions! redisCacheOptions, UiPath.Caching.CacheOptions! cacheOptions, UiPath.Caching.Redis.RedisSetCacheOptions! setCacheOptions, UiPath.Caching.ICachePolicyFactory! policyFactory, Microsoft.Extensions.Logging.ILogger! logger) -> void +*REMOVED*UiPath.Caching.Redis.RedisSetCache.RedisSetCache(UiPath.Caching.Redis.IRedisConnector! redis, UiPath.Caching.ISerializerProxy! serializer, UiPath.Caching.Policies.IResiliencePipelineProvider! resiliencePipelineProvider, UiPath.Caching.Telemetry.ICachingTelemetryProvider! telemetryProvider, UiPath.Caching.Redis.RedisCacheOptions! redisCacheOptions, UiPath.Caching.CacheOptions! cacheOptions, UiPath.Caching.Redis.RedisSetCacheOptions! setCacheOptions, UiPath.Caching.ICachePolicyFactory! policyFactory, Microsoft.Extensions.Logging.ILogger! logger) -> void +UiPath.Caching.RedisQueueCacheProvider.RedisQueueCacheProvider(Microsoft.Extensions.Options.IOptions! redisCacheOptions, Microsoft.Extensions.Options.IOptions! cacheOptions, Microsoft.Extensions.Options.IOptions! setCacheOptions, UiPath.Caching.Redis.IRedisConnector! redis, UiPath.Caching.ISerializerProxy! serializerProxy, UiPath.Caching.Policies.IResiliencePipelineProvider! resiliencePipelineProvider, UiPath.Caching.Telemetry.ICachingTelemetryProvider! cachingTelemetryProvider, Microsoft.Extensions.Logging.ILoggerFactory! loggerFactory, UiPath.Caching.ICachePolicyFactory! policyFactory) -> void +*REMOVED*UiPath.Caching.RedisQueueCacheProvider.RedisQueueCacheProvider(Microsoft.Extensions.Options.IOptions! redisCacheOptions, Microsoft.Extensions.Options.IOptions! cacheOptions, Microsoft.Extensions.Options.IOptions! setCacheOptions, UiPath.Caching.Redis.IRedisConnector! redis, UiPath.Caching.ISerializerProxy! serializerProxy, UiPath.Caching.Policies.IResiliencePipelineProvider! resiliencePipelineProvider, UiPath.Caching.Telemetry.ICachingTelemetryProvider! cachingTelemetryProvider, Microsoft.Extensions.Logging.ILoggerFactory! loggerFactory, UiPath.Caching.ICachePolicyFactory! policyFactory) -> void diff --git a/src/UiPath.Caching.Queue/RedisQueueCacheProvider.cs b/src/UiPath.Caching.Queue/RedisQueueCacheProvider.cs index 39a8798..295687e 100644 --- a/src/UiPath.Caching.Queue/RedisQueueCacheProvider.cs +++ b/src/UiPath.Caching.Queue/RedisQueueCacheProvider.cs @@ -10,7 +10,7 @@ public sealed class RedisQueueCacheProvider : IQueueCacheProvider private readonly CacheOptions _cacheOptions; private readonly RedisSetCacheOptions _setCacheOptions; private readonly IRedisConnector _redis; - private readonly ISerializerProxy _serializerProxy; + private readonly ISerializerProxy _serializerProxy; private readonly IResiliencePipelineProvider _resiliencePipelineProvider; private readonly ICachingTelemetryProvider _cachingTelemetryProvider; private readonly ILoggerFactory _loggerFactory; @@ -26,7 +26,7 @@ public RedisQueueCacheProvider( IOptions cacheOptions, IOptions setCacheOptions, IRedisConnector redis, - ISerializerProxy serializerProxy, + ISerializerProxy serializerProxy, IResiliencePipelineProvider resiliencePipelineProvider, ICachingTelemetryProvider cachingTelemetryProvider, ILoggerFactory loggerFactory, diff --git a/src/UiPath.Caching.Queue/RedisSetCache.cs b/src/UiPath.Caching.Queue/RedisSetCache.cs index 5689253..08c2744 100644 --- a/src/UiPath.Caching.Queue/RedisSetCache.cs +++ b/src/UiPath.Caching.Queue/RedisSetCache.cs @@ -1,4 +1,5 @@ using System.Runtime.CompilerServices; +using StackExchange.Redis; namespace UiPath.Caching.Redis; @@ -7,7 +8,7 @@ public sealed partial class RedisSetCache : RedisCacheBase, ISetCache private const string RedisSetKeyPrefix = "se"; private readonly ILogger _logger; - private readonly ISerializerProxy _serializer; + private readonly ISerializerProxy _serializer; private readonly IResiliencePipeline _read; private readonly IResiliencePipeline _write; private readonly IResiliencePipeline _pop; @@ -16,7 +17,7 @@ public sealed partial class RedisSetCache : RedisCacheBase, ISetCache public RedisSetCache( IRedisConnector redis, - ISerializerProxy serializer, + ISerializerProxy serializer, IResiliencePipelineProvider resiliencePipelineProvider, ICachingTelemetryProvider telemetryProvider, RedisCacheOptions redisCacheOptions, @@ -52,7 +53,7 @@ public ValueTask AddAsync(CacheKey cacheKey, IEnumerable items, Time { NotCacheableException.ThrowIfNotCacheable(); ArgumentNullException.ThrowIfNull(items); - var values = items.Select(i => _serializer.Serialize(i)).ToArray(); + var values = items.Select(i => (RedisValue)_serializer.Serialize(i)).ToArray(); return AddManyInnerAsync(cacheKey, values, Clock.ToDateTimeOffset(ResolveExpiration(expiration, policy)), token); } @@ -60,7 +61,7 @@ public ValueTask AddAsync(CacheKey cacheKey, IEnumerable items, Date { NotCacheableException.ThrowIfNotCacheable(); ArgumentNullException.ThrowIfNull(items); - var values = items.Select(i => _serializer.Serialize(i)).ToArray(); + var values = items.Select(i => (RedisValue)_serializer.Serialize(i)).ToArray(); return AddManyInnerAsync(cacheKey, values, ResolveExpiration(expiration, policy), token); } @@ -323,7 +324,7 @@ public async ValueTask RemoveItemsAsync(CacheKey cacheKey, IEnumerable< NotCacheableException.ThrowIfNotCacheable(); ArgumentNullException.ThrowIfNull(items); var redisKey = ToRedisKey(cacheKey, token); - var values = items.Select(i => _serializer.Serialize(i)).ToArray(); + var values = items.Select(i => (RedisValue)_serializer.Serialize(i)).ToArray(); long ret = 0; if (values.Length == 0) { diff --git a/src/UiPath.Caching/Config/CachingBuilder.cs b/src/UiPath.Caching/Config/CachingBuilder.cs index 198657c..e0bc451 100644 --- a/src/UiPath.Caching/Config/CachingBuilder.cs +++ b/src/UiPath.Caching/Config/CachingBuilder.cs @@ -31,7 +31,7 @@ internal void Complete() Services.TryAddEnumerable( ServiceDescriptor.Singleton, CacheKeyCasingSeeder>()); - Services.TryAddSingleton>(sp => new SystemJsonSerializerProxy(sp.GetService())); + ThrowIfLegacySerializerRegistered(); Services.TryAddSingleton>(sp => new SystemJsonByteSerializerProxy(sp.GetService())); Services.TryAddSingleton(EmptyResiliencePipelineProvider.Instance); Services.TryAddSingleton(NullChangeTokenFactory.Instance); @@ -52,6 +52,19 @@ internal void Complete() }); } + private void ThrowIfLegacySerializerRegistered() + { + if (Services.Any(d => d.ServiceType == typeof(ISerializerProxy))) + { + throw new InvalidOperationException( + $"A registration for ISerializerProxy is present, but that seam no longer exists and " + + $"nothing resolves it — the serializer would be silently ignored. Re-register the implementation " + + $"as ISerializerProxy (see docs/how-to/extending.md#custom-serializer). Note that " + + $"{nameof(SystemJsonByteSerializerProxy)} keeps the wire format unchanged, while " + + $"{nameof(RawByteSerializerProxy)} stores byte payloads verbatim."); + } + } + public void RegisterOnCompleteCallback(object key, Action callback) { ArgumentNullException.ThrowIfNull(key); diff --git a/src/UiPath.Caching/Config/DistributedCacheCollectionExtensions.cs b/src/UiPath.Caching/Config/DistributedCacheCollectionExtensions.cs index e88c631..b7fe482 100644 --- a/src/UiPath.Caching/Config/DistributedCacheCollectionExtensions.cs +++ b/src/UiPath.Caching/Config/DistributedCacheCollectionExtensions.cs @@ -1,3 +1,4 @@ +using System.Text.Json; using Microsoft.Extensions.Caching.Distributed; using UiPath.Caching.Distributed; using UiPath.Caching.Locking; @@ -310,7 +311,7 @@ private static RedisCacheProvider CreateRedisProvider( differentiator)), cacheOptions, connector, - new RedisValueSerializerProxy(sp.GetRequiredService>()), + new RawByteSerializerProxy(sp.GetService()), sp.GetRequiredService(), sp.GetRequiredService(), sp.GetRequiredService(), diff --git a/src/UiPath.Caching/Config/InMemoryRedisCollectionExtensions.cs b/src/UiPath.Caching/Config/InMemoryRedisCollectionExtensions.cs index db0972e..1f33048 100644 --- a/src/UiPath.Caching/Config/InMemoryRedisCollectionExtensions.cs +++ b/src/UiPath.Caching/Config/InMemoryRedisCollectionExtensions.cs @@ -42,7 +42,7 @@ private static ICachingBuilder AddCallback(this ICachingBuilder builder) { builder.RegisterOnCompleteCallback(typeof(InMemoryRedisCollectionExtensions), b => { - b.Services.TryAddSingleton>(); + b.Services.TryAddSingleton>(); b.Services.TryAddSingleton, CacheEventFormatter>(); b.Services.TryAddSingleton(); }); diff --git a/src/UiPath.Caching/Distributed/RedisValueSerializerProxy.cs b/src/UiPath.Caching/Distributed/RedisValueSerializerProxy.cs deleted file mode 100644 index 8835798..0000000 --- a/src/UiPath.Caching/Distributed/RedisValueSerializerProxy.cs +++ /dev/null @@ -1,17 +0,0 @@ -namespace UiPath.Caching.Distributed; - -/// Adapts an ISerializerProxy<byte[]> onto the Redis pipeline for the distributed cache's dedicated instance. -internal sealed class RedisValueSerializerProxy(ISerializerProxy inner) : ISerializerProxy -{ - public RedisValue Serialize(object? value) => - inner.Serialize(value); - - public T? Deserialize(RedisValue value) => - value.IsNullOrEmpty ? default : inner.Deserialize(value); - - public bool TryDeserialize(string? value, out T? result) => - inner.TryDeserialize(value, out result); - - public bool TryDeserialize(object? value, out T? result) => - inner.TryDeserialize(value, out result); -} diff --git a/src/UiPath.Caching/PublicAPI.Unshipped.txt b/src/UiPath.Caching/PublicAPI.Unshipped.txt index 00628a2..c0c2ebc 100644 --- a/src/UiPath.Caching/PublicAPI.Unshipped.txt +++ b/src/UiPath.Caching/PublicAPI.Unshipped.txt @@ -18,3 +18,11 @@ UiPath.Caching.Redis.RedisCacheOptions.AwaitRefresh.get -> bool UiPath.Caching.Redis.RedisCacheOptions.AwaitRefresh.set -> void const UiPath.Caching.Config.DistributedCacheCollectionExtensions.DistributedCacheServiceKey = "UiPath.Caching.Distributed" -> string! static UiPath.Caching.Config.DistributedCacheCollectionExtensions.AddDistributedCache(this UiPath.Caching.Config.ICachingBuilder! builder, string! providerName, System.Action? configure = null) -> UiPath.Caching.Config.ICachingBuilder! +UiPath.Caching.Redis.RedisCacheProvider.RedisCacheProvider(Microsoft.Extensions.Options.IOptions! redisCacheOptions, Microsoft.Extensions.Options.IOptions! cacheOptions, UiPath.Caching.Redis.IRedisConnector! redis, UiPath.Caching.ISerializerProxy! serializerProxy, UiPath.Caching.Policies.IResiliencePipelineProvider! resiliencePipelineProvider, UiPath.Caching.Telemetry.ICachingTelemetryProvider! cachingTelemetryProvider, Microsoft.Extensions.Logging.ILoggerFactory! loggerFactory, UiPath.Caching.ICachePolicyFactory! policyFactory) -> void +*REMOVED*UiPath.Caching.Redis.RedisCacheProvider.RedisCacheProvider(Microsoft.Extensions.Options.IOptions! redisCacheOptions, Microsoft.Extensions.Options.IOptions! cacheOptions, UiPath.Caching.Redis.IRedisConnector! redis, UiPath.Caching.ISerializerProxy! serializerProxy, UiPath.Caching.Policies.IResiliencePipelineProvider! resiliencePipelineProvider, UiPath.Caching.Telemetry.ICachingTelemetryProvider! cachingTelemetryProvider, Microsoft.Extensions.Logging.ILoggerFactory! loggerFactory, UiPath.Caching.ICachePolicyFactory! policyFactory) -> void +*REMOVED*UiPath.Caching.SystemJsonSerializerProxy +*REMOVED*UiPath.Caching.SystemJsonSerializerProxy.Deserialize(StackExchange.Redis.RedisValue value) -> T? +*REMOVED*UiPath.Caching.SystemJsonSerializerProxy.Serialize(object? value) -> StackExchange.Redis.RedisValue +*REMOVED*UiPath.Caching.SystemJsonSerializerProxy.SystemJsonSerializerProxy(System.Text.Json.JsonSerializerOptions? options = null) -> void +*REMOVED*UiPath.Caching.SystemJsonSerializerProxy.TryDeserialize(object? value, out T? result) -> bool +*REMOVED*UiPath.Caching.SystemJsonSerializerProxy.TryDeserialize(string? value, out T? result) -> bool diff --git a/src/UiPath.Caching/Redis/RedisCache.cs b/src/UiPath.Caching/Redis/RedisCache.cs index bc7ebce..4222372 100644 --- a/src/UiPath.Caching/Redis/RedisCache.cs +++ b/src/UiPath.Caching/Redis/RedisCache.cs @@ -6,7 +6,7 @@ namespace UiPath.Caching.Redis; internal sealed partial class RedisCache : RedisCacheBase, ICache { - private readonly ISerializerProxy _serializer; + private readonly ISerializerProxy _serializer; private readonly ILogger _logger; private readonly bool _supportsExpireTime; private readonly IResiliencePipeline _read; @@ -19,7 +19,7 @@ internal sealed partial class RedisCache : RedisCacheBase, ICache public RedisCache( IRedisConnector redis, - ISerializerProxy serializer, + ISerializerProxy serializer, IResiliencePipelineProvider resiliencePipelineProvider, ICachingTelemetryProvider telemetryProvider, RedisCacheOptions redisCacheOptions, @@ -505,7 +505,9 @@ private async ValueTask TryAddInternalAsync(RedisKey redisKey, T? value } else { - var serialized = isNull ? RedisValue.EmptyString : _serializer.Serialize(value); + // Both arms have to land on RedisValue: the serializer now yields byte[], which + // converts implicitly, but not in a var-typed conditional against EmptyString. + RedisValue serialized = isNull ? RedisValue.EmptyString : _serializer.Serialize(value); ret = await _write.ExecuteAsync(async token => { diff --git a/src/UiPath.Caching/Redis/RedisCacheProvider.cs b/src/UiPath.Caching/Redis/RedisCacheProvider.cs index 9c7666e..4b295c1 100644 --- a/src/UiPath.Caching/Redis/RedisCacheProvider.cs +++ b/src/UiPath.Caching/Redis/RedisCacheProvider.cs @@ -8,7 +8,7 @@ public sealed class RedisCacheProvider : ICacheProvider private readonly RedisCacheOptions _redisCacheOptions; private readonly CacheOptions _cacheOptions; private readonly IRedisConnector _redis; - private readonly ISerializerProxy _serializerProxy; + private readonly ISerializerProxy _serializerProxy; private readonly IResiliencePipelineProvider _resiliencePipelineProvider; private readonly ICachingTelemetryProvider _cachingTelemetryProvider; private readonly ILoggerFactory _loggerFactory; @@ -24,7 +24,7 @@ public RedisCacheProvider( IOptions redisCacheOptions, IOptions cacheOptions, IRedisConnector redis, - ISerializerProxy serializerProxy, + ISerializerProxy serializerProxy, IResiliencePipelineProvider resiliencePipelineProvider, ICachingTelemetryProvider cachingTelemetryProvider, ILoggerFactory loggerFactory, diff --git a/src/UiPath.Caching/Redis/RedisHashCache.cs b/src/UiPath.Caching/Redis/RedisHashCache.cs index 0eeaa81..206c412 100644 --- a/src/UiPath.Caching/Redis/RedisHashCache.cs +++ b/src/UiPath.Caching/Redis/RedisHashCache.cs @@ -8,7 +8,7 @@ namespace UiPath.Caching.Redis; internal sealed partial class RedisHashCache : RedisCacheBase, IHashCache { private readonly ILogger _logger; - private readonly ISerializerProxy _serializer; + private readonly ISerializerProxy _serializer; private readonly ICacheEntryFactory _cacheEntryFactory; private readonly IResiliencePipeline _read; private readonly IResiliencePipeline _write; @@ -20,7 +20,7 @@ internal sealed partial class RedisHashCache : RedisCacheBase, IHashCache public RedisHashCache( IRedisConnector redis, - ISerializerProxy serializer, + ISerializerProxy serializer, IResiliencePipelineProvider resiliencePipelineProvider, ICachingTelemetryProvider telemetryProvider, RedisCacheOptions redisCacheOptions, @@ -186,7 +186,7 @@ public RedisHashCache( private ValueTask SetEmptyMarkerAsync(CacheKey cacheKey, HashCacheEntryOptions options, CancellationToken token) { var redisKey = ToRedisKey(cacheKey, token); - var metadata = options.Metadata != null && options.Metadata.Count > 0 + RedisValue metadata = options.Metadata != null && options.Metadata.Count > 0 ? _serializer.Serialize(options.Metadata) : RedisValue.EmptyString; var entries = new[] { new HashEntry(KnownFieldNames.MetadataKey, metadata) }; diff --git a/src/UiPath.Caching/SystemJsonSerializerProxy.cs b/src/UiPath.Caching/SystemJsonSerializerProxy.cs deleted file mode 100644 index 0d93ed2..0000000 --- a/src/UiPath.Caching/SystemJsonSerializerProxy.cs +++ /dev/null @@ -1,71 +0,0 @@ -using System.Text.Json; - -namespace UiPath.Caching; - -public class SystemJsonSerializerProxy : ISerializerProxy -{ - private readonly JsonSerializerOptions? _options; - - public SystemJsonSerializerProxy(JsonSerializerOptions? options = null) => - _options = options; - - public RedisValue Serialize(object? value) => - JsonSerializer.SerializeToUtf8Bytes(value, _options); - - public T? Deserialize(RedisValue value) - { - if (value.IsNullOrEmpty) - { - return default; - } - - ReadOnlyMemory payload = value; - return JsonSerializer.Deserialize(payload.Span, _options); - } - - public bool TryDeserialize(string? value, out T? result) - { - if (string.IsNullOrWhiteSpace(value)) - { - result = default; - return false; - } - try - { - result = JsonSerializer.Deserialize(value, _options); - return true; - } - catch - { - result = default; - return false; - } - } - - public bool TryDeserialize(object? value, out T? result) - { - if (value == null) - { - result = default; - return false; - } - try - { - if(value is JsonElement jsonElement) - { - result = jsonElement.Deserialize(_options); - return true; - } - else - { - var text = value.ToString() ?? string.Empty; - return TryDeserialize(text, out result); - } - } - catch - { - result = default; - return false; - } - } -} diff --git a/tests/UiPath.Caching.Tests/Broadcast/ChangeTokenTests.cs b/tests/UiPath.Caching.Tests/Broadcast/ChangeTokenTests.cs index 241294a..44846e1 100644 --- a/tests/UiPath.Caching.Tests/Broadcast/ChangeTokenTests.cs +++ b/tests/UiPath.Caching.Tests/Broadcast/ChangeTokenTests.cs @@ -14,11 +14,11 @@ public class ChangeTokenTests : IAsyncLifetime private CacheClearEventFormatterProxy _formatter = default!; private Uri? _source = null; private ISet? _acceptedEvents = null; - private ISerializerProxy _serializer = default!; + private SystemJsonByteSerializerProxy _serializer = default!; private readonly RecordingTelemetryProvider _telemetryProvider = new(); - private ChangeToken? _sut = null; - private ChangeToken Sut => _sut ??= new ChangeToken(_key, _topic, _source, _serializer, _fixture.Freeze>>(), _telemetryProvider, _acceptedEvents); + private ChangeToken? _sut = null; + private ChangeToken Sut => _sut ??= new ChangeToken(_key, _topic, _source, _serializer, _fixture.Freeze>>(), _telemetryProvider, _acceptedEvents); [Fact] public void Verify_ActiveChangeCallbacks() @@ -232,10 +232,10 @@ public ValueTask InitializeAsync() _key = _fixture.Freeze(); _topicKey = (TopicKey)_fixture.Create(); _fixture.Inject(_topicKey); - _fixture.Freeze>>(); + _fixture.Freeze>>(); _topic = _fixture.Freeze>(); _formatter = new CacheClearEventFormatterProxy(); - _serializer = new SystemJsonSerializerProxy(); + _serializer = new SystemJsonByteSerializerProxy(); _fixture.Inject>(_formatter); return ValueTask.CompletedTask; } diff --git a/tests/UiPath.Caching.Tests/CachingBuilderTests.cs b/tests/UiPath.Caching.Tests/CachingBuilderTests.cs index 4ef3076..3992584 100644 --- a/tests/UiPath.Caching.Tests/CachingBuilderTests.cs +++ b/tests/UiPath.Caching.Tests/CachingBuilderTests.cs @@ -8,6 +8,32 @@ namespace UiPath.Caching.Tests; public class CachingBuilderTests { + /// Nothing resolves that service now, so it would be ignored rather than fail. + [Fact] + public void A_leftover_RedisValue_serializer_registration_fails_the_build() + { + var services = new ServiceCollection(); + services.AddSingleton(Substitute.For>()); + + var act = () => new CachingBuilder(services).Complete(); + + act.Should().Throw() + .WithMessage("*ISerializerProxy*") + .WithMessage("*ISerializerProxy*"); + } + + [Fact] + public void A_byte_serializer_registration_is_honored_over_the_default() + { + var services = new ServiceCollection(); + var custom = Substitute.For>(); + services.AddSingleton(custom); + + new CachingBuilder(services).Complete(); + + services.BuildServiceProvider().GetRequiredService>().Should().BeSameAs(custom); + } + [Fact] public void Two_builders_with_same_key_each_fire_their_own_callback() { @@ -64,12 +90,12 @@ public void Two_builders_with_full_pipeline_each_resolve_real_services() using var providerB = BuildContainer(); providerA.GetRequiredService() - .Should().BeOfType>(); + .Should().BeOfType>(); providerA.GetRequiredService() .Get(ResiliencePipelineNames.Read).Should().BeOfType(); providerB.GetRequiredService() - .Should().BeOfType>(); + .Should().BeOfType>(); providerB.GetRequiredService() .Get(ResiliencePipelineNames.Read).Should().BeOfType(); } diff --git a/tests/UiPath.Caching.Tests/Distributed/RedisValueSerializerProxyTests.cs b/tests/UiPath.Caching.Tests/Distributed/RedisValueSerializerProxyTests.cs deleted file mode 100644 index d4afae9..0000000 --- a/tests/UiPath.Caching.Tests/Distributed/RedisValueSerializerProxyTests.cs +++ /dev/null @@ -1,44 +0,0 @@ -using StackExchange.Redis; -using UiPath.Caching.Distributed; - -namespace UiPath.Caching.Tests.Distributed; - -public class RedisValueSerializerProxyTests -{ - private sealed record Poco(string Name); - - private readonly RedisValueSerializerProxy _proxy = new(new SystemJsonByteSerializerProxy()); - - [Fact] - public void Bytes_round_trip_unencoded() - { - var payload = new byte[] { 0x00, 0x01, 0xFF }; - RedisValue stored = _proxy.Serialize(payload); - ((byte[])stored!).Should().Equal(payload); - _proxy.Deserialize(stored).Should().Equal(payload); - } - - [Fact] - public void Null_and_empty_map_to_defaults() - { - _proxy.Serialize(null).IsNull.Should().BeTrue(); - _proxy.Deserialize(RedisValue.Null).Should().BeNull(); - _proxy.Deserialize(RedisValue.EmptyString).Should().BeNull(); - } - - [Fact] - public void Poco_round_trips_via_inner_json() - { - var value = new Poco("x"); - var stored = _proxy.Serialize(value); - _proxy.Deserialize(stored).Should().Be(value); - } - - [Fact] - public void TryDeserialize_delegates_to_inner() - { - _proxy.TryDeserialize("""{"Name":"x"}""", out var ok).Should().BeTrue(); - ok.Should().Be(new Poco("x")); - _proxy.TryDeserialize("not json", out _).Should().BeFalse(); - } -} diff --git a/tests/UiPath.Caching.Tests/InMemoryRedisCollectionExtensionsTests.cs b/tests/UiPath.Caching.Tests/InMemoryRedisCollectionExtensionsTests.cs index 4961c3a..9e7c62d 100644 --- a/tests/UiPath.Caching.Tests/InMemoryRedisCollectionExtensionsTests.cs +++ b/tests/UiPath.Caching.Tests/InMemoryRedisCollectionExtensionsTests.cs @@ -38,9 +38,9 @@ public void Multiple_AddInMemoryRedis_in_same_process_each_register_real_change_ using var providerB = BuildContainer(); providerA.GetRequiredService() - .Should().BeOfType>(); + .Should().BeOfType>(); providerB.GetRequiredService() - .Should().BeOfType>(); + .Should().BeOfType>(); } [Fact] diff --git a/tests/UiPath.Caching.Tests/InMemorySetCacheTests.cs b/tests/UiPath.Caching.Tests/InMemorySetCacheTests.cs index 1467d71..47e6336 100644 --- a/tests/UiPath.Caching.Tests/InMemorySetCacheTests.cs +++ b/tests/UiPath.Caching.Tests/InMemorySetCacheTests.cs @@ -15,7 +15,7 @@ private static MultilayerSetCache CreateSut(InMemoryQueueCacheOptions? options = return new MultilayerSetCache( KnownCacheProviderNames.InMemory, NullSetCache.Instance, new MemoryCacheFactory(null, NullLoggerFactory.Instance), - new SystemJsonSerializerProxy(), options, + new SystemJsonByteSerializerProxy(), options, NullLocalLock.Instance, localMaxExpiration: null, defaultExpiration: options.DefaultExpiration); @@ -29,6 +29,50 @@ private static ValueTask AddMany(MultilayerSetCache sut, CacheKey key, par [Fact] public void Name_is_InMemory() => CreateSut().Name.Should().Be("InMemory"); + private sealed record Member(int Id, string Name); + + /// + /// The snapshot is keyed on the serializer's byte[] output, which compares by reference. + /// A populated local tier is authoritative, so without structural equality the wrong answer is + /// never corrected against the backing tier. + /// + /// + /// With a passthrough serializer the snapshot would otherwise hold the caller's own array, so + /// mutating it after the add would change an element's hash from inside the set. + /// + [Fact] + public async Task A_member_mutated_after_being_added_does_not_corrupt_the_snapshot() + { + var sut = new MultilayerSetCache( + KnownCacheProviderNames.InMemory, NullSetCache.Instance, + new MemoryCacheFactory(null, NullLoggerFactory.Instance), + new RawByteSerializerProxy(), new InMemoryQueueCacheOptions(), + NullLocalLock.Instance, localMaxExpiration: null, defaultExpiration: null); + var payload = new byte[] { 1, 2, 3 }; + + (await sut.AddAsync("k", payload, (CachePolicy?)null, Ct)).Should().BeTrue(); + payload[0] = 9; + + (await sut.ContainsItemAsync("k", new byte[] { 1, 2, 3 }, Ct)).Should().BeTrue(); + (await sut.CountAsync("k", Ct)).Should().Be(1); + } + + [Fact] + public async Task Members_are_matched_by_their_serialized_bytes_not_by_reference() + { + var sut = CreateSut(); + var stored = new Member(7, "héllo 世界"); + var equalButDistinctInstance = new Member(7, "héllo 世界"); + + (await sut.AddAsync("k", stored, (CachePolicy?)null, Ct)).Should().BeTrue(); + + (await sut.ContainsItemAsync("k", equalButDistinctInstance, Ct)).Should().BeTrue(); + (await sut.AddAsync("k", equalButDistinctInstance, (CachePolicy?)null, Ct)).Should().BeFalse(); + (await sut.CountAsync("k", Ct)).Should().Be(1); + (await sut.RemoveItemAsync("k", equalButDistinctInstance, Ct)).Should().BeTrue(); + (await sut.CountAsync("k", Ct)).Should().Be(0); + } + [Fact] public async Task Add_single_deduplicates() { diff --git a/tests/UiPath.Caching.Tests/LegacySerializerWireCompatTests.cs b/tests/UiPath.Caching.Tests/LegacySerializerWireCompatTests.cs new file mode 100644 index 0000000..e984953 --- /dev/null +++ b/tests/UiPath.Caching.Tests/LegacySerializerWireCompatTests.cs @@ -0,0 +1,106 @@ +using System.Text.Json; +using StackExchange.Redis; + +namespace UiPath.Caching.Tests; + +/// +/// Entries written by 1.x's SystemJsonSerializerProxy must read back unchanged through the +/// default. This is what makes the seam change source-only rather than a data migration. +/// +public class LegacySerializerWireCompatTests +{ + private sealed record Sample(int Id, string Name, int[] Values); + + private readonly SystemJsonByteSerializerProxy _proxy = new(); + + /// The 1.x proxy's own output: JsonSerializer.SerializeToUtf8Bytes. + private static byte[] WrittenByLegacyProxy(object? value) => + JsonSerializer.SerializeToUtf8Bytes(value); + + /// A custom 1.x serializer returning a string-backed RedisValue, as the docs blessed. + private static byte[] WrittenByLegacyStringPath(object? value) => + ((byte[]?)(RedisValue)JsonSerializer.Serialize(value))!; + + [Fact] + public void Poco_written_by_the_legacy_proxy_still_reads() + { + var original = new Sample(42, "héllo 世界", [1, 2, 3]); + + _proxy.Deserialize(WrittenByLegacyProxy(original)).Should().BeEquivalentTo(original); + } + + [Fact] + public void Poco_written_through_the_legacy_string_path_still_reads() + { + var original = new Sample(7, "legacy", [9]); + + _proxy.Deserialize(WrittenByLegacyStringPath(original)).Should().BeEquivalentTo(original); + } + + [Fact] + public void Both_serializers_produce_identical_bytes_for_a_poco() + { + var value = new Sample(1, "x", []); + + _proxy.Serialize(value).Should().Equal(WrittenByLegacyProxy(value)); + } + + [Theory] + [InlineData("plain string")] + [InlineData(42)] + [InlineData(true)] + [InlineData(1.5)] + public void Scalars_are_byte_identical_across_both_serializers(object value) + { + _proxy.Serialize(value).Should().Equal(WrittenByLegacyProxy(value)); + } + + [Fact] + public void Null_is_byte_identical_to_the_legacy_proxy_and_reads_back_as_default() + { + _proxy.Serialize(null).Should().Equal(WrittenByLegacyProxy(null)).And.Equal("null"u8.ToArray()); + + _proxy.Deserialize(WrittenByLegacyProxy(null)).Should().BeNull(); + } + + [Fact] + public void Null_and_empty_read_back_as_default() + { + _proxy.Deserialize(null).Should().BeNull(); + _proxy.Deserialize([]).Should().BeNull(); + } + + [Fact] + public void Byte_payloads_written_by_the_legacy_proxy_still_round_trip() + { + byte[] original = [1, 2, 3]; + var legacy = WrittenByLegacyProxy(original); + + legacy.Should().Equal("\"AQID\""u8.ToArray(), "1.x base64-encoded byte[] inside JSON"); + + _proxy.Deserialize(legacy).Should().Equal(original); + _proxy.Serialize(original).Should().Equal(legacy); + } + + /// Why raw passthrough is not the default: no throw, so nothing degrades it to a miss. + [Fact] + public void The_raw_proxy_would_misread_a_legacy_byte_payload_silently() + { + byte[] original = [1, 2, 3]; + var legacy = WrittenByLegacyProxy(original); + + new RawByteSerializerProxy().Deserialize(legacy) + .Should().Equal(legacy).And.NotEqual(original); + } + + /// The same mismatch on a POCO does throw, which the caches catch and report as a miss. + [Fact] + public void A_payload_that_is_not_json_surfaces_as_an_exception_for_typed_reads() + { + byte[] raw = [0xFF, 0xFE, 0xFD]; + + var act = () => _proxy.Deserialize(raw); + + act.Should().Throw(); + } +} diff --git a/tests/UiPath.Caching.Tests/MultilayerSetCacheTests.cs b/tests/UiPath.Caching.Tests/MultilayerSetCacheTests.cs index 58f5fd8..95157b6 100644 --- a/tests/UiPath.Caching.Tests/MultilayerSetCacheTests.cs +++ b/tests/UiPath.Caching.Tests/MultilayerSetCacheTests.cs @@ -16,7 +16,7 @@ private static (MultilayerSetCache Sut, ISetCache L2) CreateSut() var l2 = Substitute.For(); var sut = new MultilayerSetCache( KnownCacheProviderNames.InMemoryRedis, l2, - MemoryFactory(), new SystemJsonSerializerProxy(), new InMemoryRedisQueueCacheOptions(), + MemoryFactory(), new SystemJsonByteSerializerProxy(), new InMemoryRedisQueueCacheOptions(), NullLocalLock.Instance, localMaxExpiration: TimeSpan.FromMinutes(5)); return (sut, l2); @@ -175,7 +175,7 @@ private static (MultilayerSetCache Sut, ISetCache L2) CreateMonitoredSut(bool co ((IConnectionState)l2).IsConnected.Returns(connected); var sut = new MultilayerSetCache( KnownCacheProviderNames.InMemoryRedis, l2, - MemoryFactory(), new SystemJsonSerializerProxy(), new InMemoryRedisQueueCacheOptions(), + MemoryFactory(), new SystemJsonByteSerializerProxy(), new InMemoryRedisQueueCacheOptions(), NullLocalLock.Instance, localMaxExpiration: TimeSpan.FromMinutes(5), connectionMonitorEnabled: true, diff --git a/tests/UiPath.Caching.Tests/RawByteSerializerProxyTests.cs b/tests/UiPath.Caching.Tests/RawByteSerializerProxyTests.cs new file mode 100644 index 0000000..3f89923 --- /dev/null +++ b/tests/UiPath.Caching.Tests/RawByteSerializerProxyTests.cs @@ -0,0 +1,147 @@ +using System.Text; +using System.Text.Json; + +namespace UiPath.Caching.Tests; + +public class RawByteSerializerProxyTests +{ + private readonly RawByteSerializerProxy _proxy = new(); + + private sealed record Poco(string Name, int Count); + + [Fact] + public void Byte_array_passes_through_by_reference() + { + var payload = new byte[] { 0x00, 0x01, 0xFF }; + _proxy.Serialize(payload).Should().BeSameAs(payload); + _proxy.Deserialize(payload).Should().BeSameAs(payload); + } + + [Fact] + public void ReadOnlyMemory_is_materialized_not_json_encoded() + { + ReadOnlyMemory memory = new byte[] { 1, 2, 3 }; + _proxy.Serialize(memory).Should().Equal(1, 2, 3); + } + + [Fact] + public void ReadOnlyMemory_round_trips() + { + ReadOnlyMemory memory = new byte[] { 1, 2, 3 }; + var stored = _proxy.Serialize(memory); + _proxy.Deserialize>(stored).ToArray().Should().Equal(1, 2, 3); + } + + [Fact] + public void Empty_byte_array_round_trips() + { + var empty = Array.Empty(); + _proxy.Serialize(empty).Should().BeSameAs(empty); + _proxy.Deserialize(empty).Should().BeSameAs(empty); + } + + [Fact] + public void Null_falls_through_to_json_and_empty_reads_as_default() + { + _proxy.Serialize(null).Should().Equal("null"u8.ToArray()); + _proxy.Deserialize(null).Should().BeNull(); + _proxy.Deserialize([]).Should().BeNull(); + } + + [Fact] + public void Poco_round_trips_as_utf8_json() + { + var value = new Poco("x", 42); + var bytes = _proxy.Serialize(value)!; + JsonSerializer.Deserialize(bytes).Should().Be(value); + _proxy.Deserialize(bytes).Should().Be(value); + } + + [Fact] + public void Deserializing_json_as_bytes_returns_raw_utf8() + { + var bytes = _proxy.Serialize(new Poco("x", 1))!; + _proxy.Deserialize(bytes).Should().BeSameAs(bytes); + } + + [Fact] + public void TryDeserialize_string_success_and_failure() + { + _proxy.TryDeserialize("""{"Name":"x","Count":1}""", out var ok).Should().BeTrue(); + ok.Should().Be(new Poco("x", 1)); + _proxy.TryDeserialize("not json", out var bad).Should().BeFalse(); + bad.Should().BeNull(); + _proxy.TryDeserialize(" ", out _).Should().BeFalse(); + } + + [Fact] + public void TryDeserialize_object_handles_bytes_json_element_and_text() + { + var raw = Encoding.UTF8.GetBytes("""{"Name":"x","Count":1}"""); + _proxy.TryDeserialize(raw, out var bytes).Should().BeTrue(); + bytes.Should().BeSameAs(raw); + + var element = JsonSerializer.SerializeToElement(new Poco("x", 1)); + _proxy.TryDeserialize(element, out var fromElement).Should().BeTrue(); + fromElement.Should().Be(new Poco("x", 1)); + + _proxy.TryDeserialize((object)"""{"Name":"x","Count":1}""", out var fromText).Should().BeTrue(); + fromText.Should().Be(new Poco("x", 1)); + + _proxy.TryDeserialize(null, out _).Should().BeFalse(); + } + + [Fact] + public void Honors_custom_serializer_options() + { + var proxy = new RawByteSerializerProxy(new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }); + Encoding.UTF8.GetString(proxy.Serialize(new Poco("x", 1))!).Should().Contain("\"name\""); + } + + /// byte[] is an object, so a naive "value is T" passthrough would hand back the raw bytes. + [Fact] + public void Deserializing_as_object_parses_json_instead_of_returning_bytes() + { + var bytes = _proxy.Serialize(new Poco("x", 1))!; + + _proxy.Deserialize(bytes).Should().NotBeOfType(); + } + + [Fact] + public void TryDeserialize_object_round_trips_its_own_output() + { + var value = new Poco("x", 1); + var bytes = _proxy.Serialize(value)!; + + _proxy.TryDeserialize((object)bytes, out var result).Should().BeTrue(); + result.Should().Be(value); + } + + /// A valid JSON null is a successful deserialization on every input shape; only an empty buffer is a failure. + [Fact] + public void TryDeserialize_object_treats_json_null_as_success_on_every_path() + { + var jsonNull = Encoding.UTF8.GetBytes("null"); + + _proxy.TryDeserialize((object)jsonNull, out var fromBytes).Should().BeTrue(); + fromBytes.Should().BeNull(); + + _proxy.TryDeserialize((object)"null", out var fromText).Should().BeTrue(); + fromText.Should().BeNull(); + + _proxy.TryDeserialize(JsonSerializer.SerializeToElement(null), out var fromElement).Should().BeTrue(); + fromElement.Should().BeNull(); + + _proxy.TryDeserialize((object)Array.Empty(), out _).Should().BeFalse(); + } + + [Fact] + public void Memory_of_byte_round_trips() + { + Memory memory = new byte[] { 4, 5, 6 }; + var stored = _proxy.Serialize(memory)!; + + stored.Should().Equal(4, 5, 6); + _proxy.Deserialize>(stored).ToArray().Should().Equal(4, 5, 6); + } +} diff --git a/tests/UiPath.Caching.Tests/Redis/RedisCacheTests.cs b/tests/UiPath.Caching.Tests/Redis/RedisCacheTests.cs index bdacd9c..5a96d7c 100644 --- a/tests/UiPath.Caching.Tests/Redis/RedisCacheTests.cs +++ b/tests/UiPath.Caching.Tests/Redis/RedisCacheTests.cs @@ -17,7 +17,7 @@ public class RedisCacheTests(ITestContextAccessor testContextAccessor) : IAsyncL private RedisCacheOptions _cacheOptions = default!; private IDatabase _database = default!; private ITransaction _transaction = default!; - private ISerializerProxy _serializer = default!; + private SystemJsonByteSerializerProxy _serializer = default!; private DateTimeOffset _now = DateTimeOffset.UtcNow; private CacheKey _cacheKey = default!; private RedisKey _redisKey = default!; @@ -1328,8 +1328,8 @@ public ValueTask InitializeAsync() _database = _fixture.Freeze(); _transaction = _fixture.Freeze(); _database.CreateTransaction().Returns(_transaction); - _serializer = new SystemJsonSerializerProxy(); - _fixture.Inject(_serializer); + _serializer = new SystemJsonByteSerializerProxy(); + _fixture.Inject>(_serializer); var opt = Options.Create(_cacheOptions); _fixture.Inject(opt); _fixture.Inject(_cacheOptions); diff --git a/tests/UiPath.Caching.Tests/Redis/RedisCacheTryAddTests.cs b/tests/UiPath.Caching.Tests/Redis/RedisCacheTryAddTests.cs index e7777c5..b07f451 100644 --- a/tests/UiPath.Caching.Tests/Redis/RedisCacheTryAddTests.cs +++ b/tests/UiPath.Caching.Tests/Redis/RedisCacheTryAddTests.cs @@ -19,7 +19,7 @@ public class RedisCacheTryAddTests(ITestContextAccessor testContextAccessor) : I private ISystemClock _clock = default!; private RedisCacheOptions _cacheOptions = default!; private IDatabase _database = default!; - private SystemJsonSerializerProxy _serializer = default!; + private SystemJsonByteSerializerProxy _serializer = default!; private readonly DateTimeOffset _now = DateTimeOffset.UtcNow; private CacheKey _cacheKey = default!; private RedisKey _redisKey = default!; @@ -276,8 +276,8 @@ public ValueTask InitializeAsync() }; _database = _fixture.Freeze(); - _serializer = new SystemJsonSerializerProxy(); - _fixture.Inject>(_serializer); + _serializer = new SystemJsonByteSerializerProxy(); + _fixture.Inject>(_serializer); _fixture.Inject(Options.Create(_cacheOptions)); _fixture.Inject(_cacheOptions); _fixture.Inject(_telemetry); diff --git a/tests/UiPath.Caching.Tests/Redis/RedisHashCacheTests.cs b/tests/UiPath.Caching.Tests/Redis/RedisHashCacheTests.cs index f462ad5..2a3c527 100644 --- a/tests/UiPath.Caching.Tests/Redis/RedisHashCacheTests.cs +++ b/tests/UiPath.Caching.Tests/Redis/RedisHashCacheTests.cs @@ -7,7 +7,7 @@ using UiPath.Caching.Policies; using UiPath.Caching.Telemetry; using UiPath.Caching.Tests.Telemetry; -using JsonSerializer = UiPath.Caching.SystemJsonSerializerProxy; +using JsonSerializer = UiPath.Caching.SystemJsonByteSerializerProxy; namespace UiPath.Caching.Tests.Redis; @@ -19,7 +19,7 @@ public class RedisHashCacheTests(ITestContextAccessor testContextAccessor) : IAs private string _prefix = default!; private IDatabase _database = default!; private ITransaction _transaction = default!; - private ISerializerProxy _serializer = default!; + private JsonSerializer _serializer = default!; private ISystemClock _clock = default!; private RedisCacheOptions _redisCacheOptions = new(); private DateTimeOffset _now = DateTimeOffset.UtcNow; @@ -1427,7 +1427,7 @@ public ValueTask InitializeAsync() RedisKeyStrategyFactory = redisKeyStrategyFactory }; _serializer = new JsonSerializer(); - _fixture.Inject(_serializer); + _fixture.Inject>(_serializer); _fixture.Inject(_telemetry); var opt = Options.Create(_redisCacheOptions); _fixture.Inject(opt); diff --git a/tests/UiPath.Caching.Tests/Redis/RedisSetCacheTests.cs b/tests/UiPath.Caching.Tests/Redis/RedisSetCacheTests.cs index 96b73e7..eaf87cc 100644 --- a/tests/UiPath.Caching.Tests/Redis/RedisSetCacheTests.cs +++ b/tests/UiPath.Caching.Tests/Redis/RedisSetCacheTests.cs @@ -13,7 +13,7 @@ public class RedisSetCacheTests(ITestContextAccessor testContextAccessor) : IAsy private string _prefix = default!; private IDatabase _database = default!; private ITransaction _transaction = default!; - private ISerializerProxy _serializer = default!; + private SystemJsonByteSerializerProxy _serializer = default!; private ISystemClock _clock = default!; private const string PopResilienceKeyName = "set-pop"; private RedisCacheOptions _redisCacheOptions = new(); @@ -196,7 +196,7 @@ public async Task Pop_single_redis_exception_returns_default() public async Task Pop_count_works() { var expected = _fixture.CreateMany().ToArray(); - var serialized = expected.Select(e => _serializer.Serialize(e)).ToArray(); + var serialized = expected.Select(e => (RedisValue)_serializer.Serialize(e)).ToArray(); _database.SetPopAsync(_redisKey, expected.Length, CommandFlags.DemandMaster).Returns(serialized); var actual = await Sut.PopAsync(_cacheKey, expected.Length, policy: null, token: testContextAccessor.Current.CancellationToken); @@ -274,7 +274,7 @@ public async Task Pop_count_uses_configured_pop_pipeline() _pipelineProvider.Get(PopResilienceKeyName).Returns(pop); _pipelineProvider.Get(ResiliencePipelineNames.Write).Returns(write); var expected = _fixture.CreateMany().ToArray(); - var serialized = expected.Select(e => _serializer.Serialize(e)).ToArray(); + var serialized = expected.Select(e => (RedisValue)_serializer.Serialize(e)).ToArray(); _database.SetPopAsync(_redisKey, expected.Length, CommandFlags.DemandMaster).Returns(serialized); var actual = await Sut.PopAsync(_cacheKey, expected.Length, policy: null, token: testContextAccessor.Current.CancellationToken); @@ -305,7 +305,7 @@ public async Task RemoveItem_uses_write_pipeline_not_pop() public async Task Members_works() { var expected = _fixture.CreateMany().ToArray(); - var serialized = expected.Select(e => _serializer.Serialize(e)).ToArray(); + var serialized = expected.Select(e => (RedisValue)_serializer.Serialize(e)).ToArray(); _database.SetMembersAsync(_redisKey, CommandFlags.PreferReplica).Returns(serialized); var actual = await Sut.MembersAsync(_cacheKey, policy: null, token: testContextAccessor.Current.CancellationToken); @@ -550,8 +550,8 @@ public ValueTask InitializeAsync() CacheKeyStrategy = _cacheKeyStrategy, RedisKeyStrategyFactory = _redisKeyStrategyFactory }; - _serializer = new SystemJsonSerializerProxy(); - _fixture.Inject(_serializer); + _serializer = new SystemJsonByteSerializerProxy(); + _fixture.Inject>(_serializer); var opt = Options.Create(_redisCacheOptions); _fixture.Inject(opt); _fixture.Inject(opt.Value); diff --git a/tests/UiPath.Caching.Tests/SetCacheProviderTests.cs b/tests/UiPath.Caching.Tests/SetCacheProviderTests.cs index 82badaf..e498035 100644 --- a/tests/UiPath.Caching.Tests/SetCacheProviderTests.cs +++ b/tests/UiPath.Caching.Tests/SetCacheProviderTests.cs @@ -8,7 +8,7 @@ public class InMemoryQueueCacheProviderTests private static InMemoryQueueCacheProvider CreateSut(InMemoryQueueCacheOptions? options = null) => new(Options.Create(options ?? new InMemoryQueueCacheOptions()), new MemoryCacheFactory(null, NullLoggerFactory.Instance), - new SystemJsonSerializerProxy(), + new SystemJsonByteSerializerProxy(), NullLocalLock.Instance); [Fact] @@ -72,7 +72,7 @@ public async Task Resolves_its_L2_from_the_factory_Redis_provider() Options.Create(new InMemoryRedisQueueCacheOptions()), new MemoryCacheFactory(null, NullLoggerFactory.Instance), () => factory, - new SystemJsonSerializerProxy(), + new SystemJsonByteSerializerProxy(), NullLocalLock.Instance); var cache = provider.CreateSetCache(); diff --git a/tests/UiPath.Caching.Tests/SystemJsonByteSerializerProxyTests.cs b/tests/UiPath.Caching.Tests/SystemJsonByteSerializerProxyTests.cs index e0a2ee0..310c97b 100644 --- a/tests/UiPath.Caching.Tests/SystemJsonByteSerializerProxyTests.cs +++ b/tests/UiPath.Caching.Tests/SystemJsonByteSerializerProxyTests.cs @@ -10,58 +10,82 @@ public class SystemJsonByteSerializerProxyTests private sealed record Poco(string Name, int Count); [Fact] - public void Byte_array_passes_through_by_reference() + public void Byte_array_is_base64_encoded_inside_json() { - var payload = new byte[] { 0x00, 0x01, 0xFF }; - _proxy.Serialize(payload).Should().BeSameAs(payload); - _proxy.Deserialize(payload).Should().BeSameAs(payload); + var payload = new byte[] { 0x01, 0x02, 0x03 }; + + var stored = _proxy.Serialize(payload)!; + + Encoding.UTF8.GetString(stored).Should().Be("\"AQID\""); + stored.Should().NotBeSameAs(payload); } [Fact] - public void ReadOnlyMemory_is_materialized_not_json_encoded() + public void Byte_array_round_trips_through_base64() { - ReadOnlyMemory memory = new byte[] { 1, 2, 3 }; - _proxy.Serialize(memory).Should().Equal(1, 2, 3); + var payload = new byte[] { 0x00, 0x01, 0xFF }; + + _proxy.Deserialize(_proxy.Serialize(payload)).Should().Equal(payload); } [Fact] public void ReadOnlyMemory_round_trips() { ReadOnlyMemory memory = new byte[] { 1, 2, 3 }; + var stored = _proxy.Serialize(memory); + _proxy.Deserialize>(stored).ToArray().Should().Equal(1, 2, 3); } [Fact] public void Empty_byte_array_round_trips() { - var empty = Array.Empty(); - _proxy.Serialize(empty).Should().BeSameAs(empty); - _proxy.Deserialize(empty).Should().BeSameAs(empty); + var stored = _proxy.Serialize(Array.Empty())!; + + Encoding.UTF8.GetString(stored).Should().Be("\"\""); + _proxy.Deserialize(stored).Should().BeEmpty(); + } + + /// + /// Must stay non-null: a null payload reaches StackExchange.Redis as RedisValue.Null, + /// which throws on SADD and on the multi-field HSET, and stores nothing on the single-value paths. + /// + [Fact] + public void Null_serializes_to_the_json_null_literal_never_to_a_null_payload() + { + var stored = _proxy.Serialize(null); + + stored.Should().NotBeNull(); + Encoding.UTF8.GetString(stored!).Should().Be("null"); } [Fact] - public void Null_maps_to_null_and_empty_to_default() + public void Null_empty_and_the_json_null_literal_all_read_back_as_default() { - _proxy.Serialize(null).Should().BeNull(); _proxy.Deserialize(null).Should().BeNull(); _proxy.Deserialize([]).Should().BeNull(); + _proxy.Deserialize(Encoding.UTF8.GetBytes("null")).Should().BeNull(); } [Fact] public void Poco_round_trips_as_utf8_json() { var value = new Poco("x", 42); + var bytes = _proxy.Serialize(value)!; + JsonSerializer.Deserialize(bytes).Should().Be(value); _proxy.Deserialize(bytes).Should().Be(value); } + /// Unlike , reading JSON as bytes parses it rather than passing it through. [Fact] - public void Deserializing_json_as_bytes_returns_raw_utf8() + public void Deserializing_json_as_bytes_decodes_rather_than_passing_through() { - var bytes = _proxy.Serialize(new Poco("x", 1))!; - _proxy.Deserialize(bytes).Should().BeSameAs(bytes); + var stored = _proxy.Serialize(new byte[] { 7, 8 })!; + + _proxy.Deserialize(stored).Should().Equal(7, 8).And.NotEqual(stored); } [Fact] @@ -78,8 +102,8 @@ public void TryDeserialize_string_success_and_failure() public void TryDeserialize_object_handles_bytes_json_element_and_text() { var raw = Encoding.UTF8.GetBytes("""{"Name":"x","Count":1}"""); - _proxy.TryDeserialize(raw, out var bytes).Should().BeTrue(); - bytes.Should().BeSameAs(raw); + _proxy.TryDeserialize(raw, out var fromBytes).Should().BeTrue(); + fromBytes.Should().Be(new Poco("x", 1)); var element = JsonSerializer.SerializeToElement(new Poco("x", 1)); _proxy.TryDeserialize(element, out var fromElement).Should().BeTrue(); @@ -95,10 +119,10 @@ public void TryDeserialize_object_handles_bytes_json_element_and_text() public void Honors_custom_serializer_options() { var proxy = new SystemJsonByteSerializerProxy(new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }); + Encoding.UTF8.GetString(proxy.Serialize(new Poco("x", 1))!).Should().Contain("\"name\""); } - /// byte[] is an object, so a naive "value is T" passthrough would hand back the raw bytes. [Fact] public void Deserializing_as_object_parses_json_instead_of_returning_bytes() { @@ -139,9 +163,9 @@ public void TryDeserialize_object_treats_json_null_as_success_on_every_path() public void Memory_of_byte_round_trips() { Memory memory = new byte[] { 4, 5, 6 }; + var stored = _proxy.Serialize(memory)!; - stored.Should().Equal(4, 5, 6); _proxy.Deserialize>(stored).ToArray().Should().Equal(4, 5, 6); } } diff --git a/tests/UiPath.Caching.Tests/SystemJsonSerializerProxyTests.cs b/tests/UiPath.Caching.Tests/SystemJsonSerializerProxyTests.cs deleted file mode 100644 index 9ae0967..0000000 --- a/tests/UiPath.Caching.Tests/SystemJsonSerializerProxyTests.cs +++ /dev/null @@ -1,44 +0,0 @@ -using System.Text.Json; -using StackExchange.Redis; - -namespace UiPath.Caching.Tests; - -public class SystemJsonSerializerProxyTests -{ - private sealed record Sample(int Id, string Name, int[] Values); - - private readonly SystemJsonSerializerProxy _proxy = new(); - - [Fact] - public void RoundTrips_complex_object() - { - var original = new Sample(42, "héllo 世界", [1, 2, 3]); - - var result = _proxy.Deserialize(_proxy.Serialize(original)); - - result.Should().BeEquivalentTo(original); - } - - [Fact] - public void Serialize_produces_utf8_json_on_the_wire() - { - var value = new Sample(1, "x", []); - - ((byte[])_proxy.Serialize(value)!).Should().Equal(JsonSerializer.SerializeToUtf8Bytes(value)); - } - - [Fact] - public void Deserializes_value_written_by_the_legacy_string_path() - { - RedisValue legacy = JsonSerializer.Serialize(new Sample(7, "legacy", [9])); - - _proxy.Deserialize(legacy).Should().BeEquivalentTo(new Sample(7, "legacy", [9])); - } - - [Fact] - public void Null_and_empty_return_default() - { - _proxy.Deserialize(RedisValue.Null).Should().BeNull(); - _proxy.Deserialize(RedisValue.EmptyString).Should().BeNull(); - } -}