Add opt-in small-blob read coalescing to GrpcStore

[erneestoc]

Summary

GrpcStore opens one ByteStream Read stream per blob regardless of size. Measured against a real in-process gRPC server, each stream carries ~1.6 ms of fixed cost (stream setup, resource-name parse, server dispatch) — for the small blobs that dominate action input trees (headers, sources, small protos), that fixed cost is 30x the transfer itself. Worker cold-start materialization of a ~4400-file tree issues thousands of these.

This adds opt-in coalescing of small-blob reads into BatchReadBlobs RPCs, configured per GrpcStore:

"grpc": {
  "endpoints": [ ... ],
  "store_type": "cas",
  "experimental_read_batching": {
    "max_blob_size_bytes": 131072,  // only blobs <= this are batched
    "max_batch_bytes": 3145728,     // payload cap per BatchReadBlobs request
    "dispatch_slots": 4,            // max concurrent batch RPCs
    "max_queued_bytes": 33554432,   // queue bound; overflow bypasses to streams
  },
}

Default off: with the field unset, behavior is byte-for-byte unchanged (verified by a test asserting BatchReadBlobs is never called).

Mechanism: slot-based group commit (no timers)

Eligible get_part calls (digest key, full read from offset 0, size <= threshold) enqueue and try to acquire one of dispatch_slots permits. A permit holder drains the queue into <= max_batch_bytes requests and demuxes responses to per-caller oneshots; ops arriving while all slots are busy accumulate and depart together when a slot frees.

• Work-conserving: an op never waits while a slot is free — batch size self-tunes to load, zero added latency when idle, no batching-window timer to tune.
• Per-item failure isolation: each response entry maps to its own caller; one NOT_FOUND does not fail its batch-mates; whole-RPC failures ride the existing retrier before being reported.
• Bounded memory: queued bytes are capped; overflow degrades gracefully to the regular stream path rather than blocking.
• Bypass: partial reads, large blobs, and non-digest keys use the existing stream path untouched; empty-blob short-circuit ordering is preserved. digest_function is captured at enqueue time from the caller's context, never in the dispatcher.

Measurements

End-to-end through this feature (get_part path), real ByteStreamServer/CasServer over TCP, 64-wide caller concurrency:

blob size	batching off	batching on	speedup
4 KiB x 2048	3218 ms (1571 us/blob)	411 ms (200 us/blob)	7.8x
32 KiB x 512	1019 ms	266 ms	3.8x
256 KiB x 128	729 ms	737 ms	1.0x (above threshold; bypasses by design)

Raw RPC-shape ceiling measured separately (batch vs stream, no coalescer): 30.2x at 4 KiB — the remaining gap is queue/demux overhead. On a production iOS RBE deployment, worker->CAS reads at ~ms-scale fixed cost per blob dominate cold input materialization; this targets exactly that.

Caveat on absolute local timings: these were captured under a test harness
whose capturing TRACE subscriber adds measurable overhead, so treat them as
relative (same-harness A/B). The per-stream fixed cost and the RPC-shape
ceiling are harness-independent; production validation is below.

Scope and companion changes

This PR addresses the wire-side per-blob fixed cost (one stream per
blob), which matters most on real-RTT networks. The disk-side per-blob
fixed cost on macOS workers (a full device-cache flush per uploaded blob,
F_FULLFSYNC) is addressed independently by the FilesystemStore flush
coalescing / sync_policy changes; the three are orthogonal and compose.
Production 4-build sweep with this flag + subtree caching on macOS workers:
per-exec setup time -67% avg / -72% P95 cold (true-cold, all caches wiped),
-48% avg incremental, queue time -89%, zero errors, warm path byte-identical.

Testing

• New grpc_read_batching_test.rs: fake CAS + ByteStream servers over real TCP recording which RPCs are used. Covers: contents correct for 200 coalesced reads (and ByteStream not used for them); above-threshold blobs use streams; partial reads bypass; per-item NOT_FOUND isolation within a shared batch (deterministic same-batch grouping via dispatch_slots: 1); config-unset never calls BatchReadBlobs.
• dispatch_slots == 0 rejected at construction (would strand queued reads).
• bazel test //nativelink-store:integration //nativelink-config:unit_test green (clippy + rustfmt aspects); cargo check --tests and cargo fmt --check clean.
• Also validated merged together with the subtree-caching (Add opt-in subtree-keyed caching to the worker DirectoryCache #2541) and
  macOS flush-coalescing (Coalesce FilesystemStore durability flushes on macOS #2539) PRs: zero conflicts, all 40 bazel tests
  across config/store/worker green on the combined branch.

---

This change is 

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
nativelink	Ready	Preview, Comment	Jul 9, 2026 5:20pm
nativelink-aidm	Ready	Preview, Comment	Jul 9, 2026 5:20pm