feat: add zstd support for http ingress and upstream connectors

[mxssl]

What

nodecore only ever spoke gzip on either edge. This adds zstd alongside it: the client-facing ingress negotiates a coding per request, and the upstream connectors offer zstd to nodes and decode whatever those nodes answer with. gzip and identity behave exactly as before.

zstd is both denser and much faster to decode than gzip, and node providers increasingly serve it. Against a live upstream that supports it, a 574KB eth_getBlockByNumber response goes out to the client as 114KB of zstd versus 137KB of gzip — 17% smaller, at a lower CPU cost per byte.

internal/compression (new)

One package shared by both edges, so the codec pools exist once rather than per edge:

• Negotiate(acceptEncoding) — RFC 9110 §12.5.3 q-value parsing. Highest q wins, zstd breaks a tie, q=0 is a refusal, anything unrecognised falls back to identity.
• Offer — the zstd, gzip nodecore advertises upstream.
• WrapReader / AcquireWriter — pooled encoders and decoders. Levels are pinned at the fastest setting of each codec, since a proxy pays compression on the critical path of every request, where CPU costs more than the last few percent of ratio.

Two limits worth calling out: encoders run at concurrency 1 (the library default spawns GOMAXPROCS goroutines per encoder, which at proxy concurrency is a goroutine count nobody asked for), and decoders cap the zstd window at 64MB (the library default is 64GB, so a hostile frame header can otherwise make you allocate on demand).

Ingress

compress.go is generalized over codecs. It is re-ported from echo's current middleware rather than patched in place, because our copy predates a fix worth having: the status line is now held until the first body byte, so an empty response no longer ships a stale Content-Encoding. Echo's MinLength buffering is deliberately left out, and the Level/Skipper config went with the pool.

decompress.go replaces middleware.Decompress(), which only ever understood gzip. It decodes gzip and zstd request bodies, passes unknown codings through untouched, and answers 400 for a body that isn't the coding it claims to be. To keep that symmetric between codings, WrapReader checks the zstd frame magic eagerly — gzip already validated its header on reset, so zstd failing lazily mid-read would have been the odd one out.

Upstream

Go's transport negotiates gzip on its own, but gzip is the only coding it knows, and it does so only while no Accept-Encoding is set. Offering zstd means setting the header explicitly, which turns that machinery off — so the connector now owns decoding for both codings.

• applyConfigHeaders sends Accept-Encoding: zstd, gzip, unless connector config pins one — an operator who pinned it has a reason, most likely a node that mishandles a coding.
• dispatch decodes before anything reads the body, so the buffered and streaming paths both see plain bytes. decodedBody ties the pooled codec's lifetime to the response body so neither path has to remember there are two things to close.
• A node answering with a coding we never offered is a partial failure rather than garbage handed to the client — the connector strips Content-Encoding, so those bytes would arrive unlabelled and unreadable.

Two adjacent fixes

Content-Encoding on decompressed requests. It was left on the request after decoding and forwarded upstream, telling a node to decompress a body that is already plain. The middleware now deletes it.

CORS clobbering Vary. setCorsHeaders did Set("Vary", "Origin"), wiping the Vary: Accept-Encoding the compression middleware had just added. With two codings negotiable that is exactly how a shared cache ends up handing a zstd body to a gzip-only client. Changed to Add.

Compatibility

No new config. Clients and upstreams that don't mention zstd see byte-identical behavior. One existing test changed contract: TestRestRequest_HopByHopClientHeadersNotForwarded asserted the upstream sees no Accept-Encoding, which is now the connector's own offer — the client's value still cannot survive, and the test says so explicitly.

gRPC and WebSocket are untouched: gRPC has its own compressor registry where zstd isn't a registered encoding, and WebSocket uses permessage-deflate. Neither speaks Content-Encoding.

Testing

~20 new cases across 4 test files, written test-first. Full suite green (go test exit 0 over 93 packages), -race clean on the three touched packages, go vet clean.

Verified against a running binary, not just tests:

• Live upstream that serves zstd: 63.8KB on the wire decoded to 264KB; client-side negotiation returned zstd for zstd and for gzip, zstd, gzip for gzip, plain for no header, and every variant decoded to the same block.
• Local node under load: 1500 requests at 48-way concurrency across five coding combinations — 0 failures, 0 errors logged, every response body validated. This was the real test of decoder pooling.
• The node saw Accept-Encoding: zstd, gzip and received an 85-byte plain, unlabelled body even when the client sent a compressed one.
• Operator pin honored end to end: gzip and identity reached the node verbatim and both decoded correctly, with the client still getting zstd.
• WebSocket upgrade still returns 101 through the rewritten response writer, with no Content-Encoding on it.