Skip to content

fix(core): gate h2 extended CONNECT on backend support and send GOAWAY on upstream failure - #3814

Open
helix-nine wants to merge 2 commits into
masterfrom
fix/h2-proxy-extended-connect-goaway
Open

fix(core): gate h2 extended CONNECT on backend support and send GOAWAY on upstream failure#3814
helix-nine wants to merge 2 commits into
masterfrom
fix/h2-proxy-extended-connect-goaway

Conversation

@helix-nine

Copy link
Copy Markdown
Contributor

Closes #3775.

What

Two fixes to run_http2_proxy (shared-libs/crates/start-core/src/net/http.rs), both pre-existing defects that become reachable for every protocol: 'https' | 'wss' binding once the client-facing ALPN is reflected (#3777).

1. Extended CONNECT is now advertised only when the backend supports it

The client-facing h2 server used to set SETTINGS_ENABLE_CONNECT_PROTOCOL unconditionally. A browser opening wss://… then uses an RFC 8441 extended CONNECT, whose :protocol we forward verbatim — against a backend without RFC 8441 support (nginx) the stream is reset and the WebSocket dies. Now the advertisement mirrors the backend.

How the decision is made: hyper's SendRequest/Connection::is_extended_connect_protocol_enabled() turns out to be unusable here for two reasons, both measured:

  • The peer's SETTINGS have not arrived when handshake().await resolves (h2's client handshake only writes our preface + SETTINGS), so it reads false.
  • With a spawning executor (TokioExecutor) the connection runs on its own spawned task; manually polling the returned Connection registers a waker nothing ever signals, so a "wait until the setting flips" loop hangs to its timeout — and absence of the param would mean waiting out that timeout on every proxied connection to an nginx-class backend. Unacceptable latency.

Instead the proxy reads the backend's opening SETTINGS frame directly off the wire before handing the stream to hyper — RFC 9113 requires it to be the first frame — decides from its entries, and replays the consumed bytes into the real handshake via a small AsyncRead wrapper (Replay). Cost on conformant backends: one round trip, zero added delay either way. If no settings arrive within 5s the connection is broken anyway; we proceed without the advertisement.

2. An upstream error no longer aborts the client connection

res? in the select loop returned early, so graceful_shutdown() ran only when the upstream ended cleanly; any error — the common "container restarted" case — dropped the browser's leg with no GOAWAY, aborting every in-flight stream. This was the exact symptom #3731 fixed for HTTP/1; the h2 path never got the counterpart. Upstream errors are now logged and answered with graceful_shutdown(), so in-flight streams drain behind GOAWAY. A handshake that fails outright (backend died before completing it) serves the client just long enough to deliver GOAWAY too.

Tests

  • extended_connect_is_advertised_only_when_the_backend_advertises_it — parses the proxy's client-facing SETTINGS frames against backends with and without the param.
  • an_upstream_error_reaches_the_client_as_goaway_not_a_bare_abort — a backend that answers its handshake and dies mid-connection reaches the browser as GOAWAY.

All 12 net::http tests pass, full start-core lib suite green (329 passed, export_ skipped per run-tests.sh).

…Y on upstream failure

Two defects in run_http2_proxy (#3775), both reachable for every
protocol: 'https' | 'wss' binding once the client-facing ALPN is
reflected (#3777):

- The client-facing h2 server advertised SETTINGS_ENABLE_CONNECT_PROTOCOL
  unconditionally, so a browser opening wss:// as an RFC 8441 extended
  CONNECT had its :protocol forwarded to backends that reject it (nginx),
  killing the WebSocket. The backend's opening SETTINGS is now read off
  the wire before the handshake (RFC 9113 requires it first) and replayed
  into it; the advertisement mirrors what was found. hyper's
  is_extended_connect_protocol_enabled() can't be used for this: with a
  spawning executor nothing ever wakes a manual poll of the Connection,
  and absence of the param would mean waiting out a timeout on every
  connection.

- An upstream error returned early, dropping the client leg without
  GOAWAY and aborting every in-flight stream — the symptom #3731 fixed
  for HTTP/1, which the h2 path never got. Upstream errors are now logged
  and answered with graceful_shutdown like a clean end, and a handshake
  that fails outright serves the client just long enough to deliver
  GOAWAY.
Review round 1 on #3814:

- A sniff that timed out or errored mid-frame dropped the bytes already
  consumed and handed hyper an empty replay, corrupting the stream; any
  incomplete opening SETTINGS now goes to goaway_only like a failed
  handshake instead.
- goaway_only's stub answered a request that slipped in during the drain
  with a fabricated empty 200; it now answers 502.
- Replay::poll_read returned Ready(Ok(())) with zero bytes filled when
  polled with a zero-capacity buffer, which reads as EOF; empty-buffer
  polls delegate to the inner stream.
- take_initial_settings builds its replay buffer in one allocation and
  shares parse_settings_entries with the tests, which also stop
  redefining the module's frame constants.
- Changelog: split the combined entry into one bullet per behavior.
@helix-nine

Copy link
Copy Markdown
Contributor Author

Review round 1 found six things worth acting on; all fixed and pushed in d9df0a6:

  • Sniff failure no longer corrupts the backend stream — if the opening-SETTINGS read timed out or errored mid-frame, the bytes already consumed were dropped and hyper got an empty replay. Any incomplete sniff now goes to goaway_only like a failed handshake.
  • goaway_only's stub answers 502 instead of a fabricated empty 200 for a request that slips in during the drain window.
  • Replay::poll_read contract edge — a zero-capacity buffer poll returned Ready(Ok(())) with zero bytes filled (reads as EOF); empty-buffer polls now delegate straight to the inner stream.
  • Allocation churntake_initial_settings builds its replay buffer in one allocation instead of copying the frame head three times, and shares parse_settings_entries with the tests (which also stop redefining the module's frame constants).
  • Changelog entry split into one bullet per observable behavior, matching house style.

Not acted on, for the record: the vectored-write downgrade through Replay (is_write_vectored() == false makes h2 coalesce less aggressively on the upstream leg — correctness is unaffected, cost is a few extra syscalls per flush); the unbounded post-GOAWAY drain against a client that never closes (intentional, mirrors the HTTP/1 path from #3731, and the vhost caller's cancel tokens bound it); the 5s worst-case sniff latency on a silent backend (bounded, and strictly better than the pre-existing behavior of hanging until the ~300s keepalive timeout). Full start-core lib suite green after the changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

vhost: run_http2_proxy advertises extended CONNECT without checking the backend, and skips GOAWAY on an upstream error

1 participant