fix(renderer): connect CDP over every resolved address, not just the first

[us]

ref #564

What changed

• crates/crw-renderer/src/cdp_conn.rs: CdpConnection::connect no longer resolves the ws_url host and rewrites the URL to the first resolved address. It builds the handshake request and overrides the Host header instead.
• The hostname stays in the URI, which is what TcpStream::connect resolves, so it walks every address the resolver returns instead of the one we pinned.
• The override is narrow: plaintext ws:// only, and only when the host is not already an IP literal. wss:// is untouched, so a hosted CDP endpoint keeps its real Host, SNI and certificate name. An IPv6 literal's brackets are stripped before the literal check, so ws://[::1]:9222 keeps its own Host.
• 12 tests, including one that binds a WebSocket listener on whichever loopback family the resolver does not return first and connects through ws://localhost, asserting the Host header the server receives.

Why

ws://localhost:9223 with a browser bound to 127.0.0.1 only hard-failed with CDP connect failed: io error whenever the resolver answered ::1 first, and every JS render silently fell back to the HTTP tier. HTTP discovery against the same host in the same request succeeded, because reqwest walks the address list.

Chromium 148+ validates the DevTools WebSocket Host header against a DNS-rebinding allowlist: only localhost and IP literals pass. The earlier workaround for that resolved the host and rewrote the URL, which is what threw the address list away. The header was always the target; rewriting the URL was the side effect.

Measured with raw WebSocket handshakes against a headless Chrome 153 bound to 127.0.0.1:9223:

Host sent	response
localhost:9223	HTTP/1.1 101
localhost	HTTP/1.1 101
127.0.0.1:9223	HTTP/1.1 101
[::1]:9223	HTTP/1.1 101
chrome:9223	HTTP/1.1 500
localtest.me:9223	HTTP/1.1 500

Verified

Live POST /v1/scrape against a running server, same IPv4-only Chrome, https://quotes.toscrape.com/js/:

configured ws_url	before	after
ws://localhost:9223/	renderedWith: http, js_escalation_failed	renderedWith: chrome, JS content, no warning
ws://localtest.me:9223/	renderedWith: chrome	renderedWith: chrome, JS content, no warning
ws://127.0.0.1:9223/	renderedWith: chrome	renderedWith: chrome, JS content, no warning
ws://localhost:9221/ (lightpanda)	renderedWith: http, CDP connect failed: io error	renderedWith: lightpanda, JS content
ws://localhost:9224/chromium?token=... (browserless)	renderedWith: http, CDP connect failed: io error	renderedWith: chrome, JS content

localtest.me is a real hostname resolving to 127.0.0.1, so that row is the docker ws://chrome:9222 shape. Chromium answers Host: localtest.me:9223 with a 500, so it only works because the header is forced, which keeps the Chromium 148 guard fix intact.

Run against a real lightpanda serve --host 127.0.0.1 --port 9221 (v0.27.1) and a real ghcr.io/browserless/chromium:v2.27.0 published on 127.0.0.1:9224:3000, the same bind docker-compose.yml uses. So Chromium, LightPanda and browserless all accept the forced Host, measured rather than assumed.

cargo test -p crw-renderer --features cdp: 977 passed, 0 failed. cargo fmt --check and cargo clippy --features cdp --all-targets -D warnings clean.

Effect for self-hosters

ws://localhost:PORT works as a renderer ws_url again when the browser listens on IPv4 only. The ws://127.0.0.1:PORT literal is no longer needed as a workaround.

This also unbreaks a shape we ship ourselves: config.stealth.toml:47 and :59 point at ws://localhost:9222/ and ws://localhost:9224/chromium?token=..., and docker-compose.yml publishes chrome-stealth as 127.0.0.1:9224:3000, an IPv4-only bind. On a host whose resolver answers ::1 first, that overlay hit the same failure.

Production impact

Read off the running production engine, not inferred:

tier	configured ws_url	effect
chrome	ws://172.30.40.31:9222/	none, IP literal takes no override
chrome_proxy	ws://172.30.40.32:9222/	none, IP literal takes no override
lightpanda	ws://lightpanda-cdp:9222/	Host becomes localhost:9222

Both Chromium tiers are byte-identical: an IP literal skips the override, so the request matches what connect_async builds on its own, header for header, which a test locks. Only the lightpanda tier changes on the wire. Production pins lightpanda/browser@sha256:15213bb6...; that exact image, started with production's own command line, answers 101 to every Host tried, including localhost:9222, and a full render through it over a non-localhost hostname succeeds.

Deploy caveats

No config change, no migration, no new dependency.

One behaviour change worth knowing: DNS resolution now happens inside the 10s CDP connect timeout instead of before it. A box whose first nameserver is dead could previously connect at around 11s and would now time out; in exchange, a wedged resolver can no longer hang the connect without a bound, which it could before.