Skip to content

usockets: decouple dns_ready_head and closed_connecting_head lists#29099

Draft
robobun wants to merge 2 commits intomainfrom
farm/3c97b97f/decouple-dns-close-lists
Draft

usockets: decouple dns_ready_head and closed_connecting_head lists#29099
robobun wants to merge 2 commits intomainfrom
farm/3c97b97f/decouple-dns-close-lists

Conversation

@robobun
Copy link
Copy Markdown
Collaborator

@robobun robobun commented Apr 10, 2026

What

Give us_connecting_socket_t a dedicated next_closed link for the deferred-free list (closed_connecting_head) instead of reusing the dns_ready_head next pointer, make us_connecting_socket_free idempotent via a scheduled_for_free bit, and have the drain loop skip entries that are already scheduled for free.

Why

The v1.3.11 linux-x64-baseline crash signature is:

panic(main thread): Segmentation fault at address 0x0
  us_internal_drain_pending_dns_resolve   loop.c:238
  ← return from us_internal_socket_after_resolve

Disassembly of the callee prologue:

mov 0x8(%rdi),%rsi   ; rsi = c->context
mov (%rsi),%rax      ; rax = context->loop   ← fault, rsi == 0
decl 0xb0(%rax)      ; loop->num_polls--

c->context (offset 8 in us_connecting_socket_t) reads as 0. Nothing ever writes c->context = NULL explicitly, so the block at c was freed and recycled by a subsequent us_calloc. #29067 added an extra context ref, which keeps the context alive but does nothing when c itself is the recycled block — the field is zero regardless. #29068 added Bun__addrinfo_cancel, which correctly refuses once request.result is set, so it doesn't change the guarded paths either.

Analysis of the pending_resolve_callback invariant

us_connecting_socket_free(c) is reachable from four sites and in every one c->pending_resolve_callback == 0:

  1. after_resolve(c) c->closed branch — pending cleared two lines earlier; c has just been dequeued from the drain snapshot.
  2. us_connecting_socket_close(c) non-pending branch — explicit pending == 0.
  3. us_connecting_socket_close(c) cancel-success branch — Bun__addrinfo_cancel only returns 1 while request.result == NULL, i.e. before the worker has snapshotted notify (both sides hold global_cache.lock), so us_internal_dns_callback(c) will never fire and c never reaches dns_ready_head.
  4. after_open(s) success — s was created by start_connections(c), called only from after_resolve(c) after it cleared pending.

And us_internal_dns_callback(c) fires exactly once per c (Bun__addrinfo_set either enqueues directly or appends to notify, never both; afterResult moves notify out under the same lock). So c is never simultaneously on closed_connecting_head and reachable from a future dns_ready_head snapshot — the invariant holds through every 2/3/4-socket interleaving traced.

The production workload (fetch via proxy + WebSocket + spawn + AbortSignal, ~3 req/s, one crash per several process-hours) nevertheless hits it, so either there is an interleaving outside the traced set or another writer is corrupting c->next. Separately, us_loop_run_bun_tick has no re-entrancy guard and waitForPromise / HTMLRewriter.transform / expect().resolves can nest it on the same loop from inside on_connect_error, which runs free_closed_sockets inside a drain — but even that path leaves the outer next untouched because next is captured before after_resolve(s).

The structural hazard this removes

The two lists previously shared c->next. us_connecting_socket_free(c) did:

c->next = c->context->loop->data.closed_connecting_head;  // overwrites dns_ready_head link

so the correctness of the drain loop depended entirely on the invariant above. With a dedicated next_closed link, enqueuing c for deferred free can no longer truncate or splice the drain chain, us_connecting_socket_free becomes safely idempotent (double-call early-returns instead of double-unlinking and cycling the close list), and the drain loop skips anything already scheduled for free (valid memory until free_closed_sockets runs) instead of derefing a released context/addrinfo_req.

This turns a subtle cross-function invariant into a structural guarantee: even if a future change violates pending_resolve_callback, the failure mode is a skipped drain entry, not a recycled pointer in after_resolve.

Testing

resolve-dns.test.ts 71/71, node-net/socket.test.ts/tcp-server.test.ts unchanged vs USE_SYSTEM_BUN=1 baseline (failures there are environmental — proxy cert / ECONNREFUSED to external hosts). The new dns-connecting-socket-stress.test.ts (ASAN-only, 25k concurrent net.connect("localhost") + random-phase abort, quarantine_size_mb=0) runs clean on the patched build; it also ran clean on a pre-#29067 ASAN build, consistent with the field hit rate. No deterministic fail-before — the exact interleaving is not reproduced here. The stress test is included as a regression guard for the lifecycle rather than a gate.

@robobun
Copy link
Copy Markdown
Collaborator Author

robobun commented Apr 10, 2026

Updated 10:57 PM PT - Apr 9th, 2026

@autofix-ci[bot], your commit 7e39306 has 3 failures in Build #44832 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 29099

That installs a local version of the PR into your bun-29099 executable, so you can run:

bun-29099 --bun

@robobun
Copy link
Copy Markdown
Collaborator Author

robobun commented Apr 10, 2026

Status: defensive hardening, not a proven root-cause fix. Rebased off e1069426f to drop #29092.

CI (build #44832): all build jobs pass on every platform; all test lanes pass except darwin-13-aarch64-test-bun, which is also failing on #29096 and #29097 — pre-existing flake, not this change. debian-13-x64-asan-test-bun passed, so the stress test ran green under ASAN in CI.

Local (debug+ASAN, e1069426f base):

  • resolve-dns.test.ts 71/71
  • socket.test.ts / tcp-server.test.ts / node-net.test.ts failures are environmental (proxy cert / ECONNREFUSED to external hosts), identical on USE_SYSTEM_BUN=1
  • Stress test (25k net.connect("localhost") + random abort, ASAN, quarantine_size_mb=0) runs clean on both patched and pre-fix(usockets): hold a context ref for the lifetime of pending_resolve_callback #29067 builds — does not reproduce, so no fail-before

Gate: fails on container-local issues (release WebKit fetch → unable to verify the first certificate from the proxy; ASAN legs truncated). Not caused by this diff. Even once those clear, the gate will correctly flag "passes both ways" — this is hardening without a deterministic repro.

Draft. The analysis in the PR body is the deliverable; the diff turns the pending_resolve_callback invariant into a structural guarantee.

@github-actions
Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. fix(usockets): snapshot loop before context unref in us_connecting_socket_free #29064 - Also fixes the v1.3.11 segfault in us_connecting_socket_free / us_internal_drain_pending_dns_resolve; modifies the same function in socket.c to address the same crash, but with a different approach (snapshotting the loop pointer vs. this PR's structural fix with separate next_closed link)

🤖 Generated with Claude Code

@robobun
Copy link
Copy Markdown
Collaborator Author

robobun commented Apr 10, 2026

Not a duplicate of #29064 — that one snapshots loop before the unlink's us_socket_context_unref, which with #29067 already merged is now redundant (the pending-resolve ref keeps the context alive past unlink). This PR instead separates the two linked lists so they use independent link fields, and makes us_connecting_socket_free idempotent. The two are compatible; this one is the broader structural change.

CI build failures are from #29092 (ZIG_PARALLEL_SEMA) which broke main — same failures on other PRs opened after it; #29097 is the follow-up fix. My change compiles clean locally (verified on e1069426f base, the commit immediately before #29092).

robobun and others added 2 commits April 10, 2026 05:06
Give us_connecting_socket_t a dedicated next_closed link for the
deferred-free list instead of reusing the dns_ready_head next pointer,
and make us_connecting_socket_free idempotent via a scheduled_for_free
bit. Drain skips entries that are already scheduled for free.

The two lists previously shared c->next, so enqueuing c for deferred
free while it was still reachable via a dns_ready_head drain snapshot
silently spliced the remaining drain entries onto the close list. The
pending_resolve_callback invariant is supposed to prevent that overlap,
but the v1.3.11 loop.c:238 SIGSEGV @ 0x0 (c->context == NULL in
after_resolve) shows a recycled us_connecting_socket_t reaching the
drain loop, and neither #29067 (context ref) nor #29068 (cancel)
addresses that fault mode. This change makes the separation structural
so a future invariant violation degrades to a skipped entry rather than
a use-after-free.
@robobun robobun force-pushed the farm/3c97b97f/decouple-dns-close-lists branch from 1e28538 to 7e39306 Compare April 10, 2026 05:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant