fix(webrtc): buffer early data channel messages until muxer adoption

[Faolain]

Summary

In a private-to-private WebRTC connection, each peer creates its DataChannelMuxerFactory at the start of SDP signaling but only creates the muxer - which attaches stream/protocol handlers to incoming data channels - when the connection upgrade completes, some time after the underlying RTCPeerConnection connects. Data channels that arrive in between are parked in earlyDataChannels without a message listener, and node-datachannel (like browser RTCDataChannels) does not buffer messages dispatched without a listener - so if the remote opens a stream as soon as the direct connection is available, its initial multistream-select bytes are silently dropped, protocol negotiation stalls, and the stream dies on the dialer's timeout even though the connection itself is healthy.

Environment

• macOS 26.3 (also diagnosed downstream on macOS/Node 22 in the ts-drp project)
• Node.js v22.15.0
• js-libp2p main @ 5459e13, which is libp2p@3.3.5, @libp2p/webrtc@6.0.26, @libp2p/circuit-relay-v2@4.2.8, @libp2p/websockets@10.1.16, @libp2p/identify@4.1.9, node-datachannel@0.32.3, @chainsafe/libp2p-yamux@8.0.1, @chainsafe/libp2p-noise@17.0.0

Reproduction

In-repo: packages/integration-tests/test/webrtc-early-stream-race.node.ts (wired into test/node.ts). Relay (the aegir fixture relay) + two Node peers; the dialer upgrades the relayed connection to a direct WebRTC connection and opens an echo stream from its connection:open event, i.e. in the same event loop turn that publishes the connection. The recipient is held at the pre-muxer upgrade checkpoint via an async denyInboundEncryptedConnection connection gater for 500ms to make the race deterministic (see below). Without the fix in this PR the test failed 10/10 consecutive runs with TimeoutError: The operation was aborted due to timeout; a sibling control test that only differs in when newStream is called (it waits for the recipient to surface the connection) passed 10/10.

Why the artificial delay: the window is inherent but only a few milliseconds wide between two Node processes on one idle machine - debug traces on this branch show the recipient adopting the early channel ~2-4ms after the dialer's first bytes would arrive. Over real networks (or with a busy event loop) the recipient's upgrade completion is delayed arbitrarily relative to the dialer's first stream data, which is how this was hit downstream. The gater hook runs immediately before muxer creation in the upgrader, so it widens exactly the window in question using only public API. Crucially, if early channels buffered their data the delay would only add latency - the echo would still complete well inside the 5s timeout. That is what happens with the fix applied: the same test passes in ~1.6s.

Trace of a failing run (DEBUG='libp2p:webrtc*'):

18:26:54.360Z libp2p:webrtc:connection:outbound:...:muxer open channel 3 for protocol /test/webrtc-early-stream/1.0.0
18:26:54.862Z libp2p:webrtc:connection:inbound:... create muxer /webrtc
18:26:54.862Z libp2p:webrtc:connection:inbound:...:muxer incoming datachannel with channel id 3, protocol  and status open
18:26:59.365Z libp2p:webrtc:connection:outbound:...:muxer:outbound:3:error abort with error - TimeoutError: The operation was aborted due to timeout
18:26:59.378Z libp2p:webrtc:connection:inbound:...:muxer:inbound:3:error closed while reading 0/1 bytes

Standalone reproduction using published packages (no repo checkout needed)

npm i libp2p @libp2p/webrtc @libp2p/circuit-relay-v2 @libp2p/websockets @libp2p/identify @chainsafe/libp2p-yamux @chainsafe/libp2p-noise @multiformats/multiaddr-matcher, then node webrtc-early-stream-race.mjs. Exits 1 with a timeout (race reproduced); set RECIPIENT_UPGRADE_DELAY to 0 (or open the stream later) and it exits 0.

// webrtc-early-stream-race.mjs
import { noise } from '@chainsafe/libp2p-noise'
import { yamux } from '@chainsafe/libp2p-yamux'
import { circuitRelayServer, circuitRelayTransport } from '@libp2p/circuit-relay-v2'
import { identify } from '@libp2p/identify'
import { webRTC } from '@libp2p/webrtc'
import { webSockets } from '@libp2p/websockets'
import { WebRTC } from '@multiformats/multiaddr-matcher'
import { createLibp2p } from 'libp2p'

const ECHO_PROTOCOL = '/test/webrtc-early-stream/1.0.0'
const STREAM_TIMEOUT = 5_000
const RECIPIENT_UPGRADE_DELAY = 500

const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms))

const relay = await createLibp2p({
  addresses: {
    listen: ['/ip4/127.0.0.1/tcp/0/ws']
  },
  transports: [webSockets()],
  connectionEncrypters: [noise()],
  streamMuxers: [yamux()],
  services: {
    identify: identify(),
    relay: circuitRelayServer({
      reservations: {
        maxReservations: Infinity,
        applyDefaultLimit: false
      }
    })
  }
})

const recipient = await createLibp2p({
  addresses: {
    listen: [`${relay.getMultiaddrs()[0]}/p2p-circuit`, '/webrtc']
  },
  transports: [circuitRelayTransport(), webSockets(), webRTC()],
  connectionEncrypters: [noise()],
  streamMuxers: [yamux()],
  connectionGater: {
    denyDialMultiaddr: () => false,
    denyInboundEncryptedConnection: async (peerId, maConn) => {
      if (WebRTC.exactMatch(maConn.remoteAddr)) {
        // hold the incoming WebRTC connection at the pre-muxer upgrade
        // checkpoint to widen the (otherwise a-few-ms-wide) race window -
        // models a recipient whose upgrade completes after the dialer's
        // first stream data arrives (network latency, busy event loop)
        await delay(RECIPIENT_UPGRADE_DELAY)
      }

      return false
    }
  },
  services: {
    identify: identify({ runOnConnectionOpen: false })
  }
})

await recipient.handle(ECHO_PROTOCOL, (stream) => {
  void (async () => {
    for await (const buf of stream) {
      stream.send(buf)
    }
    await stream.close()
  })().catch(err => {
    stream.abort(err)
  })
})

const dialer = await createLibp2p({
  transports: [circuitRelayTransport(), webSockets(), webRTC()],
  connectionEncrypters: [noise()],
  streamMuxers: [yamux()],
  connectionGater: {
    denyDialMultiaddr: () => false
  },
  services: {
    identify: identify({ runOnConnectionOpen: false })
  }
})

const webRTCAddress = recipient.getMultiaddrs().find(ma => WebRTC.exactMatch(ma))

dialer.addEventListener('connection:open', (event) => {
  if (!WebRTC.exactMatch(event.detail.remoteAddr)) {
    return
  }

  // open a stream in the same event loop turn that publishes the direct
  // connection - the recipient may not have created its muxer yet
  void (async () => {
    const stream = await event.detail.newStream(ECHO_PROTOCOL, {
      signal: AbortSignal.timeout(STREAM_TIMEOUT)
    })

    stream.send(Uint8Array.from([0, 1, 2, 3]))

    for await (const buf of stream) {
      console.log('echo response received (%d bytes) - no data was lost', buf.byteLength)
      process.exit(0)
    }
  })().catch(err => {
    console.error('early stream failed - race reproduced:', err.message)
    process.exit(1)
  })
})

console.log('dialing', webRTCAddress.toString())
await dialer.dial(webRTCAddress)

Diagnosis

packages/transport-webrtc/src/muxer.ts (at main):

• DataChannelMuxerFactory is constructed at the start of signaling (private-to-private/initiate-connection.ts for the dialer, private-to-private/transport.ts _onProtocol for the recipient). Its datachannel listener only does this.earlyDataChannels.push(evt.channel) - no message listener is attached to the parked channel.
• createStreamMuxer() is called later, from the upgrader, after the signaling stream has been closed (a relayed round-trip) and the rest of the upgrade has run. Only then does DataChannelMuxer adopt the parked channels (in a queueMicrotask) via onDataChannel() -> createStream(), which is where channel.onmessage is first assigned (src/stream.ts).
• node-datachannel's RTCDataChannel polyfill dispatches message events as they arrive; events dispatched while no listener is attached are lost (verified in isolation: 3/3 messages sent into a captured-but-unadopted channel were dropped). Browsers behave the same way, which is why src/stream.ts already carries a comment that message listeners must be attached before events occur.
• Result: the remote's optimistic multistream-select header + protocol id are dropped. The opener waits for a response that never comes and aborts on its signal (10s default in downstream apps); the adopting side's inbound stream later logs muxer ... closed while reading 0/1 bytes. Downstream (ts-drp, macOS/Node 22) the trace signature is muxer error closed while reading 0/1 bytes -> TimeoutError: The operation was aborted due to timeout -> libdatachannel error ... DataChannel is closed (node-datachannel RTCDataChannel.ts:209) as the abandoned channel is torn down.

Relationship to existing issues

• Getting Failed to send message to peer: TimeoutError: signal timed out when sending a message from b2b relayed peers #3092 - browser peers connected via a relay see TimeoutError: signal timed out when sending on a /webrtc address that both sides report as connected. Same failure family (first application stream on a freshly-upgraded private-to-private WebRTC connection stalls); this PR contributes a deterministic Node-side reproduction and a mechanism.
• transport-webrtc: private-to-private transport leaks one abort listener per connection #3567 - webrtc subsystem issue; this is a complementary Node-only reproduction against current main.

What this PR contains

• packages/integration-tests/test/webrtc-early-stream-race.node.ts - integration test that opens an echo stream from the dialer's connection:open event (fails 10/10 on main with TimeoutError), plus a control that only differs in when newStream is called (passes 10/10 on main).
• packages/transport-webrtc/src/muxer.ts - fix: onEarlyDataChannel now attaches a buffering onmessage handler to parked channels; when the muxer adopts a channel the buffered events are replayed synchronously right after createStream() assigns the real handler, preserving ordering with later messages. With the fix the race test passes 10/10 and npm run --workspace @libp2p/webrtc test:node is green.

Notes for maintainers

• The 500ms gater delay in the test widens an inherently-present window (measured at ~2-4ms between two idle local peers) using only public API; happy to restructure if you would prefer a different mechanism for making the race deterministic (e.g. separate processes plus induced latency).
• The buffering fix is intentionally minimal and confined to muxer.ts. An alternative would be to create the WebRTCStream eagerly in the factory; that is more invasive, so this PR keeps the parked-channel design and just makes it lossless. Happy to iterate if you would prefer that direction.
• If you would rather land the test without the fix, it will fail until the race is addressed - in that case we would suggest marking it .skip on landing and can split the PR accordingly.

[Faolain]

would love to get some eyes on this @tabcat whenever you get the chance 🙏

[tabcat]

Hey @Faolain thank you for reporting this bug. I've been looking at the pr for the last few days. Currently looking at whether this needs DoS mitigations.

This will likely be merged by the end of week. In the future, if you'd like me to look at something I'd prefer you message me on Discord. You can find me in the libp2p server (there should be a link in the root readme) in the js-libp2p channel.