Paired: 中文
Revision: 2026-08-20 (lab overlay UDP accepted; live Prysm discv5 from L0_ONLY .45 to the .98 DHT server over L0 accepted; .98 DHT-over-L0 toward .82: steer + ~07:13Z authorized restart-beacon (--disable-quic) — overlay TCP accepted (conntrack overlay tuple; ss may show public dest); not FOLLOW_OK / L0_ONLY; drop recovery = flush ghost conntrack first; authorized .45 restart-beacon after dial backoff restores overlay libp2p TCP; lab-only static overlay --peer (not DHT) after authorized hub-then-spoke restart-beacon proves CL catch-up: head_slot↑ / sync_distance↓; re-apply listen-DNAT not steer; prove geth via geth.pid; ss public :4200 is DNAT dest, not a leak; not a production discv5 product)
Public how-to: Applications · Developers
This page is a lab communications experiment. It does not close the P1 follow-the-chain gate. It is not a production discv5 / hybrid-P2P product. Lab UDP echo / TUN tx_dropped is not the published discv5-loss row of the slot-critical publication gate.
Prove that Layer Minus overlay (crate envelope + TUN) can carry OS UDP, including the beacon DHT port :4300, and that a L0_ONLY beacon can abandon public / static --peer and still talk to a public DHT server over L0. No new SI command. Do not use live udp_subscribe / udp_relay.
The overlay envelope is already a complete IPv4 datagram (or several concatenated, split by tot_len). Protocol 17 is in scope. Live SI UDP forward is AES frames over HTTP/SSE and is not this path.
| Script | Host | Role |
|---|---|---|
scripts/probe-l0-udp.sh |
both | Echo / :4300 send / public-ENR steer send / TUN sniff |
scripts/overlay-beacon-listen-dnat.sh |
public-advertise peer (.98) and L0_ONLY .45 |
Overlay VIP tcp/udp DNAT/SNAT into the public listen (any port except geth :8400). Sets TUN accept_local / route_localnet / rp_filter=0 so DNAT to a locally bound public IP actually reaches the socket. UDP SNAT stays the overlay VIP (not the public IP). apply flushes overlay beacon :4200/:4300 conntrack only — never public hub peers, never geth :8400. |
scripts/overlay-dht-steer.sh |
L0_ONLY .45 and public hub .98 |
.45: DNAT .98 :4300/:4200 → 100.64.0.6. .98: DNAT .82 :4300/:4200 → 100.64.0.7 (PEER_PUBLIC_IP=216.225.202.82). MARK after DNAT; apply flushes stale hub :4300/:4200 conntrack (first reconnect step; does not restart geth/beacon). Does not rewrite QUIC :13000 |
scripts/start-shared-beacon-98.sh enable-l0-dht |
.98 |
Write L0_DHT=1 toward .82 (keep public discv5 hub; no last-wins /32 isolate). Does not restart EL/CL |
scripts/start-shared-beacon-98.sh restart-beacon |
.98 |
--p2p-static-id DHT hub. With L0_DHT=1: extra .82 ENR + --disable-quic + fail-closed steer. Does not restart geth |
scripts/start-geth-beacon-only.sh enable-l0-dht |
.45 |
Write L0_ONLY=1 + L0_DHT=1 (+ bootstrap ENR). Does not restart EL/CL |
scripts/start-geth-beacon-only.sh restart-beacon |
.45 |
Apply L0_DHT: drop --no-discovery. Default with L0_DHT_BOOTSTRAP_ENR also drops static --peer. L0_DHT_NO_STATIC_PEER=0 keeps overlay --peer. Fail-closed unless CONET_L0D_DHT_STEER DNATs hub :4300/:4200 |
Do not edit daemon-owned CONET_L0D. Isolate stays CONET_L0D_P2P_ISOLATE / _OUT.
| Claim | Verdict |
|---|---|
Overlay UDP echo 100.64.0.5 ↔ 100.64.0.6:19999 |
Accepted (ECHO_OK) |
Overlay UDP to DHT port 100.64.0.6:4300 |
Accepted (.98 TUN: 100.64.0.5.56249 > 100.64.0.6.4300 UDP length 17) |
| Public-ENR steer rides L0 | Accepted (.98 TUN: 100.64.0.5.35256 > 100.64.0.6.4300 UDP length 19; isolate udp dpt:4300 DROP = 0) |
.98 DHT beacon --p2p-static-id |
Accepted (geth untouched; peer id 16Uiu2HAmF1SXGHnne9DQTHGfgGQgje3cBV8pdSLJF25ajYKr2hvS; still a public discv5 hub) |
L0_ONLY .45 abandons public / static P2P |
Accepted (no --no-discovery; no --peer; allowlist = 100.64.0.0/10 plus hub 198.251.77.98/32 so Prysm can dial the public ENR; steer DNATs that dest onto 100.64.0.6; isolate still DROPs unsteered public :4200 / :4300. After DNAT, .45 ss may show ESTAB to hub public :4200 — that is the original dest, not a leaked public path. Overlay proof: TUN VIP packets + isolate tcp dpt:4200 DROP = 0) |
| Live Prysm discv5 + libp2p over L0 | Accepted (.45 Started discovery v5; bootstrap = .98 ENR; steer TCP :4200 + UDP :4300. 2026-08-18 ~17:28Z after authorized .45 restart-beacon: connected=1; TUN 100.64.0.5:4200 ↔ 100.64.0.6:4200; hub inbound /ip4/100.64.0.5/tcp/4200; REST last_seen on .45 may stay /ip4/198.251.77.98/tcp/4200) |
A garbage UDP probe will not get a discv5 reply. Arrival on the peer TUN is the comms proof. The later live bounce used that path for real discv5 + libp2p.
Authorized beacon-only restarts (no wipe, no validator, geth untouched on both hosts):
# .98 — fixed peer id, keep public advertise
./start-shared-beacon-98.sh restart-beacon
# re-apply overlay-beacon-listen-dnat.sh
# copy localhost identity ENR
# .45 — abandon static --peer; discv5 via bootstrap ENR + steer
L0_DHT=1 L0_DHT_BOOTSTRAP_ENR='enr:…' ./start-geth-beacon-only.sh enable-l0-dht
./overlay-dht-steer.sh
./start-geth-beacon-only.sh restart-beacon.98 still advertises 198.251.77.98 and keeps public peers. .45 advertises overlay 100.64.0.5 and must not dial public :4110 ENRs. L0_DHT sets --p2p-allowlist=100.64.0.0/10 then --p2p-allowlist=198.251.77.98/32 (Prysm v7.1.4: one CIDR per flag, last wins). That last /32 is what lets Prysm connect() the bootstrap ENR; packets still ride L0 because overlay-dht-steer.sh DNATs hub :4300/:4200 to 100.64.0.6. With a bootstrap ENR, drop overlay --peer (discv5-only). L0_DHT_NO_STATIC_PEER=0 is a P1 recovery switch, not the accepted DHT path. Start is fail-closed if that steer is missing (isolate would DROP the real public dial). Do not treat this as FOLLOW_OK or a finished P2 product. EL may stay 0x0.
Static --peer |
DHT / discv5 (--bootstrap-node=<ENR>) |
|
|---|---|---|
| What you must know in advance | That peer’s multiaddr (IP + TCP + peer id) | At least one bootstrap ENR (node id and IP + UDP are inside the ENR) |
| Dial | Direct libp2p connect(); discovery optional |
UDP discv5, then libp2p TCP |
| Other nodes you never listed | Not found | Can be looked up after join |
Lab .45 with L0_DHT_BOOTSTRAP_ENR |
Dropped (accepted path) | Required. Isolate + last-wins allowlist /32 mean .45 cannot discover .98 from public :4110 ENRs |
Connecting to this DHT hub still requires its ENR. “No need to know the DHT id” is only true for later peers found in the table, not for the bootstrap node.
--p2p-static-id on .98 keeps that ENR valid across beacon restarts. REST last_seen on .45 may show /ip4/198.251.77.98/tcp/4200 (ENR + conntrack original dest) while TUN is 100.64.0.5 ↔ 100.64.0.6:4200 and ss on .98 is inbound from 100.64.0.5:4200 (wire on TUN). Do not treat the public dest in .45 ss as “isolate failed”.
Unmodified Prysm discv5 will not dial a wallet or a peer id alone. The hub ENR must carry a dialable IP. Lab .98 still advertises public 198.251.77.98 and keeps public peers — it is a public discv5 hub, not an anonymous one.
L0 hides the client public IP from the peer (isolate + steer). Both sides remaining anonymous (no IP in any discovery record, no SI entry seeing a source IP, no AddressPGP route) is not this lab and is not stock discv5.
Steer / listen-DNAT are operator objects, not CONET_L0D. conet-l0d stop tears down TUN; re-apply steer after the next start. The load watchdog only starts/stops geth on load15; it does not keep DHT up.
2026-08-18 ~11:14Z lab: .45 beacon connected 1→0; head_slot froze at 778719. Overlay geth stayed ESTAB (100.64.0.5 → 100.64.0.6:8400) because geth dials the overlay VIP, not the public ENR. Beacon conntrack showed hub UDP :4300 UNREPLIED (DNAT onto overlay, no matching reply). Isolate DROP on other public :4300 IPs is expected L0_ONLY. .98 listed .45 as disconnected inbound and did not redial (already ~14 public peers).
First recovery (do this; do not restart geth or beacon):
# .45 only — reinstalls DNAT/MARK and flushes hub :4200/:4300 conntrack
cd /home/peter/conet-l0d-lab
./overlay-dht-steer.sh apply2026-08-18 11:49Z: that apply cleared the ghost UDP mapping; geth pid 884029 and beacon pid 890053 unchanged. About 60s later connected was still 0 (disconnected outbound to the hub ENR). A new UNREPLIED UDP probe reappeared — discv5 was sending again, but Prysm stayed in in-process dial backoff and did not open TCP. Flush is necessary and sufficient to repair NAT; it is not sufficient once backoff has started.
Only if connected stays 0 after the flush (authorized .45 restart-beacon, geth untouched):
./start-geth-beacon-only.sh restart-beacon
# steer must already be applied *before* start. Do not overlay-dht-steer.sh apply
# immediately after start — that flushes SYN_SENT and re-enters backoff.Do not treat P1 overlay batch flushed for POST as HTTP 2xx. Do not add public --peer or lift isolate to “fix” DHT.
2026-08-18 ~17:00Z .98 listen-DNAT (no EL/CL restart on .98): discv5 UDP still arrives on .98 TUN and the beacon replies. TCP listen is only 198.251.77.98:4200, so overlay 100.64.0.6:4200 must DNAT onto that socket. That DNAT needs TUN accept_local=1 / route_localnet=1 / rp_filter=0, or NAT counts the SYN and the socket never sees it (no SYN-ACK on TUN). Re-apply overlay-beacon-listen-dnat.sh on both hosts. Do not flush overlay geth :8400 conntrack when applying (early apply wiped .45 net_peerCount to 0x0; geth pid unchanged). UDP SNAT must stay the overlay VIP. After that apply, .98 public connected=14 was unchanged; .45 beacon connected stayed 0 (dial backoff + UDP conntrack still UNREPLIED after reverse-NAT). .98 geth/beacon pids were not restarted.
2026-08-18 ~17:28Z authorized .45 restart-beacon (geth untouched): steer was already applied. Did not re-apply overlay-dht-steer.sh after start. geth pid 900292 stayed ESTAB to 100.64.0.6:8400. Beacon pid 899131 → 900670. Within ~18s: connected=1. TUN tcpdump: bidirectional PSH/ACK 100.64.0.5:4200 ↔ 100.64.0.6:4200. .45 ss showed 74.208.224.45:4200 → 198.251.77.98:4200 (DNAT original dest). Isolate tcp dpt:4200 DROP = 0. Hub: 198.251.77.98:4200 ← 100.64.0.5:4200. Logs stayed suitable=0 for ~1 min, then Processing blocks. head_slot left 778656 and climbed (~3–6 slots/s; overlay RTT ~0.7s; one peer). geth eth_blockNumber stayed 0x0 — CL lag, not overlay down. Do not treat EL 0x0 or the first-minute suitable=0 as a failed restart. Do not restart .45 geth while overlay :8400 is ESTAB.
2026-08-20 ~04:09Z authorized hub-then-spoke restart-beacon (static overlay --peer, not DHT): lab-only toml (channels 8400+4200; no prod .82, no :4300 / L0_DHT this run). Bounce conet-l0d hub then spoke. After each beacon PID change, re-apply overlay-beacon-listen-dnat.sh apply. Do not immediately overlay-dht-steer.sh apply. Prove geth unchanged via geth.pid (.45 971773, .98 3420373); do not use pgrep -n geth (beacon-chain argv contains the geth path). Hub public connected dipped 14→0 then recovered to 15; hub peer_id stayed static. Spoke: connected=1 outbound /ip4/100.64.0.6/tcp/4200; overlay :4200 and :8400 ESTAB; both sides :4200 duplex AES (no 409). CL catch-up proof in ~1 min: head_slot 937984→938207, sync_distance 32443→32228; is_syncing=true; is_optimistic=true; EL still 0x0. That is follow-the-chain in progress, not a peer failure. .98 geth HTTP in this lab is :8889, not :8545.
2026-08-20 production hub .82 overlay geth inbound (no EL/CL restart): .82 toml registered lab .98 geth (100.64.0.6:8400) with that host’s user PGP; .98 registered .82 overlay VIP 100.64.0.7:8400. Bounce only conet-l0d .82 then .98; re-apply listen-DNAT. Geth still advertises public @216.225.202.82:8400. .98 admin_removePeer the public enode (same node id), then admin_addPeer overlay …@100.64.0.7:8400. Proof: overlay ESTAB 100.64.0.6↔100.64.0.7:8400; duplex AES both ways; .45 still overlay-ESTAB to both hubs. geth.pid unchanged (.82=1222, .98=3420373, .45=971773). Do not change .82 --nat to the VIP. New spokes need their own hub [[peers]] user PGP. See operator-flags.md.
2026-08-20 ~07:13Z .98 DHT-over-L0 toward .82 (authorized restart-beacon): steer 216.225.202.82:4300/:4200 → 100.64.0.7 was already applied. enable-l0-dht writes env (no EL/CL restart). .98 stays a public discv5 hub for .45 (no L0_ONLY isolate, no last-wins /32). Authorized .98 restart-beacon (geth.pid 3420373 unchanged): --disable-quic + extra .82 ENR; then listen-DNAT, not steer. Overlay TCP proof is conntrack reply 100.64.0.7 ↔ 100.64.0.6:4200; ss may still show public dest 216.225.202.82:4200 (DNAT original dest, not a leak). REST last_seen may list /ip4/216.225.202.82/tcp/4200. Hybrid hub, not L0_ONLY, not FOLLOW_OK. Public verdict: GitBook lab evaluation.
- New SI commands or hostnames
- Wiping either host or restarting any validator
- Claiming P1 follow-the-chain is done
- Using live SI
udp_relayas OS UDP - Production hybrid P2P (public + L0) as the default
If this file, the whitepaper, RULES.md, or MVP changes, update GitBook Applications and Developers conet-l0d pages in the same task.