Skip to content

Latest commit

 

History

History
190 lines (137 loc) · 16.6 KB

File metadata and controls

190 lines (137 loc) · 16.6 KB

P1 — overlay TCP on current L0 primitives

Paired: 中文
Revision: 2026-08-18 (application duplex on Chat gossip + P1 gossip when the peer app does not accept; optional [[l0.channels]] per-port routing EOA + listen SSE; overlay IPv4 batch + POST 32/512; Prysm-bound follow-the-chain; ~17:28Z authorized .45 restart-beacon restored overlay DHT TCP / Processing blocks while EL stayed 0x0; separate lab UDP/DHT comms in P2.md does not close this gate)

Public how-to: Applications · Developers

Crate MVP (TUN + iptables + locator + packet counters) is accepted. This page is the next crate phase.

Goal

Carry overlay IPv4 (geth / beacon TCP toward a peer vIP) as a wallet-to-wallet byte stream. Prefer SI exclusive occupancy pipe + application duplex: l0_listen / l0_connect; offer on the long-lived Chat SSE; accept / reject / frames as AES blobs on the occupied L0 pipe. duplex_reject or a missing duplex_accept or missing pipe keeps P1 gossip. SI does not implement duplex_*. Do not invent p2p_stream_* or listenKind: "l1p2p". Never send command: "mining" with listenKind: "duplex".

Wire (duplex first, P1 gossip fallback)

Object Encrypt to HTTP
duplex_offer (overlay AES key + session listen wallet) Peer long-lived user PGP Healthy entry A ≠ B. Ordinary Chat gossip to the existing channel SSE
Exclusive L0 listen Own B route PGP l0_listen or mining + listenKind: "l0" via C ≠ B. No overlay Securitykey. Crate MVP: per-port channel EOA
l0_connect Target mailbox B route PGP Occupies idle L0 SSE; then AES blobs on the same TCP. Occupied → 409
duplex_accept / duplex_reject AES on occupied initiator L0 pipe Not user-PGP POST. First AES blob after responder occupies W_I
Pipe teardown SI l0_pipe_end (inbound line) + optional l0_listen_released (listen SSE) Connector clears pipe_tx, bumps pipe_gen, short retry on occupy. Not { "data" } gossip
Overlay IPv4 (duplex data) AES of duplex_frame JSON; payload = standard base64 of L0D1||IPv4 Occupied peer L0 pipe. One AES layer. Not PGP
P1 gossip fallback Peer user PGP, then mailbox work (NoPush) to B route PGP Same { "data" } body — never an HTTP field
Chat-pool listen (offers) B route PGP, command: mining + listenKind: "chat" on a dedicated routing EOA (optional [[l0.channels]]: one EOA + SSE per overlay port 8400 / 4200 / 4300) Healthy entry C ≠ B

HTTP JSON is only { "data": "<OpenPGP armor>" }. No NoPush / beamioNoPush on the HTTP object.

Envelope (inner plaintext, then OpenPGP):

{"type":"conet_l0d_overlay_v1","from":"0x…","seq":1,"ipv4":"<standard-base64 of one IPv4 datagram, or several concatenated>"}

ipv4 is one complete IPv4 datagram or several concatenated datagrams split inbound by IPv4 tot_len (a fixture / illegal tot_len stays one packet). Same type; HTTP is still only { "data" }. Both lab conet-l0d binaries must upgrade together — an old peer treats concatenated bytes as one TUN frame.

The overlay envelope has no signMessage. Listen uses EIP-191 on the SI wrapper only. Do not POST plaintext JSON as data.

AddressPGP: 0x684b0ac760cEE9c9b85de36d69746420648Cf9e2. RPC: https://rpc1.conet.network or https://publicrpc.conet.network.

Config

[l0] defaults to off. An authorized two-host lab may turn [l0] on (host copies; git config/conet-l0d.{45,98}.toml may stay commented off). That is still not a public mailbox product. The authorized lab wrote bidirectional IPv4 on both TUNs, completed overlay geth and beacon TCP, and is running CL initial-sync over overlay. Follow-the-chain is in progress (EL still 0x0).

[l0]
enabled = false
rpc = "https://rpc1.conet.network"
address_pgp = "0x684b0ac760cEE9c9b85de36d69746420648Cf9e2"
# entries = ["https://<existing-guardian>.conet.network"]
# listen_entries = ["https://<existing-guardian>.conet.network"]
# routing_eoa = "0x<dedicated-routing-eoa>"
# routing_key_file = "/etc/conet-l0d/routing.key"
# routing_eth_key_file = "/etc/conet-l0d/routing.eth"
# mailbox_route_pgp_file = "/etc/conet-l0d/self-mailbox-route.asc"

# [[peers]] lab override (do not log contents):
# user_pgp_file = "/etc/conet-l0d/peer-user.asc"
# route_pgp_file = "/etc/conet-l0d/peer-route.asc"

Routing EOA ≠ Chat wallet ≠ deposit keystore ≠ fee recipient. Do not log the key file.

Outbound is fail-closed: [l0].enabled plus the peer user PGP file, the peer route PGP file, and at least one entry. Do not POST an envelope encrypted only to user PGP. routing_key_file is an OpenPGP secret cert for inbound decrypt; outbound does not need it. routing_eth_key_file is a hex secp256k1 key for EIP-191 listen (not OpenPGP). Listen HTTP+SSE is fail-closed: enabled plus listen_entries (C ≠ B; do not fall back to outbound entries), mailbox_route_pgp_file (this host's B route public key), routing_eoa, routing_key_file, and routing_eth_key_file whose recovered address matches routing_eoa. Listen is EIP-191 + SI { message, signMessage } base64. Production SI listen is not opened. Tests use wiremock only. Do not put Securitykey in a B-decryptable listen command.

Optional [[l0.channels]] gives one routing EOA + OpenPGP secret + EIP-191 key + mailbox route public key + listen SSE per overlay port (8400 geth TCP, 4200 beacon TCP, 4300 discv5 UDP). Outbound classifies IPv4 by well-known source or dest port (return-path SYN-ACK has dest = ephemeral). Encrypt to the peer user PGP for that port; do not skip a second [[peers]] row on the same VIP. Empty [[l0.channels]] keeps one EOA for all ports. :4300 stays overlay IPv4 — not SI udp_relay. Do not bind two SSEs to the same routing_eoa.

What exists in-crate (this revision)

Item Status
Binary frame L0D1 + JSON envelope + { data } POST URL builder Implemented; unit-tested
AddressPGP searchKey ABI encode / decode Implemented; fixture tests only (no live RPC in cargo test)
[l0] parse + host allow-list Implemented; default enabled = false
OpenPGP encrypt (peer user PGP) + mailbox wrap (NoPush to B route PGP) + POST { data } Implemented; unit-tested; dest-aggregated IPv4 batch (16 packets / 12 KiB); concurrent worker (32) + queue 2048; inbound TUN write queue 1024; default off
TUN packet loop Counts IPv4; when enabled and keys+entry exist, queues an encrypted POST (does not block TUN). Linux clones the TUN fd for inbound write-back
Inbound decrypt (user-PGP armor → envelope → IPv4) + TUN write queue In-crate; unit-tested; unused when [l0] is off or routing_key_file is missing
Listen command encode (mining + listenKind: "chat" and l0_listen / listenKind: "l0", no overlay Securitykey) + EIP-191 SI wrap + encrypt to B route PGP In-crate; unit-tested
Occupied l0_connect pipe (JSON { data } then AES lines on the same TCP) In-crate; src/l0/pipe.rs. Live SI: l0Exclusive.ts
Listen HTTP+SSE worker (POST { data } to C, read stream, apply_inbound_armor) In-crate; mock-tested; Chat + exclusive L0 workers; unused when [l0] is off. Optional [[l0.channels]] spawns one pair per overlay port
Listen ingest of SI gossip JSON { "data": "<armor>" } (raw or SSE data:), occupy JSON, AES blobs, classic SSE armor lines In-crate; unit-tested. Matches Chat handleInbound / SI forWardPGPMessageToClient. Ignore listing/liveness JSON
Production SI listen / live mailbox SSE Lab-only workers may POST mining + listenKind: "chat" and l0_listen to entry C. l0_listen / l0_connect are current SI. Not a public product
Application duplex Offer is Chat gossip; accept / reject / frames are AES on the occupied L0 pipe. Spec: GitBook l0/duplex-forward
Authorized lab [l0] on .45 / .98 Allowed when the operator authorizes it; git example config may stay off
Production mailbox delivery Not shipped
Dedicated SI p2p_stream_* / listenKind: "l1p2p" Not current SI

Operator iron rule

Authorized L0_ONLY .45 advertises the overlay vIP (100.64.0.5) with --nat extip / --p2p-host-ip and --p2p-static-id. .98 and production proposers keep the public IP until that host is authorized L0-only or follow-the-chain completes. Do not bind Engine or HTTP to the overlay vIP.

This-stage evaluation (2026-08-17 23:12Z live L0-only)

Authorized: restart only .45 geth + beacon (no wipe, no validator). .98 geth/beacon were not restarted. Isolate used CONET_L0D_P2P_ISOLATE / _OUT (owned CONET_L0D untouched). Advertise stayed on the public IP. Dedicated routing EOAs were registered with regiestChatRoute. Both hosts had [l0] on and a listen SSE ESTAB to entry C.

Claim Verdict
Crate MVP (TUN / iptables / locator / counters) Accepted (reconfirmed)
P1 outbound: TUN IPv4 → user PGP + mailbox wrap → POST { data } HTTP 200 Accepted (88 frames to 100.64.0.6, 73-byte TCP SYNs, locator …/p2p/geth)
P1 inbound: mailbox → listen stream → TUN write-back on .98 Failed at 23:12Z (.98 l0d: no inbound / no TUN write). Crate then only scanned SSE data: BEGIN PGP lines; live SI writes raw JSON { "data": "<armor>" }. Parser now matches Chat. 23:30Z retest accepted (see next section)
.45 L0-only geth/beacon gossip + follow the chain Failed (geth peers 0; beacon connected 0; EL still 0x0; CL head stuck 766496, distance grew 172132172168)
.98 public path (unchanged clients) Still synced (EL 0xd82f1; beacon head 938664, distance 0)
HTTP 200 on entry A Not inbound delivery. Mailbox is store-and-forward; TCP handshake did not complete
Advertise overlay vIP Still forbidden

.45 (100.64.0.5): L0_ONLY + isolate; watchdog sources run/l0-only.env; no validator.
.98 (100.64.0.6): public P2P kept; admin_removePeer dropped the .45 public enode; no overlay/74.208.224.45 geth peer.

Restore public P2P on .45 with ./start-geth-beacon-only.sh stop-isolate then a normal restart (still do not wipe). Do not restart .98 geth/beacon unless the operator authorizes that host.

This-stage evaluation (2026-08-17 23:30Z listen-parser retest)

Authorized: restart only conet-l0d on .45 and .98 (not geth/beacon). Host [l0] stayed enabled. Do not overwrite host toml with git examples.

Claim Verdict
Listen ingest of SI gossip JSON { "data": "<armor>" } Accepted (crate tests + live mailbox stream)
P1 inbound: mailbox → listen → TUN write-back Accepted on both hosts (P1 inbound IPv4 queued for TUN write-back; 0 write-back failures; 0 refused armors)
Overlay geth TCP Accepted (.45 100.64.0.5:50516.98 100.64.0.6:8400; .98 admin_peers inbound from 100.64.0.5)
.45 L0-only follow the chain Not yet (EL still 0x0; beacon connected 0; CL head still 766496)
Advertise overlay vIP Still forbidden

Some outbound POST /post requests still fail under the SYN flood; that did not block geth ESTAB. HTTP 200 on entry A is still not by itself delivery.

This-stage evaluation (2026-08-18 00:17Z overlay advertise + live clients)

Authorized: restart only .45 conet-l0d + geth + beacon (no wipe, no validator). .98 geth/beacon were not restarted. Crate outbound POST is concurrent (8) with queue 64. .45 beacon uses --p2p-static-id. Load watchdog was paused only for the handshake window, then restarted.

Claim Verdict
.45 advertises overlay vIP 100.64.0.5 Accepted (geth --nat extip:100.64.0.5; beacon --p2p-host-ip=100.64.0.5)
.98 advertise stays public Unchanged (198.251.77.98)
Overlay geth TCP + net_peerCount Accepted (.45 100.64.0.5:43264.98 100.64.0.6:8400; peers 0x1)
Overlay beacon TCP + connected Accepted (.45 outbound 100.64.0.5:51152100.64.0.6:4200; peer 16Uiu2HAmDb9FNGMYhC7p7rrLEBGA9HujPZda1KHWKvJfox9YRwDA; connected=1)
.45 CL initial-sync over overlay In progress (head left 766496766911 at 00:18Z; initial-sync Processing blocks)
.45 L0-only follow the chain Not complete (EL still 0x0; ~172k slots behind tip)
.98 public path Still synced (EL 0xd8582; CL head 939321, distance 0)

Prysm --p2p-host-ip is advertise-only. The host that still listens on the public IP (.98) needs overlay :4200 DNAT/SNAT into that listen; do not edit daemon-owned CONET_L0D. Stopping geth while beacon is batch-syncing makes Engine :8200 refuse and skips that batch; start geth again and sync resumes.

Later lab snap (same advertise, no further restart): .45 overlay TCP still ESTAB (100.64.0.5:43264:8400, 100.64.0.5:51152:4200); geth --nat extip:100.64.0.5; beacon --p2p-host-ip=100.64.0.5 --p2p-static-id; net_peerCount=0x1; beacon connected=1; CL head_slot=767295, sync_distance=172055, is_syncing=true; EL still 0x0. .98 still advertises 198.251.77.98; inbound overlay geth 100.64.0.5:43264 and beacon :4200 from 100.64.0.5:51152; EL 0xd859f; CL head 939350, distance 0.

This-stage evaluation (2026-08-18 overlay batch + POST concurrency)

Per-packet OpenPGP + HTTP was dropping overlay frames (queue-full on both hosts) and kept CL initial-sync far behind tip. Crate now dest-aggregates IPv4 into one envelope and raises the POST worker to concurrency 32 / queue 512.

Authorized: copy both new binaries first, then restart only conet-l0d on .45 and .98 together (~00:37Z). Do not restart geth/beacon.

Claim Verdict
Both hosts run the batching binary Accepted (7.4 MiB @ 00:37Z; dest-aggregated flushes, many at 16 packets / 12 KiB)
POST / inbound queue-full after restart Accepted (0 drops after 00:37:40 on both hosts; older log lines are pre-upgrade)
Overlay TCP after l0d bounce Accepted (.45 100.64.0.5:43264:8400, :51152:4200; geth 0x1; beacon connected=1)
CL initial-sync rate Improved, not done (head_slot 769791 / sync_distance 169755 @ 00:41Z; ~256 slots in 90s after restart)
Follow-the-chain Not complete (EL still 0x0)

This-stage evaluation (2026-08-18 00:44Z Prysm-bound follow-the-chain)

After the batching binary, overlay is not the limiter: queue-full is 0 after 00:37:40, overlay TCP stays ESTAB, geth 0x1, beacon connected=1. Prysm initial-sync on .45 processes 3.2 blocks/s (batchSize=64; one 6.4 blip during the l0d bounce), self-estimate ~15 h. Do not raise crate batch size or invent a SI stream to “speed this up.” This follow-the-chain gate stays open. A separate lab UDP/DHT comms experiment may run (P2.md); it does not close this gate and is not a finished P2 product. 2026-08-18 ~17:28Z: authorized .45 restart-beacon (geth untouched) restored overlay discv5/libp2p TCP after dial backoff; CL returned to Processing blocks. EL still 0x0 is CL lag, not overlay down.

Follow-the-chain is accepted only when all of these are true on authorized L0_ONLY .45:

Gate Pass
Overlay TCP ESTAB to peer vIP :8400 and :4200
Peers geth net_peerCount ≥ 1, beacon connected ≥ 1
CL is_syncing=false and sync_distance ≤ 64
EL eth_blockNumber leaves 0x0
Overlay queues no sustained queue full after the last conet-l0d start

Read-only operator snapshot (no EL/CL/l0d restart):

# on .45
./scripts/watch-l0-follow.sh
WATCH=1 INTERVAL_SECONDS=60 ./scripts/watch-l0-follow.sh

FOLLOW_OK is the only pass. UNTRUSTED (RPC fail) must not be treated as empty/zero.

Out of scope (not a crate bug)

  • New SI commands or new hostnames
  • Advertising overlay vIP on production proposers or unauthorized hosts
  • Restarting or wiping .98 / any validator
  • Production discv4 / discv5 (lab UDP comms: P2.md; after DNAT, .45 ss may show hub public :4200 — original dest, not a leak; not a closed P2 product)

Sync rule

If this file, the whitepaper, RULES.md, or MVP changes, update GitBook Applications and Developers conet-l0d pages in the same task. Do not expand L0 protocol pages unless a new SI command exists.