Broadcast to every server at once, and stop leaking the NOWNodes key - #458
Broadcast to every server at once, and stop leaking the NOWNodes key#458peachbits wants to merge 2 commits into
Conversation
`ServerConfig` had a single type, `blockbook-nownode`, so when Edge's own Blockbook hosts were added in Dec 2024 (9962d7e) they were filed under it alongside the real NOWNodes entries. Every HTTP broadcast therefore sent the NOWNodes key to Edge's servers as well. They are not the same kind of server: GET https://btc-wusa1.edge.app/api/v2 -> 200, no key needed GET https://btcbook.nownodes.io/api/v2 -> 401 Unknown API_key The type becomes `blockbook | blockbook-nownode`. The ten edge.app HTTP entries are now plain `blockbook`, the nownodes.io entries are unchanged, and the key is attached per target rather than to the whole batch. This also fixes the no-key case. A missing `nowNodesApiKey` previously rejected the entire HTTP path with "Missing connection key for fallback servers", even though the public servers need no key. NOWNodes targets are now skipped with a warning and the public servers still carry the broadcast; the reject is reserved for having no usable server at all. Adds test/common/utxobased/engine/ServerStates.spec.ts covering the header routing, both no-key cases, and the existing HTTP broadcast behavior it has to preserve.
`broadcastTx` used the HTTP servers only when no WebSocket reported itself connected. A socket that looks connected but never answers `sendTransaction` therefore failed the entire broadcast, after the 30 second request timeout, without a single HTTP attempt being made. That is the shape of the incident this task exists for (Asana 1217135300337949): the send failed in the UI while the transaction was already on the network. The gate is worse than it looks, because the NOWNodes WebSocket URI is docked 400 points at score load (ServerScores.serverScoresLoad penalises any URI carrying key params) and so rarely wins one of the two connection slots. HTTP is the realistic route to those servers, and it was reachable only when every socket was visibly down. broadcastTx now fires at every cached blockbook and every HTTP server in parallel. Sockets that are still connecting are included, since a queued request transmits as soon as the socket opens and times out otherwise. The first success resolves. It rejects only once every attempt has failed, logging each failure against the server that produced it. Extends the ServerStates spec with a real WebSocket server that accepts `sendTransaction` and never answers, asserting the broadcast still resolves over HTTP in well under the socket timeout.
fa5f837 to
f809273
Compare
j0ntz
left a comment
There was a problem hiding this comment.
Reviewed alongside EdgeApp/edge-react-gui#6190; the findings are inline.
Nit: the commit subject Broadcast to every server at once instead of gating HTTP on the sockets is 71 characters, over Edge's 50-character limit. Broadcast to every server at once is 33 and the body already carries the rest. (Send the NOWNodes API key only to NOWNodes servers is exactly 50 and fine.)
| if (blockbook == null) continue | ||
| attempts++ | ||
| blockbook | ||
| .broadcastTx(transaction) |
There was a problem hiding this comment.
A failing WebSocket attempt never settles, so this promise can hang instead of rejecting.
blockbook.broadcastTx goes through promisifyWsMessage (Blockbook.ts), which awaits a Deferred that is only ever resolved from inside its own generator. Every failure path in Socket.ts delivers the error as request.generator.throw(err).catch(e => log.error(e.message)): the error-response branch in onMessage, the 30s expiry in onTimer, and onSocketClose. That generator has no try/catch, so the rejection is logged and the deferred is left pending forever. No onFailure ever runs for that attempt, failures never reaches attempts, and the promise settles neither way.
sequenceDiagram
participant SS as ServerStates.broadcastTx
participant Sock as Socket blockbook
participant Http as HTTP servers
SS->>Sock: broadcastTx, attempts = 1
SS->>Http: sendtx, attempts = 2
Http-->>SS: rejects, onFailure, failures = 1
Sock->>Sock: 30s expiry, generator.throw
Sock->>Sock: Deferred left pending, no onFailure
SS->>SS: failures 1 < attempts 2, never settles
The deferred bug is pre-existing, but the denominator change is what exposes it. Before this commit, the fully-disconnected case took the !isAnyBlockbookConnected branch, skipped sockets entirely and rejected off the HTTP count, so a genuinely failed broadcast surfaced as a failure. Sockets are now always counted, so an all-fail broadcast with any entry in serverStatesCache never rejects. Paired with EdgeApp/edge-react-gui#6190 that is a locked slider spinning forever with no message at all, since the scene's finally never runs.
The new spec covers socket-hangs-but-HTTP-succeeds; it is the all-fail variant that hangs. Either bound each attempt before counting it, or have promisifyWsMessage reject its deferred when the generator throws.
| 'api-key': nowNodesApiKey | ||
| }, | ||
| body: transaction.signedTx | ||
| for (const { uri, headers } of httpTargets) { |
There was a problem hiding this comment.
Every send now discloses the transaction to the NOWNodes HTTP endpoint even when the sockets and Edge's own servers are healthy, where HTTP was previously a fallback only. For BTC that is btc-wusa1.edge.app, btc-eu1.edge.app and btcbook.nownodes.io on every broadcast, unconditionally.
Unblocking HTTP is the right call; making it unconditional on the happy path is a separate product decision that the PR description does not call out. Worth confirming it is intended, or staggering the third-party targets behind a short delay so they only fire when the first wave has not resolved.
j0ntz
left a comment
There was a problem hiding this comment.
Correcting the verdict on my earlier review: those findings are blocking, not advisory.
The broadcast hang is the one that gates this. An all-fail broadcast with any entry in serverStatesCache settles neither way, and paired with EdgeApp/edge-react-gui#6190 the send scene has no error to show and no finally to unlock the slider.
CHANGELOG
Does this branch warrant an entry to the CHANGELOG?
Dependencies
none
Description
Engine half of the "send failed in the UI but the money moved" incident. Asana: https://app.asana.com/1/9976422036640/project/1213880789473005/task/1217135300337949
GUI companion (independent, neither blocks the other): EdgeApp/edge-react-gui#6190
Two separate problems in
ServerStates.broadcastTx, one commit each. The first commit stands alone and can be reviewed, or landed, without the second.1.
Send the NOWNodes API key only to NOWNodes serversServerConfighad a single type,blockbook-nownode, so when Edge's own Blockbook hosts were added in Dec 2024 (9962d7e) they were filed under it alongside the real NOWNodes entries. Every HTTP broadcast therefore sent the NOWNodes key to Edge's servers too.They are not the same kind of server:
The type becomes
blockbook | blockbook-nownode. The tenedge.appHTTP entries are now plainblockbook, thenownodes.ioentries are unchanged, and the key is attached per target instead of to the whole batch.This also fixes the no-key case. A missing
nowNodesApiKeypreviously rejected the entire HTTP path with "Missing connection key for fallback servers", even though the public servers need no key. NOWNodes targets are now skipped with a warning and the public servers still carry the broadcast.2.
Broadcast to every server at once instead of gating HTTP on the socketsThe HTTP servers were used only when no WebSocket reported itself connected. A socket that looks connected but never answers
sendTransactiontherefore failed the whole broadcast, after the 30 second request timeout, without a single HTTP attempt being made. That is the shape of the incident behind this task.The gate is worse than it looks: the NOWNodes WebSocket URI is docked 400 points at score load (
ServerScores.serverScoresLoadpenalises any URI carrying key params) and so rarely wins one of the two connection slots. HTTP is the realistic route to those servers, and it was reachable only when every socket was visibly down.broadcastTxnow fires at every cached blockbook and every HTTP server in parallel. Sockets still connecting are included, since a queued request transmits as soon as the socket opens and times out otherwise. The first success resolves; it rejects only once every attempt has failed, logging each failure against its server.Testing
New
test/common/utxobased/engine/ServerStates.spec.ts, seven tests in the first commit and one more in the second:api-keyheader reaching NOWNodes servers and not the public onessendTransactionand never answers, while HTTP still resolves in well under the 30 second socket timeoutFull suite: 1261 passing at the first commit, 1262 at the second.
tscandeslintclean at both.Note for reviewers
This deliberately does not include the broadcast-failure classification from #455. Per the direction agreed for this task, the app now hard-fails an ambiguous broadcast and says so, rather than trying to infer whether the transaction landed. The
saveTx"No addresses to process" fix from #455 is still valid and is not in this branch.