Skip to content

scsd: send a sequenced recv_ack after a peer's DISCONNECT_RSP (vms-2f3) - #1030

Draft
baron-3dl wants to merge 1 commit into
mainfrom
vms-2f3-disconn-ack
Draft

scsd: send a sequenced recv_ack after a peer's DISCONNECT_RSP (vms-2f3)#1030
baron-3dl wants to merge 1 commit into
mainfrom
vms-2f3-disconn-ack

Conversation

@baron-3dl

Copy link
Copy Markdown
Contributor

The stall

The booted-node cluster join reaches provisional membership on a real VAX (SDA CLUB Nodes=3, OVMXJ0 with member flags) but never commits — the barrier stalls, VAX1 times out ~90s later and resets the VC, and CLUSTER_NODES never ticks 2→3.

Root cause (wire + VAX-internal SDA, both-ways confirmed)

When VAX1 tears down a transient SCS$DIRECTORY connection, its final DISCONNECT_RSP is a sequenced frame (send_seq=N). OVMX advances its VC recv_seq and answers with a standalone 0x48 credit-return (send_seq==0) — but VMS does not count a credit-return as the acknowledgment for a sequenced control frame; it clears "unacked" only on a recv_ack piggybacked on a sequenced frame (send_seq≠0). After the teardown OVMX goes sequenced-silent to VAX1 (0 sequenced frames +20→110s in the 1029c run; only credits), so its piggybacked recv_ack never reaches N. VAX1 retransmits its DISCONNECT_RSP every ~3s the whole run (37×; a real joiner leaves ~2, because its ongoing DLM-rebuild traffic keeps recv_ack advancing) — the exact VAX1 SDA "Unacked messages: 1" (VAX2=0) that keeps VAX1's SCS channel to OVMX from ever going clean, blocking the transition commit.

Fix

On a received DISCONNECT_RSP, arm a one-shot obligation (peer_state.vc_owe_seq_ack) to emit a sequenced cat-0x04 ack on the still-open CM VC (same per-peer VC), carrying recv_ack = vc.seq.recv_seq (≥ the DISCONNECT_RSP send_seq). scsd_ack_flush_tick fires it promptly (no flush timer) so the retransmit loop never starts; any cat-0x04 (incl. an organic op-0x06 ack) disarms it → at most one extra frame per teardown, none when other sequenced traffic exists.

Regression safety

Narrow by construction: armed only by a DISCONNECT_RSP, keyed on the VC recv_seq — a different axis from ps->sysap_recv (the SYSAP high-water the op-0x06/join ack cadence uses). The STRAYACK census warns against widening the sysap-keyed trigger; this doesn't touch it. Reuses the existing cm_send_ack/scs_member_build_ack emitter — no new frame builder, FSM row, or credit-path change.

Tests

test_disconnect_rsp_arms_and_a_cat04_clears_the_sequenced_ack — a real captured teardown arms the obligation on the VC recv_seq, and a cat-0x04 clears it. All 26 vmsscs unit tests pass.

⚠ Verification bar

H2 is OVMX↔OVMX and won't exercise the real-VAX teardown/retransmit — CI-green is necessary-not-sufficient; the lab is the proof. Lab-adjudicated three signals: (1) 0x7b DISCONNECT_RSP retransmits 37→~2, (2) SDA Unacked→0, (3) barrier commit → CLUSTER_NODES 2→3. Signals 1+2 prove the channel-clean fix landed; signal 3 tells us whether this prerequisite alone fires the commit or whether the full DLM rebuild with VAX1 is also needed. Milestone stays un-cut until sustained OVMXJ0=member at CLUSTER_NODES=3, cap-denied, both consoles.

🤖 Generated with Claude Code

…teardown acks (vms-2f3)

The booted-node cluster join reaches PROVISIONAL membership on a real VAX
(SDA CLUB Nodes=3, OVMXJ0 member flags) but never COMMITS: the barrier
stalls, VAX1 times out ~90s later and resets the VC, and CLUSTER_NODES
never ticks 2->3.

Root cause (wire + VAX-internal SDA, both-ways confirmed): when VAX1 tears
down a transient SCS$DIRECTORY connection, its final DISCONNECT_RSP is a
sequenced frame (send_seq=N). OVMX advances its VC recv_seq and answers
with a standalone 0x48 credit-return (send_seq==0) -- but VMS does NOT
count a credit-return as the acknowledgment for a sequenced control frame;
it clears "unacked" only on a recv_ack PIGGYBACKED on a sequenced frame
(send_seq!=0). After the teardown OVMX goes sequenced-silent to VAX1 (0
sequenced frames +20->110s in the 1029c run; only 0x48 credits), so its
piggybacked recv_ack never reaches N. VAX1 therefore retransmits its
DISCONNECT_RSP every ~3s for the whole run (37x; a real joiner leaves ~2,
because its ongoing DLM-rebuild traffic keeps recv_ack advancing) -- the
exact VAX1 SDA "Unacked messages: 1" (VAX2=0) that keeps VAX1's SCS channel
to OVMX from ever going clean, blocking the transition commit.

Fix: on a received DISCONNECT_RSP, arm a one-shot obligation (peer_state
vc_owe_seq_ack) to emit a SEQUENCED cat-0x04 ack on the still-open CM VC
(same per-peer VC), carrying recv_ack=vc.seq.recv_seq (>= the DISCONNECT_RSP
send_seq). scsd_ack_flush_tick fires it promptly (no flush timer) so the
retransmit loop never starts; any cat-0x04 (incl. an organic op-0x06 ack)
disarms it, so at most one extra frame per teardown and none when other
sequenced traffic exists.

NARROW BY CONSTRUCTION: armed ONLY by a DISCONNECT_RSP, keyed on the VC
recv_seq -- a different axis from ps->sysap_recv (the SYSAP high-water the
op-0x06/join ack cadence uses), so it cannot perturb the currently-working
provisional-member path (the STRAYACK census warns against widening the
sysap-keyed trigger; this does not touch it).

Reuses the existing cm_send_ack / scs_member_build_ack cat-0x04 emitter --
no new frame builder, FSM row, or credit-path change. Test added
(test_disconnect_rsp_arms_and_a_cat04_clears_the_sequenced_ack): a real
captured teardown arms the obligation on the VC recv_seq and a cat-0x04
clears it. All 26 vmsscs unit tests pass.

Lab-adjudicated (necessary prerequisite; may not alone fire the commit if
that also needs the full DLM rebuild): (1) 0x7b DISCONNECT_RSP retransmits
37->~2, (2) SDA Unacked->0, (3) barrier commit -> CLUSTER_NODES 2->3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhM3QcmFEY3p8YNnHGaYwP
@baron-3dl
baron-3dl marked this pull request as draft September 1, 2026 16:40
@baron-3dl

Copy link
Copy Markdown
Contributor Author

Held as draft — the fix fired cleanly but is inert against VAX1's retransmit accounting. Lab re-fire (run 10301): SCSD-I-DISCSEQACK fired ×2 at +17s emitting the sequenced cat-0x04 ack (recv_seq=30, on the CM VC), but VAX1 retransmitted its DISCONNECT_RSP 30× anyway until the ~90s VC-reset timeout ended the loop — not the fix. All three signals negative (retransmit stayed ~30, Unacked=0 only post-reset, barrier 5/12, CN=2).

Key finding: a cat-0x04 ack is already an env-0x4b sequenced frame carrying recv_ack on the shared per-peer VC — so 'wrong frame type' is an incomplete explanation. VMS's SCS-transport retransmit-clear accounting counts something our model doesn't capture (candidate: per-connection sequence, so a frame on the CM conid doesn't clear a retransmit on the closed directory conid). Holding until the exact field-level diff between the reference's DISC_RSP-stopping frame and this cat-0x04 is grounded — not another blind frame-swap. The arm-on-DISCONNECT_RSP / emit-on-VC scaffolding is sound and reusable; only the ack vehicle is unresolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant