Skip to content

scsd: send a sequenced recv_ack after a peer's DISCONNECT_RSP (vms-2f3) #1030

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
baron-3dl wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

scsd: send a sequenced recv_ack after a peer's DISCONNECT_RSP (vms-2f3) #1030

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions src/vmsscs/scsd.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1096,6 +1096,16 @@ struct peer_state {
uint16_t sysap_acked; /* vms-760: highest peer send-msg# we have cat-0x04 acked */
long cm_acks; /* cat-0x04 acks emitted to this peer */
long cm_last_ack_ms; /* monotonic_ms() of our last cat-0x04 ack */
/* vms-2f3: a peer's DISCONNECT_RSP advanced our VC recv_seq while OVMX is
* otherwise sequenced-silent to that peer. A standalone 0x48 credit-return
* (send_seq==0) does NOT clear VMS's "unacked" for a sequenced control frame;
* only a recv_ack piggybacked on a SEQUENCED frame (send_seq!=0) does. So OVMX
* owes the peer one sequenced cat-0x04 ack, else the peer retransmits its
* DISCONNECT_RSP every ~3s forever (SDA "Unacked messages 1") and the cluster
* transition never commits. Armed on the received DISCONNECT_RSP, fired by
* scsd_ack_flush_tick on the still-open CM VC, disarmed by any cat-0x04. */
int vc_owe_seq_ack;
uint16_t vc_owe_seq_ack_seq; /* the VC recv_seq the owed ack must cover (for the log line) */
/* vms-760: cluster-wide state-transition barrier (spec 4p). */
uint32_t barrier_epoch; /* body[12:16] latched from the coordinator's op 0x09 */
uint8_t xition_class; /* vms-e4b: body[17] of the transition-open in
Expand Down Expand Up @@ -5207,6 +5217,15 @@ static int scsd_disconnect_dialogue(int sock, int ifindex, struct peer_state *ps
disc_req_recv++;
} else {
disc_rsp_recv++;
/* vms-2f3: a received DISCONNECT_RSP has already advanced ps->vc.seq.recv_seq
* (the credit block runs before this control dispatch). OVMX will otherwise
* answer only with a 0x48 credit-return, which VMS does NOT count as the ack
* for this sequenced frame -- so arm a SEQUENCED cat-0x04 recv_ack on the
* still-open CM VC (same per-peer VC) to cover it. Without this the peer
* retransmits the DISCONNECT_RSP every ~3s (Unacked=1) and blocks the
* transition commit; a real joiner's ongoing sequenced traffic covers it. */
ps->vc_owe_seq_ack = 1;
ps->vc_owe_seq_ack_seq = ps->vc.seq.recv_seq;
}

/* Step 1. Answering a peer's DISCONNECT_REQ is an answer, not an
Expand Down Expand Up @@ -6447,6 +6466,12 @@ static int cm_send_ack(int sock, int ifindex, struct peer_state *ps,
ps->sysap_acked = ps->sysap_recv;
ps->cm_last_ack_ms = monotonic_ms();
ps->cm_acks++;
/* vms-2f3: ANY cat-0x04 carries recv_ack=vc.seq.recv_seq, which satisfies a
* pending DISCONNECT_RSP sequenced-ack obligation -- so if an organic ack
* (op-0x06 burst etc.) already fired, the standalone disc-ack never emits.
* At most one extra cat-0x04 per teardown, and none when other sequenced
* traffic exists. */
ps->vc_owe_seq_ack = 0;
/* vms-584 STRAY-ACK INSTRUMENTATION. A 26-capture census of the
* reference gives a rule we do NOT currently match:
*
Expand Down Expand Up @@ -6532,6 +6557,26 @@ static void scsd_ack_flush_tick(struct scsd_rx *rx, long now_ms)
(now_ms - (long)ps->cm_last_ack_ms) >= (long)SCS_CM_ACK_FLUSH_MS) {
cm_send_ack(rx->sock, (int)rx->ifindex, ps, rx->our_hw_mac,
rx->our_src_logical);
} else if (ps->vc_owe_seq_ack) {
/* vms-2f3: OVMX owes this peer a SEQUENCED recv_ack for a received
* DISCONNECT_RSP (see peer_state.vc_owe_seq_ack). Fire PROMPTLY (no
* flush timer) so the peer's ~3s retransmit loop never even starts.
* cm_send_ack emits a cat-0x04 carrying recv_ack = vc.seq.recv_seq
* (>= the DISCONNECT_RSP send_seq) on the still-open CM VC -- the same
* per-peer VC the directory teardown rode -- and disarms the flag on
* success. NARROW BY CONSTRUCTION: armed ONLY by a DISCONNECT_RSP,
* keyed on the VC recv_seq, never on ps->sysap_recv -- so it is a
* different axis from the op-0x06/join ack cadence above and cannot
* perturb the currently-working path (STRAYACK census, cm_send_ack). */
log_ts(stdout);
printf(" SCSD-I-DISCSEQACK, owe a sequenced recv_ack for a peer's"
" DISCONNECT_RSP (VC recv_seq=%u) -- emitting a cat-0x04 on the"
" CM VC so VMS counts it, else the peer retransmits +"
" Unacked=1 blocks the transition commit (vms-2f3)\n",
(unsigned)ps->vc_owe_seq_ack_seq);
fflush(stdout);
cm_send_ack(rx->sock, (int)rx->ifindex, ps, rx->our_hw_mac,
rx->our_src_logical);
}
}
}
Expand Down
65 changes: 65 additions & 0 deletions tests/vmsscs/test_scsd_wire.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9195,6 +9195,70 @@ static void test_matching_disconnect_rsp_closes_the_connection(void)
"the full teardown scored %lu illegal events", conn_illegal_events);
}

/*
* vms-2f3: a received DISCONNECT_RSP arms a SEQUENCED recv_ack obligation, and a
* cat-0x04 on the CM VC clears it. OVMX otherwise answers a peer's teardown only
* with a 0x48 credit-return (send_seq==0), which VMS does NOT count as the ack for
* the sequenced DISCONNECT_RSP -- so the peer retransmits it every ~3s (SDA
* "Unacked messages 1") and the cluster transition never commits. Confirmed on the
* lab-2 wire: OVMX went sequenced-silent to VAX1 after the teardown (0 sequenced
* frames +20->110s) while a real joiner's ongoing sequenced traffic covers it.
* This is that fix as a test: the DISC_RSP arms the obligation on the VC recv_seq,
* and any cat-0x04 (which carries recv_ack=vc.seq.recv_seq) satisfies + clears it.
*/
static void test_disconnect_rsp_arms_and_a_cat04_clears_the_sequenced_ack(void)
{
struct rxworld r;
struct scs_cdt *cdt = disc_world_init(&r);
if (cdt == NULL) {
return;
}
rx_feed(&r, cap_disconnect_req_to_ovmx, sizeof(cap_disconnect_req_to_ovmx));
if (scs_conn_state_of(cdt) != SCS_CONN_DISC_MATCH) {
return;
}
struct peer_state *rps =
peer_find_or_add(&r.w.cfg, &r.w.pdt, r.w.peers, ovmx760_member_mac);
CHECK(rps != NULL, "peer slot for the answering peer");
if (rps == NULL) {
return;
}
/* No obligation yet, and no SYSAP ack owed -- so only the DISC_RSP path can
* arm the sequenced-ack obligation. */
rps->vc_owe_seq_ack = 0;
rps->sysap_acked = rps->sysap_recv;

rx_feed(&r, cap_disc_rsp_to_ovmx, sizeof(cap_disc_rsp_to_ovmx));

/* ARM (scsd_disconnect_dialogue): the received DISCONNECT_RSP advanced the VC
* recv_seq and armed the sequenced-ack obligation. A 0x48 credit-return alone
* does not clear VMS's unacked for a sequenced frame; a piggybacked recv_ack
* on a sequenced frame does. */
CHECK(rps->vc_owe_seq_ack == 1,
"a received DISCONNECT_RSP did not arm the sequenced-ack obligation "
"(vc_owe_seq_ack=%d) -- OVMX would answer only with an uncounted 0x48 "
"credit and the peer would retransmit forever", rps->vc_owe_seq_ack);
CHECK(rps->vc_owe_seq_ack_seq == rps->vc.seq.recv_seq,
"owed-ack seq %u != advanced VC recv_seq %u",
rps->vc_owe_seq_ack_seq, rps->vc.seq.recv_seq);

/* FIRE + DISARM: a cat-0x04 on the CM VC (what scsd_ack_flush_tick's new branch
* emits) carries recv_ack=vc.seq.recv_seq (>= the DISCONNECT_RSP send_seq) and
* clears the obligation, so it fires at most once per teardown. */
rps->cm_local_conid = 0x08000002u; /* CM VC up, as when a teardown blocks the join */
rps->cm_remote_conid = 0xD6000002u;
long acks_before = rps->cm_acks;
(void)cm_send_ack(r.rx.sock, r.rx.ifindex, rps, r.rx.our_hw_mac,
r.rx.our_src_logical);
CHECK(rps->cm_acks == acks_before + 1,
"the sequenced cat-0x04 ack did not go out (cm_acks %ld -> %ld)",
acks_before, rps->cm_acks);
CHECK(rps->vc_owe_seq_ack == 0,
"the cat-0x04 did not clear the sequenced-ack obligation "
"(vc_owe_seq_ack=%d) -- it would re-fire every tick",
rps->vc_owe_seq_ack);
}

/*
* (3) THE p. 2-27 SIMULTANEOUS CASE. "When each node receives the
* DISCONNECT_REQ from the other node, it replies with a DISCONNECT_RSP. It
Expand Down Expand Up @@ -10999,6 +11063,7 @@ int main(void)
* OLD gate would have refused outright. */
test_control_dispatch_survives_a_frame_the_legacy_marker_would_have_refused();
test_matching_disconnect_rsp_closes_the_connection();
test_disconnect_rsp_arms_and_a_cat04_clears_the_sequenced_ack();
test_simultaneous_disconnect_sends_no_second_request();
test_shutdown_disconnects_every_open_connection();
test_clean_shutdown_kill_switch_through_the_daemon();
Expand Down
Loading