Skip to content

scsd: re-emit op 0x02 to the coordinator at the ADD transition (vms-a84d) #1029

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
baron-3dl wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

scsd: re-emit op 0x02 to the coordinator at the ADD transition (vms-a84d) #1029

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8ee5485
scsd: re-emit op 0x02 to the coordinator at the ADD transition (vms-a…
baron-3dl Sep 1, 2026
4a6f5b3
scsd: send the transition op-0x02 re-emit SYNCHRONOUSLY, before the b…
baron-3dl Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions src/vmsscs/scsd.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1089,6 +1089,11 @@ struct peer_state {
* peer-initiated connection and answer its config reactively
* on the member VC, like OVMX_PURE_SERVER. */
int joiner_cfg2_sent; /* vms-760: the DEFERRED op 0x02 config/topology went out */
uint32_t cfg2_reemit_epoch; /* vms-a84d: the ADD-transition epoch we last SENT a fresh
* op 0x02 re-emit for (one per epoch, never per frame). The
* re-emit is sent SYNCHRONOUSLY in the XITION handler so it
* precedes the coordinator's barrier fan-out (a poll-loop
* deferral lands ~4s late, after the barrier commits). */
int rejoin_credit_first_sent; /* vms-46f: OVMX's op-6 special-credit request rode the
* coordinator's SCS$DIRECTORY connection AHEAD of op 0x02
* (spec 4(O.17), Davis pp. 2-43/2-44). Gates the deferred
Expand Down Expand Up @@ -7537,6 +7542,79 @@ static void scsd_sysap_msg_input(struct scs_cdt *cdt, const void *msg, size_t ms
"UNKNOWN-CLASS");
fflush(stdout);
}
/* vms-a84d: ARM a FRESH op 0x02 to the coordinator at the ADD
* transition. The deferred op 0x02 (CMCONFIG2) is one-shot on a
* 4.9s timer, decoupled from the coordinator's transitions. A real
* 2->3 join shows the coordinator emits its op 0x12 relay to the
* OTHER existing member ONLY in response to a FRESH joiner op 0x02
* landed AT the transition (vax3-2to3: joiner op 0x02 @+34.7634 ->
* COORD op 0x12 relay to the member @+34.7637, 0.3ms; the member
* then rebuilds toward the joiner and clears its own barrier step).
* In the 1025 stall OVMX's single op 0x02 DID fire the relay -- but
* for the pre-epoch transition; epoch-0xE (the OVMX-ADD) opened
* after it and got no fresh op 0x02, so COORD never relayed OVMX
* into VAX1's rebuild set, VAX1 withheld its 0b#5, and OVMX stalled
* at barrier step 5. Re-emit once per ADD-epoch, only while WE are
* the joiner. The CONTENT is byte-identical to the deferred op 0x02
* (honest -- proven byte-equal to a real joiner's); only the TIMING
* changes. The send itself rides the poll-loop choke point below. */
if (mv.opcode == SCS_MEMBER_OP_XITION &&
ps->xition_class == SCS_MEMBER_CLASS_ADD &&
rx->do_connect && !ps->appeared_after_join &&
ps->barrier_epoch != 0) {
struct peer_state *coord = cm_pick_coordinator(rx->peers);
if (coord != NULL && coord->cfg_sent && coord->pb != NULL &&
coord->cfg_remote_conid != 0 &&
coord->cfg2_reemit_epoch != ps->barrier_epoch) {
/* Send the fresh op 0x02 SYNCHRONOUSLY here, in the
* frame-receive path, the instant the ADD XITION opens --
* so it precedes the coordinator's barrier fan-out. The
* reference joiner's relay-triggering op 0x02 is 0.04s
* BEFORE the barrier (vax3-2to3: op 0x02 +34.7634 ->
* barrier +34.807); a poll-loop-deferred send lands ~4s
* LATE, after the barrier commits at step 5, and the
* coordinator ignores it (#1029b: re-emit @+35.26, barrier
* stalled @+31.2). Content byte-identical to the deferred
* op 0x02 -- only the timing changes. Rides the SAME VC the
* coordinator's config burst used (cfg_*_conid). */
struct scs_member_params mp;
memset(&mp, 0, sizeof(mp));
memcpy(mp.dst_mac, ps_port_addr(coord), 6);
memcpy(mp.src_mac, rx->our_hw_mac, 6);
memcpy(mp.src_logical, rx->our_src_logical, 6);
memcpy(mp.peer_logical, ps_sys_addr(coord), 6);
mp.remote_conid = coord->cfg_remote_conid;
mp.local_conid = coord->cfg_local_conid;
mp.incarnation = coord->incarnation;
mp.recv_ack = coord->vc.seq.recv_seq;
mp.send_seq = scs_seq_advance(&coord->vc.seq);
mp.sysap_send_msg = coord->sysap_send++;
mp.sysap_ack_msg = coord->sysap_recv;
cm_apply_rejoin_form(&mp); /* vms-2f3 */
uint8_t rframe[SCS_MEMBER_FRAME_LEN];
if (scs_member_build_config(&mp, rframe) == 0 &&
send_frame_vc(rx->sock, (int)rx->ifindex, coord, coord->pb,
"transition-phase RE-EMIT op 0x02 (config/topology)",
rframe, sizeof(rframe)) > 0) {
scs_vc_record_sent(&coord->vc, mp.send_seq, monotonic_ms());
coord->cfg2_reemit_epoch = ps->barrier_epoch;
rx->cm_config_frames++;
log_ts(stdout);
printf(" SCSD-I-CFG2REEMIT, ADD transition epoch=0x%08X"
" opened -- sent a FRESH op 0x02 to coordinator node"
" %u SYNCHRONOUSLY (before the barrier fan-out) on VC"
" local=0x%08X remote=0x%08X (send_msg=%u) so it"
" relays us into the other members' rebuild set"
" (vms-a84d)\n",
(unsigned)ps->barrier_epoch,
(unsigned)(peer_node_number(coord) & 0x03ff),
(unsigned)coord->cfg_local_conid,
(unsigned)coord->cfg_remote_conid,
mp.sysap_send_msg);
fflush(stdout);
}
}
}
/* vms-e81 (T1.1): the barrier must arm for EVERY transition,
* not just our own join.
*
Expand Down
Loading