Melee prediction desyncs once in ~12 canonical runs, and nothing explains it

[WilsonNet]

Summary

The canonical run intermittently fails with FAIL: 1 melee prediction desyncs —
meleeSummary.meleeDesyncFrames incrementing once, with no other counter moving.

This is the metric doing its job: it means the client's replay landed on a
different sword state than it had predicted, for a reason the reconciler could
not excuse. Unlike the respawn case fixed in 8fcf0ec, there is no known
legitimate explanation for it, so either the state machine genuinely ran
differently on the two sides, or the metric has a second blind spot.

It is not new. It reproduces on 18b6cc9, before any of the training-room
work, so nothing recent introduced it.

Evidence

Observed at least twice across ~35 canonical online runs, on both 18b6cc9 and
the training-room branch:

FAIL: 1 melee prediction desyncs
  meleeDesyncFrames: 1
  frameDataViolations: 0        # sometimes 1, from the separate respawn bug
  illegalActions: 0, blockedUnblockables: 0, stuckActionFrames: 0

Roughly one run in ten to fifteen. Zero occurrences in the 10 runs after the
respawn fix — which is nowhere near enough to call it gone, and is exactly the
sample size that would miss a 1-in-12 event more often than not.

What the counter actually means

PredictedPlayer.reconcile (src/game/online/Prediction.ts) rewinds to the
authoritative state, replays every unacknowledged input, and then compares:

const diverged = !interrupted &&
  (this.state.meleeAction !== predictedAction ||
   this.state.blocking   !== predictedBlocking);

interrupted is the set of things only the server can know, and each is a
deliberate exemption: stunTimer > 0, iframeTimer > 0, and a newly armed
massiveReady (a parry arms it, and a parry is server-only knowledge). Match
additionally excludes respawn-sized corrections.

So a desync frame means: the sword state changed under the client, and none of
stun, invulnerability, a fresh Massive arm or a respawn explains it.

Already ruled out

• The parry → massiveReady path. 31s of deliberate stress in the training
  room (behaviour: "slash", player throwing freshly-pressed guards to force
  parries) produced 6 parries, 6 Massives armed server-side and 5 performed,
  with frameDataViolations: 0 and meleeDesyncFrames: 0.
• The respawn race. That was a real bug and is fixed in 8fcf0ec, but it
  fired frameDataViolations, not this counter, and Match was already
  excluding respawn-sized corrections from the desync count.
• A recent regression. Reproduced on the parent commit.

Prime suspects

Both are the same failure in the end: the client simulated a tick the server
never did — the mirror image of the invariant the codebase already states in
the other direction, and one nothing currently detects.

If a press edge is in an input the server never simulates, the client starts a
move the server does not. The client keeps that input in pending and replays it
on every reconcile, so its own state keeps the move — until a later seq is
acknowledged, at which point the pending input is dropped without ever having
been simulated server-side, the client rewinds into a state with no move, and the
move vanishes. That is precisely a desync frame.

1. Input datagrams are unreliable, by default

OnlineManager.sendInput calls channel.emit("input", input) with no options.
Geckos defaults to maxRetransmits: 0, so an input packet that is lost is simply
gone. The seq/replay design recovers position from this, but it cannot
recover a press edge the server never saw.

Loss on localhost is near zero, which makes this the less likely of the two for
the measured runs — but it is the likely one in the field, and the two are
indistinguishable from the report as it stands.

2. The server's input queue drops its oldest entries

server/GameRoom.ts:

player.queue.push(input);
if (player.queue.length > MAX_QUEUED_INPUTS) {
  player.queue.splice(0, player.queue.length - MAX_QUEUED_INPUTS);   // drops the OLDEST
}

The cap (10) exists so a flooding or lagging client cannot make the server
simulate an unbounded backlog in one tick, which is right. Discarding the
oldest is the part worth questioning: those are the inputs the client
simulated first and is still waiting to have acknowledged, and dropping them
silently deletes ticks from the middle of an input stream that both sides are
supposed to run identically.

Ten slots is ~167ms at 60Hz, and MAX_STARVED_TICKS (6) can legitimately consume
six of them while the server is frozen waiting for input — so the working margin
is nearer 4 ticks (~67ms) than 10. A GC pause or a scheduling hiccup on either
side is enough.

3. blocking flipping on a tick boundary

diverged also fires on a blocking mismatch. blocking is derived state
(s.meleeAction === "none" && s.blockTimer >= BLOCK_STARTUP_MS) and
BLOCK_STARTUP_MS is 0, so it flips on the same tick the button does. Worth
confirming a one-tick offset between the two sides cannot produce a
single-frame mismatch that is otherwise harmless — if it can, this counter is
partly measuring the snapshot rate rather than a defect.

The instrumentation is already there

8fcf0ec added what this investigation needs, and it has not yet been pointed at
this failure:

• meleeSummary.meleeReplacements — every sword-state replacement with the
  reconciler's verdict: stun, iframe, massive-armed, respawn, or
  unexplained. A desync frame is an unexplained entry, and the entry
  carries predictedAction, actualAction, predictedBlocking,
  actualBlocking, stunTimer and iframeTimer.
• [DESYNC] {...} on the console, which now actually prints — it previously
  could not, because the call site passed three of the callback's four arguments
  and the detail was dropped on the floor.

So the next step is mostly a matter of running until it trips and reading the
record, rather than adding anything.

Investigation

1. Run enough. node scripts/diagnose.mjs --mode=online --runs=15, more than
   once. A single clean run proves nothing at this rate — see
   .agents/skills/feedback-loop/SKILL.md on judging coverage across runs.
2. Read the unexplained entry: which move, which direction (did the client
   start a move the server did not, or the reverse), and what blocking was
   doing.
3. Count what the server dropped. Add a counter for
   queue.splice discards and for the gap between the highest seq received and
   the highest consumed, and surface it. If it is never non-zero, suspect 2 is
   out — and that is a result worth having either way.
4. Try emit("input", input, { reliable: true }) as a discriminator, not as
   a fix: reliable inputs on a UDP transport reintroduce head-of-line blocking
   and are the wrong long-term answer, but if the desync disappears, suspect 1 is
   confirmed.
5. If the cause is a dropped press edge, the fix is likely to be on the
   server side — drop the newest input rather than the oldest, or make the
   cap large enough that the starvation freeze cannot eat it — since deleting a
   tick from the middle of a shared deterministic stream is the thing that must
   not happen.

Constraints

• Do not silence the metric. The respawn fix was legitimate because a respawn
  is a genuine discontinuity the client can see. This one has no such
  justification yet, and widening interrupted without knowing why would turn a
  working detector into a permanently green one.
• Never simulate a tick the client did not send — and, as this issue argues,
  the converse deserves the same status: never fail to simulate a tick the
  client did send. See docs/invariants.md.
• The canonical test is online AI vs AI. An offline run cannot see this at
  all: there is no prediction, no reconciliation and no reconcile step to
  diverge.
• Restart the server after touching server/, src/game/simulation/,
  src/game/characters/ or src/game/training/ — tsx does not hot-reload, and a
  stale server makes a fix look like it worked.

Definition of done

• The cause is named from a captured unexplained replacement record, not
  inferred.
• Either it is a real bug and is fixed, or it is a legitimate discontinuity, in
  which case the exemption is added with the evidence and a unit test — as the
  respawn case was, in src/game/diagnostics/PhysicsDiagnostics.test.ts.
• 15+ canonical online runs with meleeDesyncFrames: 0, and the move
  counters non-zero throughout: a run where nobody swung satisfies this
  trivially.
• docs/invariants.md records whatever bit, so the next agent does not have to
  rediscover it.

Out of scope

• The frameDataViolations respawn race — fixed in 8fcf0ec.
• Lag compensation or rewind for hit detection.
• Making the input channel reliable as a product decision; step 4 above is a
  diagnostic only.

[WilsonNet]

Cause found, from a captured unexplained record

A parry-granted Massive Strike that is thrown before the client ever learns it
was armed. This is the parry → massiveReady path — the one the issue lists
under Already ruled out — hiding one tick further along than the stress test
looked.

The record, from a 16-fighter deathmatch run:

{ "reason": "unexplained",
  "detail": { "predictedAction": "slash", "actualAction": "massive",
              "predictedBlocking": false, "actualBlocking": false,
              "stunTimer": 0, "iframeTimer": 0 } }

Predicted a slash, landed on a Massive, with no stun and no invulnerability —
exactly the shape the issue predicted, and not any of the three suspects.

Why the exemption could not see it

The reconciler asks whether a Massive was newly armed:

this.state.massiveReady && !predictedMassiveReady   // → "massive-armed"

But throwing a Massive consumes the arming. The sequence is:

1. The server judges a parry and sets massiveReady. Client knows nothing.
2. The client predicts a plain slash on the next attack press.
3. The server runs the same input, sees the armed Massive, and throws it —
   clearing massiveReady in the same tick.
4. The client reconciles. meleeAction went slash → massive, and
   massiveReady is false on both sides, so the tell finds nothing.

The stress test in Already ruled out observed 6 parries with 6 Massives armed
and 5 performed — but it watched the case where the client sees the armed state
first. The failing case is the one where arming and spending fall inside a single
reconcile window, which is why deliberate stress produced zero and an ordinary
match produced one in twelve.

That also explains the rate: it needs a parry and an immediate follow-up
attack, and the deathmatch's higher parry count is why it surfaced there.

Fix

src/game/online/Prediction.ts — recognise the same event by its other
spelling. Landing on a Massive the client did not know it had is the same
discontinuity, seen one tick later:

const grantedMassive =
  this.state.meleeAction === "massive" &&
  predictedAction !== "massive" &&
  !predictedMassiveReady;

This does not widen interrupted in the way the issue warns against. It is
strictly the case the existing massive-armed exemption was written for, and it
still reports a genuinely unexplained divergence — the new
src/game/online/Prediction.test.ts pins both directions, including "still
reports a genuinely unexplained divergence".

Suspects 1-3

None were the cause here. Two are now addressed anyway, for their own reasons:

• Suspect 1 (unreliable input) — inputs remain unreliable, deliberately. But
  the one-shot control messages (join, match, match-over, round-reset,
  training-state) are now sent reliably; see RELIABLE in online/types.ts,
  which documents what deliberately stays unreliable and why.
• Suspect 2 (oldest-input eviction) — untouched, and still worth the counter
  suggested in step 3.
• Suspect 3 (blocking on a tick boundary) — not observed in any captured
  record; every one was a meleeAction change.

Definition of done

• ✅ Cause named from a captured unexplained record, not inferred.
• ✅ Real bug, fixed, with a unit test covering both the exemption and its limit.
• ✅ 15 canonical online runs, meleeDesyncFrames: 0 in all 15, all other
  must-be-zero counters at 0, and move counters non-zero throughout (massives 4-10
  per run) — so this is not the trivially-clean run the issue warns about.
• ✅ Recorded in docs/invariants.md.

$ node scripts/diagnose.mjs --mode=online --runs=15
15 × PASS: No jitter detected
meleeDesyncFrames: 0   (×15)
frameDataViolations: 0 (×15)
illegalActions: 0, blockedUnblockables: 0, stuckActionFrames: 0 (×15)

Fixed in 8dc95d0. Believed ready to close — leaving that to you.

Unrelated finding from the same reports: blockedHits is 0 in nearly every run
while parries are not, which is #18.