Problem
The offline MTP path generates tokens and reports the count and rate, then prints none of the text.
$ ./target/release/mlxcel generate -m models/qwen3.8-27b-4bit -n 120 --temp 0 \
--draft-model models/qwen3.8-27b-mtp-bf16 --draft-kind mtp --draft-block-size 3 \
-p "Explain in detail how a transformer neural network works, ..."
Generating...
Explain in detail how a transformer neural network works, ...Loading MTP drafter from "models/qwen3.8-27b-mtp-bf16"...
MTP drafter loaded and bound (block_size = 3).
[Generated 120 tokens in 3.34s = 35.90 tok/s]
Two blank lines where the completion should be. The same command without --draft-model prints the text normally.
The tokens are real
The round-loop diagnostics account for them, so this is a reporting gap and not a generation failure:
rounds=51 proposed_tokens=101 accepted_draft_tokens=69 acceptance_rate=0.683
emitted_from_verify_tokens=119 emitted_per_verify=2.333 decode_ms=3179.99
The text is on neither stream: stdout holds the banner and the timing line, stderr holds only the tracing output.
Scope
Why it matters beyond cosmetics
It removes the only way to check MTP's own contract from the CLI. MTP promises temperature-0 output byte-identical to classic decode, and the natural check is to diff generate with and without --draft-model. That check is impossible today, so every byte-identity claim on this path rests on the model-level exactness probe instead of on observed output. Both #1187 and #1189 are in that position right now.
It also means a user cannot use offline MTP for anything, since the feature's output never reaches them.
Acceptance criteria
Problem
The offline MTP path generates tokens and reports the count and rate, then prints none of the text.
Two blank lines where the completion should be. The same command without
--draft-modelprints the text normally.The tokens are real
The round-loop diagnostics account for them, so this is a reporting gap and not a generation failure:
The text is on neither stream:
stdoutholds the banner and the timing line,stderrholds only the tracing output.Scope
mainat4e414bd6,models/qwen3.8-27b-4bitwith theqwen3.8-27b-mtp-bf16drafter.RUST_LOG=info, and with and withoutMLXCEL_MTP_ALLOW_INEXACT=1, so it is not a logging interleave.9a795735(chore(mlx): bump pin to 9a795735 and rebase the four drifted overlays #1198), so neither that bump nor the perf(speculative): recover MTP throughput on Apple GPU generation 15+ without giving up temperature-0 byte-identity #1187 branch caused it.mlxcel-serveris affected, and whether the other drafter kinds (dflash, plain speculative) share the path.Why it matters beyond cosmetics
It removes the only way to check MTP's own contract from the CLI. MTP promises temperature-0 output byte-identical to classic decode, and the natural check is to diff
generatewith and without--draft-model. That check is impossible today, so every byte-identity claim on this path rests on the model-level exactness probe instead of on observed output. Both #1187 and #1189 are in that position right now.It also means a user cannot use offline MTP for anything, since the feature's output never reaches them.
Acceptance criteria
mlxcel generate --draft-kind mtpstreams the generated text the same way the non-speculative path does.mlxcel-serverand the other drafter kinds have the same gap, with the answer recorded either way.generatewith and without--draft-modelon a checkpoint whose gate passes, and record whether they match. That is the end-to-end confirmation the probe currently stands in for.