Is grammar-constrained decoding effectively mandatory on abliterated checkpoints?

[behrnt-slatgng]

Disclosure first: I build Grunz, a hosted chat + coding agent on abliterated open-weight models. Cloud, not local, so nothing to sell here — this is about a model property that interacts directly with the structured-output path in these bindings.

Abliterated models degrade obedience before they degrade knowledge

There are thousands of abliterated / decensored GGUFs now, and a lot of them get loaded through here. What breaks first is not what people test for.

The model still knows the material. What gets measurably worse:

• respecting the instruct template and prefill
• honouring stop sequences
• staying inside a structured-output contract (JSON schema, tool-call syntax)
• holding a system-prompt constraint across a long context

So a variant that scores within noise of its base on knowledge benchmarks will fail structured output at a materially higher rate. It knows what to do; it has become worse at doing what it was told.

Why it matters specifically for llama-cpp-python

Three places it lands:

response_format / JSON schema. This is where the damage shows up first and most expensively, because the failure is a parse error rather than a bad answer. If you are converting a schema to GBNF and constraining generation, you are fine — the grammar forecloses malformed output at the token level. If you are prompting for JSON and hoping, the ablated model is meaningfully worse than the base at the same quant.

Stop sequences. These get flaky in a way that reads as a config bug. People assume stop=[...] is broken, or that the chat format is wrong, when the model has simply got worse at honouring stop tokens.

Chat format autodetection. When template adherence degrades, a correctly-detected format still produces drifting output, which sends people off debugging the wrong layer entirely.

The practical upshot

Grammar-constrained decoding is doing more load-bearing work on these models than on clean ones. Worth saying out loud in the JSON-mode docs, because the naive prompt-for-JSON path has a much worse failure rate on ablated weights and people will attribute it to the bindings.

Evaluate format compliance separately from quality. N requests with an exact output contract, scored binary on compliance and independent of whether the answer is correct. A model returning a well-formed wrong answer scores 1.0; a correct answer wrapped in prose that breaks the schema scores 0. That number is what predicts whether an application works, and it will separate two variants that look identical in a chat window. Perplexity will not — on a generic corpus it barely moves while compliance falls off a cliff.

Question

Has anyone here measured GBNF-constrained vs unconstrained structured-output failure rates on an ablated checkpoint against its base, at the same quant? My working hunch is that the gap between constrained and unconstrained is much wider on ablated weights, which would make grammar mode close to mandatory for that class of model. I have the symptom at volume but not a controlled comparison, and I would rather cite a real result than keep repeating a hunch.