Skip to content

Preserve paragraph structure when line comments separate inline text#9

Open
colluca wants to merge 1 commit into
sou1118:mainfrom
colluca:fix-parbreaks
Open

Preserve paragraph structure when line comments separate inline text#9
colluca wants to merge 1 commit into
sou1118:mainfrom
colluca:fix-parbreaks

Conversation

@colluca
Copy link
Copy Markdown

@colluca colluca commented May 14, 2026

Problem

When a // line comment sits on its own line between two sentences with no blank line, the compiled diff output incorrectly introduces a paragraph break between them.

First sentence.
// a comment
Second sentence.

Typst compiles the original correctly as one paragraph. The diff output breaks it into two.

Root cause

typst_syntax tokenises \n// comment\n as three CST nodes: Space("\n"), LineComment(…), Space("\n"). Because markup.exprs() drops LineComment nodes (they carry no semantic content), both surrounding Space("\n") nodes are exposed to the iterator.

typdiff concatenates their raw text into the paragraph buffer, producing "first sentence.\n\nsecond sentence.". When that string is written to the diff output file and compiled by Typst, the bare \n\n is a Parbreak — a paragraph break that was never in the original source.

Note: this does not affect Typst itself, which processes the AST directly through its layout engine and never reconstructs source text from individual nodes.

Fix

Split Expr::Space out of the combined inline arm and skip a Space("\n") node when it would append \n to a buffer already ending with \n. This is always safe: bare \n\n in source is always tokenised as a Parbreak token, never as two consecutive Space nodes — so two adjacent Space("\n") in the expression iterator can only arise from a dropped comment between them.

Expr::Space(_) => {
    let text = node_text(&expr);
    if !(text == "\n" && paragraph_buf.ends_with('\n')) {
        paragraph_buf.push_str(&text);
    }
}
` ` `

## Edge cases

| Source pattern | Before | After |
|---|---|---|
| `text\n// comment\ntext` | paragraph break ✗ | same paragraph ✓ |
| `text\n// c1\n// c2\ntext` | paragraph break ✗ | same paragraph ✓ |
| `text\n\ntext` (blank line) | paragraph break ✓ | paragraph break ✓ |
| `text\n\n// comment\ntext` | paragraph break ✓ | paragraph break ✓ |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant