Skip to content

fix: Keep whitespace at inline element boundaries in Markdown output - #78

Closed
shuvamk wants to merge 1 commit into
EmilStenstrom:mainfrom
shuvamk:fix/markdown-inline-boundary-whitespace
Closed

fix: Keep whitespace at inline element boundaries in Markdown output#78
shuvamk wants to merge 1 commit into
EmilStenstrom:mainfrom
shuvamk:fix/markdown-inline-boundary-whitespace

Conversation

@shuvamk

@shuvamk shuvamk commented Jul 31, 2026

Copy link
Copy Markdown

to_markdown() drops the space that sits just inside an inline element, welding the words on either side together. to_text() on the same input is correct, and so is <span>, which makes the two spellings of the same document disagree.

input to_text() to_markdown() renders as
<p>This is <b>very </b>important.</p> This is very important. This is **very**important. This is <strong>very</strong>important.
<p>Visit <a href="https://e.com">our site </a>today.</p> Visit our site today. Visit [our site](https://e.com)today. …</a>today.
<p>See <em>the docs </em>for details.</p> See the docs for details. See *the docs*for details. …</em>for details.
<p>Visit<a href="https://e.com"> our site</a> today.</p> Visit our site today. Visit[our site](https://e.com) today. Visit<a…>

This is the shape a rich-text editor produces when the user's selection includes the trailing space, so it shows up in pasted content rather than in hand-written HTML.

Cause

a, em, i, strong and b render their children into a separate _MarkdownBuilder, so the boundary whitespace never reaches the parent. A trailing space is still sitting in the child's _pending_space when finish() runs, and a leading space is discarded because the child's buffer is empty at that point. <span> and the other inline containers share the parent builder and go through _pending_space normally, which is why they already came out right.

Fix

Record on the builder whether whitespace was collapsed before any output, and re-emit both boundaries into the parent when the marker or link is closed.

Re-emitting alone injects a second space after a structural marker: both places the builder can flush a pending space, raw() and text(), appended one whenever the buffer was non-empty without checking what it ended with — and headings and list items write "## ", "- " and "1. " through raw(). In a list that destroys structure rather than spacing. The extra space moves the item content to column 3 while nested lists indent by 2, so the inner list in

<ul><li><b> outer</b><ul><li>inner</li></ul></li><li>second</li></ul>

no longer clears its parent item and renders as a flat sibling. Both flushes now skip a buffer that already ends in a space or tab.

I put the guard in the builder rather than at the two re-emission sites deliberately: the re-emissions work because they are ordinary text(" ") calls, which is what lets a boundary space collapse against adjacent text, disappear at block and document ends, and propagate outward through nested inline elements such as <a>x <b>y </b></a>z. Guarding at the call sites would have meant four copies of the check reaching into parent_builder._buf, and would have left the rule false for the other text(" ") callers. Happy to switch if you'd rather have it the other way.

The predicate also covers whitespace that predates this change, so <ul><li> Item</li></ul> now renders - Item instead of - Item, and <h2> Title</h2> renders ## Title. Both render identically in CommonMark; where the column does matter, for a nested list, the new output is the correct one. That is pinned by a test so the change is deliberate rather than incidental.

Tests

Eight tests in tests/test_node.py alongside the other to_markdown cases, covering both directions: the space that must appear at an inline boundary, and the space that must not appear after a ##/- /1. marker or inside flatten_list_item. The nested-list test asserts the exact string, including the existing blank-line shape, so a future regression in either direction fails.

I checked each test fails without the corresponding part of the change — the boundary tests fail on main, and the marker tests fail against an earlier revision of this branch that had the re-emission without the guard.

Verified on main at fdab7f5:

  • python run_tests.py3965/3965 passed (100.0%), 6 skipped; baseline on main is 3957/3957, so +8 is exactly the new tests
  • coverage run run_tests.py && coverage report100%, statements and branches, with both new conditions exercised in both directions
  • ruff check, ruff format --check, mypy src — clean
  • benchmarks/html5lib_engine_diff.py --fail-under-rate 1.0 --fail-on-current-exceptions — exit 0

I also ran a differential over ~68,000 generated cases (inline elements × whitespace shapes including empty and whitespace-only content × block contexts, in both compact and html_passthrough mode), comparing output against main. Every difference is either a separating space restored at an element boundary or a duplicate space collapsed — no case differs in a non-whitespace character, none loses a separator main had, and none gains a double space.

One thing I noticed while testing and deliberately left alone: <ol> nested-list indentation is already off on main (1. a\n\n 1. n flattens under CommonMark, because a 2-space indent doesn't clear an ordered marker's content column). It's unrelated to this change and unaffected by it — happy to file it separately if useful.

Disclosure: written with AI assistance (Claude Code). Every repro and every number above was executed locally against this branch; I've reviewed the change and can speak to it.

`to_markdown()` dropped the space that sits just inside an inline
element, welding the words on either side together:

    <p>This is <b>very </b>important.</p>
      -> This is **very**important.
    <p>Visit<a href="https://e.com"> our site</a> today.</p>
      -> Visit[our site](https://e.com) today.

Both render as one word, while `to_text()` on the same input returns
"This is very important." and "Visit our site today.".

`a`, `em`, `i`, `strong`, and `b` render their children into a separate
`_MarkdownBuilder`, so the boundary whitespace never reaches the parent:
a trailing space is still held in the child's `_pending_space` when
`finish()` runs, and a leading space is discarded because the child's
buffer is empty at that point. `span` and the other inline containers
share the parent builder and already came out correctly, so the two
spellings of the same document disagreed.

Record on the builder whether whitespace was collapsed before any output
and re-emit both boundaries into the parent when the marker or link is
closed.

Re-emitting alone injects a second space after a structural marker,
because both places the builder can flush a pending space, `raw()` and
`text()`, appended one whenever the buffer was non-empty without
checking what it ended with, and headings and list items write `"## "`,
`"- "`, and `"1. "` through `raw()`. In a list that destroys structure
rather than spacing: the extra space moves the item content to column 3
while nested lists indent by 2, so the inner list in
`<ul><li><b> outer</b><ul><li>inner</li></ul></li><li>second</li></ul>`
no longer clears its parent item and renders as a flat sibling. Both
flushes now skip a buffer that already ends in a space or tab. Guarding
them, rather than the two re-emission sites, keeps the rule in the
builder and lets the boundary space propagate out of nested inline
elements as ordinary collapsed whitespace.

The predicate also covers whitespace that predates this change, so
`<ul><li> Item</li></ul>` renders as `- Item` rather than `-  Item`.
Across a 37,632-case matrix of inline elements, whitespace shapes and
block contexts, every difference from the previous output is either a
separating space restored at an element boundary or a duplicate space
collapsed; nothing differs in any other character.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@EmilStenstrom

Copy link
Copy Markdown
Owner

Thank you for the report. Will re-implement in a new PR, but really appreciate the report. Good find!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants