Skip to content

fix(delta): detect GFM tables without leading/trailing pipes - #3432

Open
handnewb wants to merge 3 commits into
vectorize-io:mainfrom
handnewb:fix/gfm-table-lenient-parser
Open

fix(delta): detect GFM tables without leading/trailing pipes#3432
handnewb wants to merge 3 commits into
vectorize-io:mainfrom
handnewb:fix/gfm-table-lenient-parser

Conversation

@handnewb

Copy link
Copy Markdown
Contributor

Summary

The table parser in structured_doc.py required pipes at both start AND end of every row (^\\s*\|.*\|\\s*$). GFM-compliant tables without outer pipes (e.g. Header | Header\n---|---\nCell | Cell) fell through to ParagraphBlock which joins lines with spaces, permanently destroying table structure during delta refreshes.

Changes (1 file, +25/-1 lines)

  • structured_doc.py: Add lenient fallback that detects GFM tables by:
    1. Finding a separator line with 3+ dashes (optional colons for alignment)
    2. Checking every non-separator line for at least one pipe character
    3. Routing to _parse_table_block instead of ParagraphBlock

Before/After

Input Before After
`Header | Header\n--- ---\nCell | Cell` "Header | Header Cell | Cell" (paragraph)

Closes #3361

The table parser required pipes at both start AND end of every
row (^\s*\|.*\|\s*$). GFM-compliant tables without outer
pipes fell through to ParagraphBlock which joins lines with
spaces, permanently destroying table structure during delta
refreshes.

Add a lenient fallback that:
1. Detects a separator line with 3+ dashes (with optional colons)
2. Checks that every non-separator line contains at least one pipe
3. Routes to _parse_table_block instead of ParagraphBlock

This preserves tables like:
  Header | Header
  ---|---
  Cell | Cell

Closes vectorize-io#3361
ruff formatter (v0.14.9, project-pinned version) collapses
multi-line list comprehensions into single-line expressions
when they fit within the 120-char line limit.
The original regex r"^\s*\|?[\s:]*-{2,}..." had [\s:]* (zero or
more spaces/colons) between the optional pipe and dashes. A previous
edit accidentally changed it to [\s:] (exactly one), which would
break tables with separators like |---| (no space between pipe and
dashes). Restore the original * quantifier.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Delta refresh of a knowledge page collapses stored markdown tables onto one line, permanently

1 participant