fix(splitter): preserve blank lines in parsable SQL - #788
Open
edjubert wants to merge 4 commits into
Open
Conversation
The named-parameter normalization is purely lexical and is about to be needed by the statement splitter as well. Moving it to pgls_lexer keeps the splitter and the workspace in agreement about what counts as parsable SQL. Pure move: no signature, body or comment changed.
Splitter::new now splits on ';' and structural statement starts only; Splitter::with_blank_line_boundaries keeps the historical behaviour. No behaviour change: split() opts into blank-line boundaries, so the whole test suite is unchanged. This only puts the switch in place.
split() now runs two passes. The first splits on ';' and on structural statement starts, ignoring blank lines. The second asks PostgreSQL about each fragment: a fragment that parses is kept whole, unless splitting it on its blank lines yields several statements that are each valid on their own, which is what a missing ';' between two statements looks like. A blank line inside a statement is a common readability convention and is not a terminator in PostgreSQL, so valid SQL is no longer cut in two and reported as a syntax error. Error recovery while typing is preserved, but scoped to the fragment being edited instead of the whole file. Fixes supabase-community#784.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Bug fix.
What is the current behavior?
The statement splitter treats blank lines as statement boundaries. Valid SQL formatted
with blank lines can therefore be split into invalid fragments and reported as syntax
errors.
Fixes #784.
This also addresses the underlying behavior reported in #372, #380, and #780.
What is the new behavior?
Statement splitting now uses two passes:
;and structural statement starts while ignoring blank lines.parse the fragment:
A blank line now splits SQL only when the complete fragment cannot be parsed.
This preserves recovery for incomplete SQL while typing, including cases such as a
missing
;beforeBEGIN.Additional context
the recovery-path token scan.
valid statements.
Added coverage for:
FROMclause;BEGIN.Validated with: