Leading comment support added for AlterTable
, CreateTable
, and ColumnDef
#2069
+469
−115
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR refactors the internal representation of comments in the tokenizer and parser and adds leading comment support for
AlterTable
,CreateTable
, andColumnDef
.The
SingleLine
andMultiLine
comment variants previously defined in theWhitespace
enum are now represented by a new, dedicatedComment
enum. TheComment
is used for both whitespace comments (interstitial comments) and leading comments. Leading comment support propagated forAlterTable
,CreateTable
, andColumnDef
, to parse leading comments alongside the associatied struct (see below for example).Rationale
This change improves the semantic clarity of comment handling in the SQL tokenizer and parser.
As discussed in #2065, comments preceding a table or column definition may serve as inline documentation, and should be distinguishable from interstitial (whitespace) comments.
For example:
Term Definitions
Intersitial Comment: a comment preceded by something (if nothing then defined as leading comment) that is not a comma or semicolon:
Leading Comment: a comment that is preceded by either nothing, a comma, or a semicolon.
currently the variants covered include single line comments:
and multi-line comments:
By separating comment handling from generic whitespace, the parser can now support more context-aware interpretations and contribute to a lossless syntax tree, addressing #175 and complementing PR #189.
Summary of Changes
Comment
enum encapsulatingSingleLine
andMultiLine
comment variants.Whitespace
to includeInterstitialComment(Comment)
variant.Tokenizer.rs
to emitComment
values instead ofWhitespace::SingleLineComment
/Whitespace::MultiLineComment
.parser/mod.rs
implementedLeadingComment
forCreateTable
,ColumnDef
,AlterTable
parsing.leading_comment: Option<Comment>
to theCreateTable
,ColumnDef
,AlterTable
:InterstitialComment
andLeadingComment
.