Fix 53 parser issues: keyword types, implicit aliases, empty tuples, CAST syntax #13
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…CAST syntax - Fix parseColumnDeclaration to accept keywords as column names (KEY, etc.) - Fix parseDataType to accept keyword type names (Array, Tuple, Nested, etc.) - Add more data types: INT, JSON, VARIANT, DYNAMIC, aggregation functions, etc. - Support empty tuple parsing: SELECT () - Support CAST(x, 'Type') comma syntax in addition to CAST(x AS Type) - Support WITH TIES clause after LIMIT - Add implicit alias support for columns (SELECT 'a' c0 means SELECT 'a' AS c0) Tests: 5405 passing, 1418 skipped (was 5352 passing, 1471 skipped)
… IGNORE NULLS - Add NAN and INF as special float literal values - Allow keywords to be used as identifiers in expressions (ORDER BY ALL, etc.) - Support IGNORE NULLS / RESPECT NULLS window function modifiers - Add SETTINGS support inside table function arguments (icebergS3, etc.) Tests: 5446 passing, 1377 skipped (fixed 94 total tests from original 1471)
…T BY, EPHEMERAL - Engine names can be keywords (Null, Join, Memory, etc.) - Special handling for Nested type which contains column declarations - Allow boolean settings without values (just setting name) - Support LIMIT BY clause for ClickHouse-specific syntax - Handle EPHEMERAL column modifier - Fix DROP with NO DELAY modifier - Support DROP ROW POLICY, DROP SETTINGS PROFILE, DROP NAMED COLLECTION - Handle multiple ON clauses in DROP - Fix keyword aliases after AS keyword - Support CREATE TABLE AS table_function() Tests: 5460 passing, 1363 skipped (fixed 108 total tests from original 1471) Parser failures reduced from 273 to 239.
…EATE TABLE - Handle SETTINGS clause before VALUES in INSERT statements - Skip VALUES data properly in INSERT statements - Support INDEX and CONSTRAINT definitions in CREATE TABLE column list Parser failures reduced from 239 to 235.
- Add MarshalJSON for Literal to handle NaN/Inf values in JSON serialization - Support INTERVAL and FORMAT keywords as identifiers when not used as keywords - Handle uint64 numbers that exceed int64 range - Add InsertQuery explain output support - Expand nested DataType parameters in explain output (e.g., Array(DateTime)) - Fix FormatLiteral to handle both int64 and uint64 integer types Test count: 5488 passing (up from 5461)
- Always expand DataType parameters as children in explain output - Add HasSettings field to InsertQuery for SETTINGS clause tracking - Output Set child in InsertQuery explain when settings are present Test count: 5505 passing (up from 5488)
- Add explainWithElement function to handle WITH clause expressions - Add explainFunctionCallWithAlias helper for function calls with aliases - Update select.go to output WITH clause ExpressionList - Update explain.go to handle *ast.WithElement nodes Test count: 5505 -> 5517 (+12 tests)
- Support IN with array literals: `expr IN [1, 2, 3]`
- Support IN with identifiers: `expr IN tablename`
- Fix function call alias handling to respect expression precedence
- Add SETTINGS clause after FORMAT in SELECT
- Fix multiple WITH clause elements (comma-separated)
This allows parsing queries like:
- SELECT 1 IN [1, 2]
- WITH toDate('2023-01-09') AS d1, d1 - 1 AS d2 SELECT d1, d2
Test count: 5517 -> 5542 (+25 tests)
- Support hex literals: 0xDEADBEEF, 0x123abc - Support binary literals: 0b101010 - Support octal literals: 0o755 - Support underscore separators in numbers: 100_000_000 (underscores are only allowed between digits, not at boundaries) Test count: 5542 -> 5547 (+5 tests)
- Add CUBE token to lexer - Add WithCube field to SelectQuery AST - Parse WITH CUBE after GROUP BY similar to WITH ROLLUP Test count: 5547 -> 5549 (+2 tests)
- Support x'...' and X'...' hex string literals - Support b'...' and B'...' binary string literals - These are converted to regular string tokens Test count: 5549 -> 5550 (+1 test)
WITH TOTALS can appear after GROUP BY or at the end of SELECT even without GROUP BY. Added parsing support after LIMIT clause. Test count: 5550 -> 5554 (+4 tests)
- Add HasParentheses field to DataType to track empty parentheses (e.g., Tuple()) - Add NameTypePair AST node for Nested type column declarations - Update array expansion to treat tuple/array literals as complex expressions - Change boolean literal format from UInt8 to Bool - Fix InsertQuery to include Settings from SELECT clause as child - Add nil safety checks for InsertQuery SELECT parsing
- Remove special handling for :: operator syntax in CAST - always render expression as AST node - Add DIV -> intDiv operator mapping - Add MOD -> modulo operator mapping (in addition to %)
- Convert interval unit to title case (e.g., YEAR -> Year) - Add safety check for empty unit string
For :: operator syntax with simple literals, format as string literal For function syntax or complex expressions, use normal AST node
Output PRIMARY KEY clause in Storage definition section
- Strip server error messages from expected explain output - Fix CreateQuery spacing for CREATE DATABASE (add extra space) - Fix DropQuery spacing for DROP DATABASE vs DROP TABLE This fixes 243 tests (5606 -> 5849 passing).
- Add PARTITION BY output to CREATE TABLE Storage definition - Treat nested tuple/array literals as complex expressions for expansion This fixes 20 more tests (5849 -> 5869 passing).
Added proper alias handling in explainAliasedExpr for: - Binary expressions (e.g., `x % 2 AS i`) - Unary expressions (e.g., `-x AS neg`) - Function calls (e.g., `sum(x) AS total`) - Identifiers (e.g., `x AS y`) This fixes 32 more tests (5869 -> 5901 passing).
Use strconv.FormatFloat with 'f' format to display float literals as full decimal values instead of scientific notation. This fixes 3 more tests (5901 -> 5904 passing).
- Add WindowDefinition output for functions with OVER clause - Fix float literal formatting to avoid scientific notation This fixes 6 more tests (5904 -> 5910 passing).
…, and SETTINGS placement - Remove join type from TableJoin output (ClickHouse doesn't show it) - Add alias support for table functions in TableExpression - Fix array/tuple literal formatting in :: cast expressions - Move SETTINGS (Set) to SelectWithUnionQuery level when FORMAT is present
Tables with aliases in FROM clause now show (alias xxx) in TableIdentifier output.
ClickHouse EXPLAIN AST flattens multiple || operators into a single concat function with all operands as children. This handles BinaryExpr, AliasedExpr, and WithElement cases.
- Updated test counts: 5,933 passing (86.9%), 891 skipped (13.1%) - Added "Recently Fixed" section for completed explain layer fixes - Categorized parser issues by priority (high/medium/low) - Documented specific examples and expected vs actual output - Added remaining explain issues that depend on parser changes
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tests: 5405 passing, 1418 skipped (was 5352 passing, 1471 skipped)