You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c51a246
Browse filesBrowse the repository at this point in the historyBrowse files
feat: support MySQL TABLE and VALUES queries with query CTEs
Parse TABLE name and VALUES ROW(...) as structured query expressions,
including set operations, ordered/limited branches, scalar subqueries,
IN/EXISTS predicates, derived tables, and LATERAL sources. Retain explicit
ROW syntax with a node-local flag so ordinary INSERT VALUES emission is
unchanged. Preserve set-operation precedence and parenthesized grouping.
Replace the permissive MySQL WITH path with a shared query-only parser
that requires names, AS, delimiters, complete bodies, and a main query.
Preserve quoted CTE names, output-column lists, and RECURSIVE. CTE-prefixed
UPDATE and DELETE remain explicit unsupported errors rather than partial
query trees. Production parsing has no MySQL server dependency.
Extend query locking tails to TABLE/VALUES and compound queries, retaining
repeated clauses and LOCK IN SHARE MODE. Distinguish VALUES(col) in existing
INSERT updates from a VALUES query. Inspect parenthesized query boundaries
in a linear scan without discarding EXISTS bodies or consuming enclosing
scalar arithmetic and IN-list expressions as queries.
Validate MySQL LIMIT operands as unsigned 64-bit integers, placeholders,
or identifier references. Reject wildcard values and DEFAULT embedded in
expressions while retaining standalone DEFAULT and empty ROW constructors:
those forms are grammar-valid but native semantic errors. Keep quoted star
identifiers and COUNT(*) intact. Guard new query constructors and legacy
share-lock syntax against unsupported local execution; parameterization
preserves ROW spelling and ORDER BY constants.
Expand the compatibility corpus from 74 to 113 independently specified
witnesses. Promote TABLE, VALUES, and recursive CTE coverage, retain the
remaining 17 known gaps, and verify 110 grammar anchors per pinned version.
Document native semantic errors separately from syntax rejection. Add 13
focused C++ regression tests for strict parsing, faithful emission, nested
query boundaries, parameterization, and planner guards.
Validation:
- Forced full build and make test: 1,552 passed, 37 external DB tests skipped.
- ASan/UBSan with leak detection: all 36 focused MySQL tests passed.
- All 113 corpus expectations match: 62 CHECKED, 34 malformed controls
REJECTED, and 17 explicitly tracked gaps; 11 harness tests passed.
- Native MySQL 8.4.8 and 9.7.2 report zero validity mismatches. Semantic
errors and three skipped DDL cases remain separately unresolved.
- PostgreSQL harness: 87 passed; 5,122 committed fixtures and the full
51,415-statement pinned baseline match unchanged.
- Independent final review verifies faithful stable emission for 14
affected queries; seven parser/emitter benchmark smoke cases passed.
Benchmark code, reports, and benchmark-only README/.gitignore edits are
excluded from this commit.
Copy file name to clipboardExpand all lines: docs/mysql-compatibility.md
+8-4Lines changed: 8 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,11 @@ This harness tracks a small, deliberately selected set of MySQL syntax witnesses
4
4
5
5
ParserSQL now preserves MySQL JSON arrows, named/inherited windows, `GROUP_CONCAT` modifiers and separators, and `GROUP BY ... WITH ROLLUP` as structured ASTs. Database-qualified column names are retained. The local engine rejects these unsupported semantics rather than executing a simplified interpretation; parameterization remains conservative for JSON arrows and `GROUP_CONCAT`. Empty MySQL SELECT target lists are rejected.
6
6
7
-
ParserSQL also preserves partition lists, `USE`/`FORCE`/`IGNORE INDEX` or `KEY`, hint scopes and empty `USE INDEX ()`, and `LATERAL` derived SELECTs with required aliases and optional alias-column lists. The local engine rejects these unsupported table-reference semantics. `SELECT` and `UPDATE` place `PARTITION` before the table alias; single-table `DELETE` places it after the alias. The current LATERAL implementation scope is SELECT-based queries, including UNION: both native releases also accept `LATERAL (TABLE t) AS q` and `LATERAL (VALUES ROW(1)) AS q`, but those forms remain outside this implementation and are not malformed controls.
7
+
ParserSQL also preserves partition lists, `USE`/`FORCE`/`IGNORE INDEX` or `KEY`, hint scopes and empty `USE INDEX ()`, and `LATERAL` derived SELECTs with required aliases and optional alias-column lists. The local engine rejects these unsupported table-reference semantics. `SELECT` and `UPDATE` place `PARTITION` before the table alias; single-table `DELETE` places it after the alias. LATERAL derived queries include `SELECT`, `TABLE`, and `VALUES ROW(...)` expressions and their set operations.
8
+
9
+
MySQL query expressions also preserve explicit `TABLE name` and `VALUES ROW(...)` constructors in standalone, compound, derived, LATERAL, scalar, IN, EXISTS, and query-only CTE contexts. `INTERSECT` binds more tightly than `UNION` and `EXCEPT`; explicit parentheses and branch-local ordering/limits are retained. Recursive CTEs retain their keyword and optional column lists. Query locking clauses, including `LOCK IN SHARE MODE`, remain structured. This is parser/AST coverage; the local engine rejects unsupported query forms.
10
+
11
+
The native grammar permits empty `VALUES ROW()`, `VALUES ROW(DEFAULT)`, and repeated locking clauses. Both pinned servers return semantic errors 3942, 3943, and 3569 respectively for the curated standalone or overlapping-lock witnesses. These remain `SERVER_ERROR` observations, not syntax rejections, while ParserSQL preserves the grammar-valid ASTs. `TABLE` takes a table identifier and query-level tails; aliases, partitions, index hints, and WHERE clauses directly after its table name are malformed controls. The [8.4.8 query-primary productions](https://github.com/mysql/mysql-server/blob/0896fcd61dec11a0904166911a0126f59daaa1bf/sql/sql_yacc.yy#L9831) and [9.7.2 query-primary productions](https://github.com/mysql/mysql-server/blob/008e09c2834b98143a8c067d4d225c90953050cf/sql/sql_yacc.yy#L10145) include both constructors. The corpus records exact anchors for their rows, subquery contexts, locking, CTEs, and malformed controls.
8
12
9
13
Table identifiers and aliases use the common 259-word reserved baseline observed in the pinned 8.4.8 and 9.7.2 releases. ParserSQL has no MySQL version setting, so the five 9.7-only reserved additions (`CUBE`, `EXTERNAL`, `LIBRARY`, `QUALIFY`, and `TABLESAMPLE`) remain accepted as names. Strict version-specific identifier rules remain a gap.
10
14
@@ -42,7 +46,7 @@ Malformed controls are handled separately: a complete AST is `MALFORMED_ACCEPTED
42
46
43
47
Every fixture freezes one exact expected outcome. Known gaps do not make arbitrary failures acceptable. Both regression and unexpected improvement cause exit status 1, so a fixed gap must be reviewed and explicitly promoted in the corpus. When promoting, add an independently derived `expected_sql`, set `known_gap=false`, and require `CHECKED`. Harness/transport/source errors return 2. Exit status 0 means the expected baseline matched, including its documented known gaps.
44
48
45
-
The 74-case baseline has 33`CHECKED` witnesses and 21`REJECTED` negative controls. Its remaining known gaps are 13`TRAILING_INPUT`, five`NO_AST`, one `PARTIAL`, and one `EMISSION_MISMATCH`. Source verification checks 71 exact grammar anchors per pinned release. These counts describe this corpus only.
49
+
The 113-case baseline has 62`CHECKED` witnesses and 34`REJECTED` negative controls. Its remaining known gaps are ten`TRAILING_INPUT`, three`NO_AST`, two `ERROR`, one `PARTIAL`, and one `EMISSION_MISMATCH`. The two `ERROR` witnesses are CTE-prefixed UPDATE/DELETE: the query-only WITH parser now rejects these unsupported main statements directly. Source verification checks 110 exact grammar anchors per pinned release. These counts describe this corpus only.
46
50
47
51
The JSON report includes all original and reparsed records, source verification counts, pinned revisions, SQL-mode metadata, and SHA-256 hashes of the corpus and probe. Counts apply only to these witnesses. The probe uses hex-line input and hex fields in its TSV response, preserving tabs, newlines and embedded NUL bytes. It emits while both the input string and parser arena are alive; Python reparses owned copies in a second probe invocation.
48
52
@@ -61,9 +65,9 @@ Each curated case explicitly chooses direct query execution, `EXPLAIN` for DML (
61
65
62
66
Native results are stored separately in `native_oracle`: `ACCEPTED`, `SYNTAX_REJECTED` (1064/1149), `SERVER_ERROR`, `CLIENT_ERROR`, or **NOT VERIFIED**. For example, a derived table missing its required alias returns error 1248; this remains a server error, even though the corpus requires ParserSQL to reject that input. Accepted invalid controls and syntax-rejected valid controls count as expectation mismatches, using per-version validity when specified. Server errors remain explicitly unresolved; client errors return status 2. Thus exit status 0 still does not mean all native cases were verified. The default corpus metadata remains **NOT VERIFIED**, while the optional report describes the actual native observations.
63
67
64
-
A native check of all 74 witnesses under `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION` found no native expectation mismatches. MySQL 8.4.8 accepted 46, syntax-rejected 24, and returned error 1248 for the missing-alias control; MySQL 9.7.2 accepted 48, syntax-rejected 20, and returned three server errors (1248, 1235, and 3889). Both runs skipped the three DDL/procedure witnesses and cleaned up their owned schemas. All shared positive query/DML witnesses, including partition selection and index hints, were accepted. Re-run the command on each isolated socket to reproduce current observations; reports are intentionally not committed.
68
+
A native check of all 113 witnesses under `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION` found no native expectation mismatches. MySQL 8.4.8 accepted 69, syntax-rejected 36, and returned five server errors (two 1248 missing-alias errors, 3942, 3943, and 3569); MySQL 9.7.2 accepted 71, syntax-rejected 32, and returned seven server errors (the same five plus 1235 and 3889). Both runs skipped the three DDL/procedure witnesses and cleaned up their owned schemas. All shared positive query/DML witnesses except the three explicitly grammar-valid semantic-error controls were accepted, including partition selection, index hints, query constructors, and recursive CTEs. Re-run the command on each isolated socket to reproduce current observations; reports are intentionally not committed.
65
69
66
-
The corpus is deliberately selected and is not a representative workload. It contains 70 shared witnesses and four version-specific query witnesses; this does not imply identical dialects in 8.4.8 and 9.7.2. The version-specific entries record `applicability`, source productions for both versions, `native_validity` per version, and observed native outcomes. Their generic `validity=valid` denotes the 9.7.2 grammar target. The native helper selects the intended validity using the actual server version, so an expected 8.4.8 syntax rejection is not treated as a corpus error. Stored native observations are context, not substitutes for a current run. These additions remain ParserSQL gaps. The following differences include those four query witnesses and two separate DDL spot checks:
70
+
The corpus is deliberately selected and is not a representative workload. It contains 109 shared witnesses and four version-specific query witnesses; this does not imply identical dialects in 8.4.8 and 9.7.2. The version-specific entries record `applicability`, source productions for both versions, `native_validity` per version, and observed native outcomes. Their generic `validity=valid` denotes the 9.7.2 grammar target. The native helper selects the intended validity using the actual server version, so an expected 8.4.8 syntax rejection is not treated as a corpus error. Stored native observations are context, not substitutes for a current run. These additions remain ParserSQL gaps. The following differences include those four query witnesses and two separate DDL spot checks:
67
71
68
72
| Reduced feature | MySQL 8.4.8 observation | MySQL 9.7.2 observation |
0 commit comments