Repository navigation
fix(compat): close 21 defects blocking real MySQL clients - #171
Merged
Merged
Conversation
MySQL has no transactional DDL. CREATE, ALTER, and DROP are on its implicit-commit list: each ends the transaction a session has open before it runs, then runs on its own, leaving no transaction open. Clients written against MySQL depend on that, and schema migration tools rely on it directly - a migration adds a column and then updates it in what looks like one transaction, which works only because the DDL committed first. Marmot buffered every statement in an explicit transaction until COMMIT, so DML could not see a column added earlier in the same transaction. The statement failed with "no such column", and no wire-protocol work could fix it. LLDAP's v2 migration is exactly this shape and could not run. DDL in an open transaction now commits the buffered statements and closes the transaction before executing, controlled by transaction. ddl_implicit_commit and defaulting to true for MySQL compatibility. Set it false to keep DDL inside the transaction and replicate it atomically with the surrounding statements, accepting that DML in that transaction cannot see the pending schema change; a test pins that trade-off. Write forwarding needed no change: ForwardQueryResponse.InTransaction is read from the leader's session state after execution, so a replica observes the implicit commit without being told about it. Also fixes the coordinator test harness, which never initialized the query pipeline and left TranspilationEnabled false on its sessions. BEGIN parsed as StatementUnsupported there, so tests meaning to exercise the explicit-transaction path silently ran in autocommit and passed for the wrong reason. Both are corrected, which is what surfaced this defect. The setting is documented in every shipped config with a [transaction] section and in the configuration reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015eBWcTYK93Zy5KAAwLMzZc
Found by driving LLDAP 0.6.3 (Rust, sqlx + sea-orm) against a live node
and auditing the paths it exercised. Each fix is covered by tests that
fail on the prior code.
CDC correctness
- Replicate rowid as identity for tables with no declared PRIMARY KEY;
UPDATE/DELETE on such tables previously failed to replicate at all.
Refuse capture when a column shadows rowid/oid/_rowid_.
- Preserve BLOB vs TEXT storage class across replication. go-sqlite3
hands both back as []byte, so the choice is now made at capture time
from column affinity (only BLOB affinity stays msgpack Bin), with a
strict CDC decode that no longer coerces Bin to string.
- Refuse capture on tables with VIRTUAL generated columns instead of
segfaulting: go-sqlite3's preupdate row() dereferences a NULL value
for virtual column indices.
- Use PRAGMA table_xinfo true column positions. table_info renumbers
cid when hidden columns exist, which silently broke PK indexing.
- Propagate schema-cache reload failures after DDL instead of logging
and continuing with a stale cache.
- Fail loud on schema/value count mismatch rather than truncating.
- Log zero-rows-affected CDC applies at Debug so divergence is
diagnosable; add FK ON DELETE CASCADE convergence tests.
Interactive transactions
- Execute DML eagerly against a pinned SQLite transaction instead of
buffering until COMMIT, so clients get real rows-affected and
last_insert_id and can read their own writes. CDC is captured
incrementally and replayed through the existing 2PC path at COMMIT;
the pinned transaction is always rolled back locally.
- Release pinned sessions before 2PC to avoid a single-writer deadlock.
- Roll back pinned state on client disconnect, 2PC rejection and
rollback; fix a forwarded-session leak that never released
transaction state.
- Serialize forward-session eviction against in-flight statements. An
eviction racing a slow COMMIT could discard captured CDC and let the
commit report success with the write lost. COMMIT now fails loudly if
pinned state disappears rather than reporting an empty transaction.
- Add transaction.ddl_implicit_commit (default true): DDL commits the
open transaction as MySQL does.
SQL transpilation
- Reject partially-parsed DDL instead of forwarding a truncated
statement into 2PC. Vitess returns a partial AST on syntax errors.
- Serialize HAVING as HAVING. Vitess models WHERE and HAVING with the
same node, and every HAVING was being emitted as WHERE.
- Transpile index DDL through one path: ADD CONSTRAINT UNIQUE, ADD
INDEX, ADD UNIQUE and standalone CREATE [UNIQUE] INDEX all emit
SQLite CREATE INDEX; DROP INDEX ... ON drops the ON clause and
handles backtick-quoted names.
- Strip CHARACTER SET/COLLATE on ALTER TABLE ADD/MODIFY/CHANGE COLUMN,
sharing the CREATE TABLE implementation.
MySQL wire protocol
- Report real column count, names and types in COM_STMT_PREPARE. sqlx
builds its column index from the prepare response, so an empty one
made every prepared query fail to find its columns.
- Infer column types from data and declared types instead of always
VAR_STRING; encode time.Time and binary DATETIME correctly.
- Set SERVER_STATUS_IN_TRANS so clients can see transaction state.
- Implement COM_STMT_SEND_LONG_DATA (no response, per-parameter
accumulation) and COM_STMT_RESET; the unhandled default previously
sent an ERR packet and desynced the stream.
- Honor the unsigned parameter flag; values above MaxInt64 now return
ER_WARN_DATA_OUT_OF_RANGE instead of a driver error or a negative.
Parameter handling
- Merge wire parameters with extracted literals positionally via
Statement.MergeExecParams. Literal extraction turns the injected
auto-increment id into a placeholder, so a prepared INSERT needed
more arguments than the client bound ("want 6 got 5"). Every
execution site now uses the shared helper.
- Add Statement.WithResolvedParams for rewritten statements so stale
placeholder provenance cannot misinterleave already-resolved args.
Test harness
- Initialize the coordinator test pipeline with a real ID generator and
enable transpilation. Auto-increment injection and BEGIN were both
inert in tests, which is why several of these defects shipped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015eBWcTYK93Zy5KAAwLMzZc
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by driving LLDAP 0.6.3 (Rust,
sqlx+sea-orm) against a live Marmot node and auditing the code paths it exercised. Each fix is covered by tests that fail on the prior code.Result: LLDAP went from dying on its second schema migration to completing all 11 migrations. It is still blocked afterward by #170 (generated ids overflow its 32-bit id type), which is a design decision, not a defect in this branch.
CDC correctness
PRIMARY KEYcould not replicateUPDATE/DELETEat all. The capture side never emitted the rowid the apply side required. Rowid is now the replicated identity; capture is refused when a column shadowsrowid/oid/_rowid_.go-sqlite3hands TEXT and BLOB back identically as[]byte, so the distinction was already lost before decoding. The choice is now made at capture time from column affinity (only BLOB affinity stays msgpackBin), with a strict CDC decode that no longer coercesBinto string. Would have corrupted password hashes and UUID columns.VIRTUALgenerated column —go-sqlite3's preupdaterow()dereferences a NULL value for virtual column indices. Capture now refuses such tables before touchingOld()/New().PRAGMA table_inforenumberscidwhen hidden columns exist, which silently broke PK positional indexing. Now usestable_xinfotrue positions. (Latent pre-existing bug.)ON DELETE CASCADEconvergence tests.Interactive transactions
DML inside
BEGIN...COMMITwas buffered untilCOMMITand answered with a fabricatedRowsAffected: 1, so ORMs got no reallast_insert_idand no read-your-own-writes.COMMIT. The pinned transaction is always rolled back locally — the write lands only via CDC replay.COMMITcould discard captured CDC, after whichCOMMITreported success with the write silently lost.COMMITadditionally fails loudly if pinned state disappears rather than treating it as an empty transaction.transaction.ddl_implicit_commit(defaulttrue): DDL commits the open transaction, as MySQL does.SQL transpilation
alter table users add CONSTRAINT ... UNIQUE (email)becamealter table users. Now rejected with a clean parse error.HAVINGwas emitted asWHERE. Vitess models both with the same node and the serializer ignored the discriminator, so anyGROUP BY ... HAVING(including inside subqueries) produced invalid SQL.ADD CONSTRAINT UNIQUE,ADD INDEX,ADD UNIQUEand standaloneCREATE [UNIQUE] INDEXall emit SQLiteCREATE INDEX;DROP INDEX ... ONdrops theONclause and handles backtick-quoted names.CHARACTER SET/COLLATEare stripped onALTER TABLE ADD/MODIFY/CHANGE COLUMN, sharing theCREATE TABLEimplementation.MySQL wire protocol
COM_STMT_PREPAREreported zero columns.sqlxbuilds its column-name index from the prepare response, so every prepared query failed to find its columns — this was the actual cause of the reported client panic. Now sends real column count, names and types.VAR_STRING;time.Timeand binaryDATETIMEencode correctly.SERVER_STATUS_IN_TRANSis now set so clients can see transaction state.COM_STMT_SEND_LONG_DATA(no response, per-parameter accumulation, cleared only byRESET/CLOSE) andCOM_STMT_RESET; the unhandled default sent an ERR packet and desynced the connection.MaxInt64returnER_WARN_DATA_OUT_OF_RANGErather than a confusing driver error or a negative value.Parameter handling
not enough args to execute query: want 6 got 5— literal extraction turns the injected auto-increment id into a placeholder, but every execution site chose either wire params or extracted params, never both. AddedStatement.MergeExecParams, which interleaves them by per-placeholder provenance (proven by a test where the literal precedes the bound param).Statement.WithResolvedParamsso rewritten statements cannot carry stale placeholder provenance and misinterleave already-resolved args.Test harness
The coordinator test package initialized the pipeline with a nil ID generator and left transpilation disabled, so auto-increment injection and
BEGINwere both inert in tests — which is why several of these defects shipped. Both fixed; full suite still green.Verification
go buildclean with the required CGO tags.-raceclean oncoordinator,protocol/...,grpc,encoding,cfg.go vet: only the two pre-existing findings (one filed as vet copylocks: cloneForwardResponse copies a protobuf message by value #169).gofmtclean.Follow-ups filed
#164, #165, #166, #167, #168, #169, #170
🤖 Generated with Claude Code
https://claude.ai/code/session_015eBWcTYK93Zy5KAAwLMzZc