Repository navigation
fix: drop pool_pre_ping, it is unusable on aiomysql - #25
Merged
Merged
Conversation
Every ETL job has failed since the last deploy — 8 of 8 upserts plus the
dimensional refresh, which had never failed before:
sqlalchemy/dialects/mysql/pymysql.py, in do_ping
dbapi_connection.ping()
TypeError: AsyncAdapt_aiomysql_connection.ping() missing 1 required
positional argument: 'reconnect'
SQLAlchemy's pymysql dialect chooses its ping call by inspecting PyMySQL's
signature: with `reconnect` defaulting to True it sends ping(False), otherwise
ping(). The aiomysql async adapter declares ping(self, reconnect) with no
default, so the second form always raises — and every pooled checkout goes
through it once pool_pre_ping is on. The first connection of each job succeeds
(new connection, not pinged) and the reused one dies, which is why jobs failed
~4s in.
PyMySQL was unpinned (transitive via aiomysql), so it resolved to 1.1.2 locally
and 1.2.0 in the image, and 1.2.0 is where the `reconnect` default flipped to
False. That is why this passed local verification and broke in production; the
check should have run in the image, and now does.
Replaces it with pool_recycle=600, which addresses the original problem — a
connection dropped while idle being reused ~12 minutes later — without touching
the ping path, and works under both PyMySQL versions. Verified in
python:3.12-slim against MySQL:
PyMySQL 1.2.0 pool_pre_ping=True -> TypeError (reproduces prod)
PyMySQL 1.2.0 pool_recycle=600 -> OK
PyMySQL 1.1.2 pool_recycle=600 -> OK
PyMySQL is pinned as well, so local and the image stop diverging.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every ETL job has failed since the last deploy — 8 of 8 upserts plus the dimensional refresh, which had never failed before:
sqlalchemy/dialects/mysql/pymysql.py, in do_ping
dbapi_connection.ping()
TypeError: AsyncAdapt_aiomysql_connection.ping() missing 1 required
positional argument: 'reconnect'
SQLAlchemy's pymysql dialect chooses its ping call by inspecting PyMySQL's signature: with
reconnectdefaulting to True it sends ping(False), otherwise ping(). The aiomysql async adapter declares ping(self, reconnect) with no default, so the second form always raises — and every pooled checkout goes through it once pool_pre_ping is on. The first connection of each job succeeds (new connection, not pinged) and the reused one dies, which is why jobs failed ~4s in.PyMySQL was unpinned (transitive via aiomysql), so it resolved to 1.1.2 locally and 1.2.0 in the image, and 1.2.0 is where the
reconnectdefault flipped to False. That is why this passed local verification and broke in production; the check should have run in the image, and now does.Replaces it with pool_recycle=600, which addresses the original problem — a connection dropped while idle being reused ~12 minutes later — without touching the ping path, and works under both PyMySQL versions. Verified in python:3.12-slim against MySQL:
PyMySQL 1.2.0 pool_pre_ping=True -> TypeError (reproduces prod)
PyMySQL 1.2.0 pool_recycle=600 -> OK
PyMySQL 1.1.2 pool_recycle=600 -> OK
PyMySQL is pinned as well, so local and the image stop diverging.