Repository navigation
Name the psycopg2 dialect explicitly for the MetaTracker engine - #23
Merged
Merged
Conversation
SQLAlchemy 2.1 (24 September 2026) made psycopg (v3) the default driver for plain "postgresql://" URLs. The image installs psycopg2-binary, so every processing image built since 7 October fails the MetaTracker step with "No module named 'psycopg'": the file is still calibrated and uploaded, but its metadata never reaches RDS. dev-padre has logged this since its 22 October image. Spell out "postgresql+psycopg2://" where the SQLAlchemy engine is built. The failed-file reprocessing query already uses psycopg2 directly with a libpq URL and is unaffected. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the change explicitly selecting psycopg2 for MetaTracker database connections.
No open issues 🎉
What changed in this PR
This PR makes MetaTracker explicitly use the installed psycopg2 driver, preventing metadata tracking failures caused by SQLAlchemy selecting psycopg v3.
Changes:
- Changes the engine URL scheme to
postgresql+psycopg2://. - Adds a comment explaining why the driver must be specified.
| File | Description |
|---|---|
lambda_function/src/file_processor/file_processor.py |
Explicitly selects psycopg2 for the MetaTracker database connection and documents the reason. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
SQLAlchemy 2.1 (released 24 September 2026) changed the default driver for plain
postgresql://URLs from psycopg2 to psycopg (v3). Theswxsoc[tracker]extra pins onlysqlalchemy<3, and this image installspsycopg2-binary, so every processing image built since 7 October fails the MetaTracker step:The file is still calibrated and uploaded, so the run reports success, but its metadata never reaches RDS. dev-padre has logged this on every file since its 22 October image; prod gdctb and today's prod padre images carry it too. Build logs confirm the cause: images built 9 September got SQLAlchemy 2.0.52, every image since 7 October got 2.1.x.
What
Spell out the DBAPI in the engine URL:
postgresql+psycopg2://. The failed-file reprocessing path callspsycopg2.connectdirectly with a libpq URL and is unaffected.Verification
science_file_idpopulated and no psycopg error.The same URL pattern exists in
sdc_aws_concating_lambda; a matching PR is open there.🤖 Generated with Claude Code