Repository navigation
Fix the remaining known bugs from the end-to-end suite - #36
Merged
Merged
Conversation
Fixes the 10 bugs the end-to-end suite still marks as xfail on the bug-fix branch, and removes their markers: - POSTGRES_USER reaches libpq as PGUSER, so a role name with a space works; surrounding whitespace is still dropped, as before. The database name is passed as "-d dbname='...'", so a name starting with "-" is no longer read as pg_dump options (which backed up a different database) and a name containing "=" is no longer read as a connection string. This applies to pg_dump, psql and pg_restore, and restore.sh no longer runs basename on BACKUP_FILE, which read a key starting with "-" as options. - Restores of encrypted backups check that the decrypted file is a readable backup (it decompresses, or pg_restore --list reads it). AES-CBC has no integrity check, so about 1 decryption in 256 with the wrong key "succeeds". Before, about 1 legacy backup in 256 could not be restored, and a wrong password with DROP_DATABASE=yes could drop the database before the restore failed. When no decryption is readable, the log says why. - Retention and BACKUP_FILE=latest read each key from the 'aws s3 ls' line as it is, instead of rebuilding it with awk. Keys with leading, trailing or repeated spaces or tabs are no longer mixed up with other keys, and a key containing " PRE " is no longer skipped. - A DELETE_OLDER_THAN in the future, such as "7 days" without "ago", is rejected before the backup starts. Before, it deleted everything else in the prefix and reported success. - HOOK_INHERIT_ENV=yes passes HOOK_EVENT, and HOOK_EXIT_CODE to error hooks, like the default allowlist does. - A custom-format restore into PostgreSQL 16 or older no longer fails when its only errors come from SET transaction_timeout, which pg_restore 17 and later send on every connection. pg_restore's messages are still shown as they come. The e2e harness now passes database names to psql as connection strings too. New tests cover backing up and restoring such names (plain and custom format, one and two jobs, BACKUP_FILE=latest at the bucket root), BACKUP_FILE=latest for names with repeated spaces or " PRE ", a future DELETE_OLDER_THAN deleting nothing, and a custom-format restore with pg_restore-only options in POSTGRES_EXTRA_OPTS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UdSRPLaq3mTA6Djdzte7C5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the known bugs that #34 left open. Each one has an end-to-end test marked
xfail; this PR removes the last of those markers, so the suite now checks every bug it found.#34 (merged) fixes 31 of the 41 known bugs. This PR fixes the other 10, and finishes one that #34 fixes only in part. It was opened against #34's branch and is now based on
master, which holds #34; the diff is the same single commit.Fixes
-is read aspg_dumpoptions, so another database is backed up under that namepg_dump,psqlandpg_restoreas-d "dbname='…'", andrestore.shno longer runsbasenameonBACKUP_FILE=, or aPOSTGRES_USERcontaining a space, breaks the dumpPGUSERDROP_DATABASE=yescan drop the database before the restore failspg_restore --listreads it). Otherwise the legacy key derivation is tried, then the restore stops before it touches the database and says whypg_restore18 sendsSET transaction_timeout)pg_restorecarried on past them, the restore succeedsPREis never deletedaws s3 lsline as they are; only sub-prefix lines are skipped.BACKUP_FILE=latestuses the same parserDELETE_OLDER_THAN="7 days", missing "ago") deletes everything else in the prefix and exits 0. #34 already keeps the new backupHOOK_INHERIT_ENV=yesdropsHOOK_EVENTandHOOK_EXIT_CODEBehaviour changes
POSTGRES_USERreaches libpq asPGUSERinstead of-U. Leading and trailing spaces are still dropped, and-UinPOSTGRES_EXTRA_OPTSstill overrides it. Host and port stay on the command line. Hooks run withHOOK_INHERIT_ENV=yesnow also seePGUSER; they already sawPGPASSWORD.DELETE_OLDER_THANin the future now fails the run before the backup, as an invalid date already does.Review
An independent review of this PR and of my commits on #34 found these problems, and they're fixed here or on #34:
POSTGRES_EXTRA_OPTS=-x;DELETE_MATCH_DATABASE=yesalso deletedapp_archive_…backups for a database namedapp;ENCRYPTION_PASSWORD_FILEuploaded an unencrypted backup.psqlbefore every custom-format restore, which brokepg_restore-only options such as--no-ownerinPOSTGRES_EXTRA_OPTS;-breakbasename.Tests
End-to-end:
2a8cf9d, and all 11 case-file jobs pass. The suite now has noxfailmarkers.New end-to-end tests:
-d…_src=x'y\z): plain, custom format with one and two jobs, andBACKUP_FILE=latestfrom the bucket root;--no-owner --no-privilegesinPOSTGRES_EXTRA_OPTS;DELETE_OLDER_THANdeleting nothing.The harness's own
psqlcalls now pass database names as connection strings too.Unit tests (
test/hooks_test.sh, 172): new checks cover:BACKUP_FILE=latestkey parsing;PGUSER;HOOK_INHERIT_ENV=yes;pg_restorecompatibility rule, including--single-transaction, wherepg_restorestops at the first error and the restore must fail.They pass under dash, bash and the image's BusyBox
sh.ShellCheck 0.11, as pinned in CI, is clean.
🤖 Generated with Claude Code
https://claude.ai/code/session_01UdSRPLaq3mTA6Djdzte7C5