Skip to content

fix(reporting-storage): check for reporting tables in the current schema only - #1843

Open
bokelley wants to merge 1 commit into
nextfrom
claude/issue-1838-migrations-current-schema
Open

bokelley wants to merge 1 commit into
nextfrom
claude/issue-1838-migrations-current-schema

Conversation

@bokelley

@bokelley bokelley commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Closes #1838. The fix is wider than the issue: the same unqualified to_regclass('name') existence check appears in two shipped migrations as well as in the migrations ledger lookup.

The bug

to_regclass('name') resolves through the whole search_path. Suppose search_path=tenant,public, and public already holds a ledger. Every unqualified check then finds public's table while the code is working on tenant:

Site Effect
migrations/__init__.py applied_migrations() Reads public's adcp_schema_migrations. python -m adcp migrate prints "up to date" and leaves tenant unmigrated; postgres_storage().check_schema() passes (#1838).
reporting_ledger_account_generations.sql (migration 0002), ×4 Treats public's reconciliation / materializer / status / production tables as present and runs ALTER TABLE … ADD COLUMN offering_id … REFERENCES reporting_configurations on the unqualified name. If public is migrated, tenant's upgrade fails with DuplicateColumn (reproduced, see below). If not, tenant's upgrade alters public's tables.
reporting_status_checkpoint_offerings.sql (migration 0006), ×3 The same pattern for the status checkpoint and event tables.
reporting_ledger_reconciliation.sql Takes another schema's feed as installed and skips this schema's feed backfill.
ledger/pg.py _period_close_generations_on Takes another schema's reporting_materializer_candidates as present, then reads it with an unqualified FROM.

Each site now uses to_regclass(quote_ident(current_schema()) || '.name'), which every other lookup in the package already uses (row_storage.py, period_retirement.py, pg.py:867, …). applied_migrations() reads from sql.Identifier(current_schema, "adcp_schema_migrations"). Every other catalog query in the package already filters on current_schema(); I checked information_schema and pg_class usage.

Only setups with more than one schema on the search_path are affected. A single-schema deployment (ESA's included) behaves exactly as before. The edited SQL sits inside DO blocks, so a fresh or migrated catalog is unchanged. Migrations are recorded by version, so already-upgraded databases aren't re-run.

Tests

tests/conformance/reporting/test_reporting_storage_migrations_pg.py:

  • test_no_storage_existence_check_resolves_through_the_search_path: a static test that refuses any unqualified to_regclass('…') in the package's .py / .sql files.
  • test_migrate_upgrades_its_own_schema_beside_a_migrated_one (real PostgreSQL): fully migrates schema A, then runs migrate with search_path=B,A. All six versions apply to B, B's catalog equals A's, and A's catalog (columns, constraints, recorded versions) is unchanged.
  • test_create_schema_builds_its_own_schema_beside_a_migrated_one: the same setup through PgReportingLedgerStore.create_schema(). B records 0001–0003, with 0004–0006 left pending, and A is unchanged.

Without the fix:

  • On the released source, all three new tests fail; pending_migrations returns [] for the unmigrated schema.
  • With only the applied_migrations fix and the DDL reverted, both Postgres tests fail with psycopg.errors.DuplicateColumn: column "offering_id" of relation "reporting_reconciliation_records" already exists. That is migration 0002 altering the other schema.

Local runs (PostgreSQL 15):

  • test_reporting_storage_migrations_pg.py: 8 passed.
  • Full tests/conformance/reporting/ run:
    • My first run had environment failures: the disk filled, my test DSN wasn't in a form the fixtures could append options to, and killed runs left schemas behind.
    • After fixing those, 15 tests still failed. All 15 fail the same way on unmodified next in a fresh database: they are the installed-distribution / rolling-upgrade tests (*_rolling, *_installed, notification_packaging), and CI excludes most of those files.
    • Nothing fails with this change that passes without it.
  • make lint, make typecheck-all: pass.

Found in the post-release review of the #1787 migration.

🤖 Generated with Claude Code

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean fix for #1838, no blocking or medium findings.

What I checked:

  • Qualifies all 10 unqualified to_regclass('name') existence checks with quote_ident(current_schema()), closing a multi-schema isolation gap where a ledger on another schema in the search_path could be read, skipped, or ALTERed as the current schema's.
  • Verified the base package's 10 unqualified sites map exactly to the diff; each qualified guard pairs with a subsequent unqualified FROM/ALTER/::regclass that resolves to current_schema()'s table (first on path), so guard and use stay consistent.
  • applied_migrations() reads via sql.Identifier(current_schema, …) and returns {} when current_schema is null — no None passed to Identifier.
  • Migrations are recorded by version with no body checksum, so editing shipped migrations 0002/0006 is safe for already-migrated DBs; single-schema deployments unchanged.
  • Test coverage: a static test refuses any new unqualified to_regclass('; two real-Postgres tests prove a schema migrates in place beside a migrated one without touching it.

Gate checks: high_risk false; gated_paths false; no no-auto-approve team match; no prior decision. No critical/high/medium findings. Falls through to row 9.

…ema only

`to_regclass('name')` resolves through the whole search_path. With
`search_path=tenant,public` and a ledger already in `public`, every
unqualified existence check in the package could find `public`'s table
while working on `tenant`:

- `applied_migrations()` read `public`'s `adcp_schema_migrations`, so
  `python -m adcp migrate` printed "up to date" and left `tenant`
  unmigrated, and `postgres_storage`'s `check_schema` passed (#1838).
- Migration 0002 (`reporting_ledger_account_generations.sql`) and 0006
  (`reporting_status_checkpoint_offerings.sql`) took another schema's
  managed-delivery and status tables as present and ran their
  `ALTER TABLE` on the unqualified name. `tenant`'s upgrade then altered
  `public`'s tables, or failed with DuplicateColumn when `public` was
  already migrated.
- The reconciliation DDL took another schema's feed as installed and
  skipped this schema's feed backfill.
- `_period_close_generations_on` took another schema's materializer
  candidates as present and read them.

Qualify each check with `quote_ident(current_schema())`, the pattern the
rest of the package already uses, and read the recorded migrations from
the current schema's table. A static test refuses any new unqualified
`to_regclass('...')` in the package. Real PostgreSQL tests run `migrate`
and `create_schema()` for a schema beside a migrated one on the same
search_path. They check that the schema is built in place and that the
other schema's catalog is unchanged.

Closes #1838

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bokelley
bokelley force-pushed the claude/issue-1838-migrations-current-schema branch from 037ddb6 to 2391f8b Compare November 3, 2026 16:33

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean pass, no blocking or medium findings.

This PR qualifies every unqualified to_regclass('name') existence check across the reporting-storage package with quote_ident(current_schema()), closing a real multi-tenant boundary break where another schema's ledger on the search_path could skip a tenant's migration or run ALTER TABLE against the wrong schema. applied_migrations reads the existence flag and row from the same current_schema() (row[0] only used when row[1] is true), and the surviving ::regclass casts resolve to current_schema, confirmed by the preceding guard. Three tests back it — a static grep-guard against new unqualified checks plus two real-Postgres side-by-side-schema tests. fix(reporting-storage): is the correct semver signal (no public API change).

Decision table: no critical/high/medium findings (row 1 N/A). gated_paths false (row 2 N/A); review_decision is APPROVED regardless. high_risk false (rows 3/5 N/A). No medium findings (rows 4/8 N/A). Prior decision was approve (row 6 N/A). No no-auto-approve team match (row 7 N/A). Falls through to row 9 → approve. Prior pass was also clean and nothing re-surfaces.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant