diff --git a/AGENTS.md b/AGENTS.md index e491bd5a..d642727f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,6 +4,7 @@ The current implementation is documented in `doc/README.md` and public Dartdoc. Old code and upstream architecture are retired. - Keep one product package. Add abstractions only to support real use cases. +- During beta, do not add old API or schema compatibility unless explicitly requested. - Use independent Dart libraries; never use `part` or `part of`. - Keep implementation in `lib/src/` and public entrypoints as explicit exports. - Document public behavior, ownership and failure boundaries with Dartdoc. diff --git a/CHANGELOG.md b/CHANGELOG.md index 82ddab5e..b4447777 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,21 @@ +## Unreleased + +Breaking PostgreSQL schema change: table identities now include the database +schema, including `public`. Regenerate clients and snapshots and review the +required migration. Old unqualified snapshots are not automatically normalized; +generated table replacement requires explicit destructive opt-in and deletes data. + +- Discover PostgreSQL models in `schema/{schema}/*.dart` and other supported + databases in `schema/*.dart`, alongside an optional default `schema.dart`. +- Generate schema-grouped clients when PostgreSQL models use a non-default + namespace. Preserve mixed-case identifiers and qualify table references, + cross-schema relationships, migrations and catalog verification explicitly. +- Reject dotted model table names and conflicting declarations. Keep snapshots + stable when definitions are split or renamed within the same database schema. +- PostgreSQL cursor tokens now identify schema-qualified tables, including + `public`. Regenerated clients reject tokens issued with the earlier unqualified + table identity; applications that persist cursors must reissue them. + ## 6.0.0-beta.3 Breaking schema authoring change: replace annotated entities and hand-written row diff --git a/assets/sqlite/worker-4360a5ed33bec6997af5514e04d393eb8fae00e331f2532a696ce34b0b9e44a1.js b/assets/sqlite/worker-a49ab41eb8d74a1dba9d1dd8062e938daa4ab55ec26178f47a83c129806012f4.js similarity index 99% rename from assets/sqlite/worker-4360a5ed33bec6997af5514e04d393eb8fae00e331f2532a696ce34b0b9e44a1.js rename to assets/sqlite/worker-a49ab41eb8d74a1dba9d1dd8062e938daa4ab55ec26178f47a83c129806012f4.js index 84b9d683..5995257d 100644 --- a/assets/sqlite/worker-4360a5ed33bec6997af5514e04d393eb8fae00e331f2532a696ce34b0b9e44a1.js +++ b/assets/sqlite/worker-a49ab41eb8d74a1dba9d1dd8062e938daa4ab55ec26178f47a83c129806012f4.js @@ -4889,7 +4889,7 @@ $S:21} A.fO.prototype={ W(a6,a7){var s=0,r=A.aM(t.X),q,p=2,o=[],n=this,m,l,k,j,i,h,g,f,e,d,c,b,a,a0,a1,a2,a3,a4,a5 var $async$W=A.aN(function(a8,a9){if(a8===1){o.push(a9) -s=p}for(;;)A:switch(s){case 0:if(a6==="hello"){q=A.x([1,"1ffcc1c93a300242a8c4ca7badc270240d0550d23254799eda18232f8bd3312d"],t.f) +s=p}for(;;)A:switch(s){case 0:if(a6==="hello"){q=A.x([1,"1a12f6159d3f3b9a21811c2431bd7cd0d7986d0d8530f6085246adfbff6d049e"],t.f) s=1 break}s=a6==="open"?3:4 break diff --git a/build.yaml b/build.yaml index 46989b53..84f80e8b 100644 --- a/build.yaml +++ b/build.yaml @@ -4,6 +4,7 @@ builders: builder_factories: ["ormBuilder"] build_extensions: ".dart": [".orm.dart", ".snapshot.dart"] + "$lib$": ["schema.orm.dart", "schema.snapshot.dart"] auto_apply: none build_to: source queries: diff --git a/doc/README.md b/doc/README.md index 8b9f719f..444ec5a2 100644 --- a/doc/README.md +++ b/doc/README.md @@ -4,7 +4,8 @@ Define a schema, generate typed models and queries, and evolve your database wit reviewed Dart migrations. Start with [schema declarations](https://github.com/medz/dart-orm/blob/main/doc/authoring.md) and the [company example](https://github.com/medz/dart-orm/blob/main/example/company/README.md). -> **Version:** These guides describe `6.0.0-beta.3` and its Record schema API. +> **Version:** These guides follow the repository source. Use the corresponding +> release tag for the exact behavior of a published package. > When upgrading, replace annotated entities with `model(...)` and regenerate > clients. Keep existing migration history unchanged. @@ -13,6 +14,7 @@ and the [company example](https://github.com/medz/dart-orm/blob/main/example/com | Task | Guide | | --- | --- | | Define models, columns, keys and relationships | [Schema declarations](https://github.com/medz/dart-orm/blob/main/doc/authoring.md) | +| Organize files and PostgreSQL database schemas | [Database schemas](https://github.com/medz/dart-orm/blob/main/doc/namespaces.md) | | Generate a client and use build_runner | [Generation](https://github.com/medz/dart-orm/blob/main/doc/generation.md) | | Choose imports and open a database | [Entrypoints](https://github.com/medz/dart-orm/blob/main/doc/api.md) | | Filter, select, join and paginate | [Queries](https://github.com/medz/dart-orm/blob/main/doc/queries.md) | diff --git a/doc/authoring.md b/doc/authoring.md index 8e43067f..8a085d78 100644 --- a/doc/authoring.md +++ b/doc/authoring.md @@ -211,6 +211,14 @@ The `relations` callback must return a literal named Record containing direct ## Split schemas and static boundaries +For automatic file collection, use `schema/{schema}/*.dart` on PostgreSQL or +`schema/*.dart` on SQLite, MySQL and MariaDB. PostgreSQL directory names establish +database schema ownership; model declarations remain `model(...)`. Neither +pattern is recursive. See [database schemas](https://github.com/medz/dart-orm/blob/main/doc/namespaces.md) +for layouts, generated accessors, cross-schema relationships and single-file coexistence. + +A single-file root can also select definitions through Dart exports: + Use ordinary independent Dart libraries. A schema root can export selected models: ```dart diff --git a/doc/capabilities.md b/doc/capabilities.md index b9097ff9..bc428043 100644 --- a/doc/capabilities.md +++ b/doc/capabilities.md @@ -74,6 +74,14 @@ roles, command, mode and expressions plus enabled/forced row-security flags. Baseline does not certify grants, every extension, authorization behavior or the entire database environment. See [migrations](https://github.com/medz/dart-orm/blob/main/doc/migrations.md) and [importing](https://github.com/medz/dart-orm/blob/main/doc/importing.md). +## Database schemas + +PostgreSQL models can belong to named schemas in the same database, including +cross-schema relationships. The generator records the default `public` schema +explicitly when PostgreSQL is selected. MySQL/MariaDB database namespaces and +SQLite attached databases are not exposed by this model layout feature. See +[database schemas](https://github.com/medz/dart-orm/blob/main/doc/namespaces.md). + ## Connections, platforms and tools PostgreSQL uses its driver's pool and supports borrowed-pool ownership. SQLite diff --git a/doc/cli.md b/doc/cli.md index 9047837c..8f7274fd 100644 --- a/doc/cli.md +++ b/doc/cli.md @@ -101,7 +101,8 @@ renames/conversions, baseline and recovery behavior. These commands also work without project configuration: ```sh -dart run orm generate lib/schema.dart lib/generated/database.dart +dart run orm generate lib/schema.dart lib/generated/database.dart --database sqlite +dart run orm generate lib/schema --database postgres dart run orm migration registry migrations --dialect sqlite dart run orm db inspect --sqlite app.sqlite --table tasks dart run orm db import --sqlite app.sqlite --output lib/imported.dart @@ -116,9 +117,16 @@ verifyFull|require|disable` configures server TLS; `--database-schema` is specif to PostgreSQL. Catalog import writes a Dart draft and a separate review report. It does not migrate an existing database. -Without a config, `generate` uses `lib/schema.dart`. If a config's generated -snapshot was deleted, recover with the explicit source command -`dart run orm generate lib/schema.dart`, which does not load that config. +Generation uses the project configuration even when a schema path is supplied. +`--database` selects the engine without loading a configuration, which is useful +when recreating a deleted snapshot: +`dart run orm generate lib/schema --database postgres`. When an explicitly loaded +configuration and engine disagree, generation rejects the mismatch. Without a +configuration or path, the source defaults to `lib/schema.dart`. + +Directory layouts require an engine. `lib/schema`, `lib/schema/` and +`lib/schema.dart` identify the same root and combine the file and directory when +both exist. See [database schemas](https://github.com/medz/dart-orm/blob/main/doc/namespaces.md). ## Help and automation diff --git a/doc/generation.md b/doc/generation.md index a8d88a1e..7a786c83 100644 --- a/doc/generation.md +++ b/doc/generation.md @@ -40,6 +40,11 @@ output path: dart run orm generate lib/schema.dart lib/generated/database.dart ``` +For directory generation, the output must not match a schema input path, +including the optional sibling `schema.dart`. Choose a path outside the layout +or use an excluded `.orm.dart` filename inside it. Invalid paths are rejected +before either generated file is written, including on the first run. + The CLI checks source errors and resolves imports using the project's package configuration. Use it when a single explicit generation step suits the project. @@ -73,7 +78,7 @@ dart run build_runner build dart run build_runner watch ``` -The builder is opt-in and uses the explicit `generate_for` list. Select schema +For individual libraries, the builder is opt-in and uses the explicit `generate_for` list. Select schema libraries containing or exporting `model` declarations. Referenced models are included transitively; unrelated imports are not additional schema roots. Do not select every @@ -82,10 +87,40 @@ next to their input files. Import generated clients with prefixes if their declaration names overlap. The output of `lib/schema.dart` is always `lib/schema.orm.dart` and -`lib/schema.snapshot.dart`. There are no ORM-specific builder options. Use the CLI's -output argument for other locations. `build_runner` owns its build cache and output +`lib/schema.snapshot.dart`. Set `options.database: postgres` for a PostgreSQL +file root, so default tables explicitly belong to `public`. Use the CLI's output +argument for other locations. `build_runner` owns its build cache and output cleanup; do not manually edit that cache or generated source. +## Definition directories + +For a directory, select its root and engine through builder options. This runs +once for the root, without requiring an empty `schema.dart` or a `generate_for` +entry for each model file: + +```yaml +targets: + $default: + builders: + orm:orm: + enabled: true + options: + schema: lib/schema + database: postgres +``` + +PostgreSQL collects `lib/schema/{schema}/*.dart`; `sqlite`, `mysql` and `mariadb` +collect `lib/schema/*.dart`. Both also include a sibling `lib/schema.dart` when +present. Additional nesting is not recursive. The builder tracks matching file +additions and deletions as well as imported metadata. Outputs remain +`lib/schema.orm.dart` and `lib/schema.snapshot.dart`. + +The CLI accepts `lib/schema`, `lib/schema/` and `lib/schema.dart` for this same root. +It reads the engine from the project history, or accepts an explicit +`--database postgres` when no configuration should be loaded. See +[database schemas](https://github.com/medz/dart-orm/blob/main/doc/namespaces.md) for +namespace ownership, name collisions, relationships and migration behavior. + ## What triggers regeneration The builder resolves source through `BuildStep.resolver` and writes through diff --git a/doc/importing.md b/doc/importing.md index 86e538e4..7eb2f6ef 100644 --- a/doc/importing.md +++ b/doc/importing.md @@ -33,6 +33,13 @@ PostgreSQL uses a read-only, repeatable-read transaction in the selected schema. Each invocation uses one catalog snapshot. Existing outer transactions are rejected; a borrowed session without an active transaction is allowed. +For a non-default PostgreSQL schema, place the imported file in its matching +directory, for example `--database-schema auth --output lib/schema/auth/imported.dart`, +then run `dart run orm generate lib/schema --database postgres`. Import currently +reads one database schema per invocation. Cross-schema relationships remain +reported as unmanaged and must be declared against the imported target models +before generation. See [database schema layouts](https://github.com/medz/dart-orm/blob/main/doc/namespaces.md). + To import one table, use `--table accounts`. For a selected related group, use `importSchema(db, tables: ['accounts', 'notes'])` in Dart. Without a selection, discovery covers SQLite main, the current PostgreSQL schema or the selected diff --git a/doc/namespaces.md b/doc/namespaces.md new file mode 100644 index 00000000..d9e8ee58 --- /dev/null +++ b/doc/namespaces.md @@ -0,0 +1,135 @@ +# Database schemas and definition files + +Keep model declarations unchanged when splitting a schema. PostgreSQL uses the +first directory name as the database schema. MySQL, MariaDB and SQLite use a flat +directory of definition files for one configured database. + +| Source | PostgreSQL | MySQL, MariaDB, SQLite | +| --- | --- | --- | +| `lib/schema.dart` | Default `public` schema | Configured database | +| `lib/schema/{schema}/*.dart` | Named database schema | Not a declaration layout | +| `lib/schema/*.dart` | Use a schema subdirectory | Split model declarations | + +Neither directory pattern is recursive. Imported enums, codecs and default +factories can live elsewhere. In directory mode, referenced models must be +declared in the selected layout; an import or export cannot change their schema. +Generated `.orm.dart` and `.snapshot.dart` files are not declaration inputs. + +## PostgreSQL + +```text +lib/schema/ + public/ + profiles.dart + auth/ + users.dart + sessions.dart +``` + +`lib/schema/auth/users.dart`: + +```dart +import 'package:orm/schema.dart'; + +final user = model('Users', ( + id: identity(), + displayName: text(name: 'DisplayName'), +)); +``` + +`lib/schema/public/profiles.dart`: + +```dart +import 'package:orm/schema.dart'; +import '../auth/users.dart' as auth; + +final profile = model('profiles', ( + id: identity(), + accountId: integer(), +), relations: (p) => ( + account: references(p.accountId, () => auth.user), +)); +``` + +Generate offline using the project configuration, or select the engine explicitly: + +```sh +dart run orm generate lib/schema --database postgres +``` + +The client and snapshot are `lib/schema.orm.dart` and `lib/schema.snapshot.dart`. +Only default `public` models produce flat access such as `db.user`. Declaring a +non-default schema groups all model access by schema: + +```dart +final account = await db.auth.user.create(displayName: 'Alice'); +await db.public.profile.create(accountId: account.id); +``` + +Grouped row types include their schema, such as `AuthUser` and `PublicProfile`. +Different schemas may declare the same model and table names. Names must remain +distinct within their schema. Schema directory names must also be usable as Dart +database members; invalid names and generated type collisions are reported. +Case-sensitive database names cannot always be represented on a case-insensitive +source filesystem; the generator does not silently rename or merge them. + +Relationships use the target declaration's schema. Cross-schema foreign keys and +queries use the same PostgreSQL database and connection; they do not open another +database or introduce distributed transactions. + +## Single files and generation roots + +`lib/schema`, `lib/schema/` and `lib/schema.dart` select the same definition root. +If both the file and directory exist, generation combines their declarations. +The sibling file contributes to the default schema. Exporting a declaration that +was already discovered does not duplicate it; independent conflicting models +produce an error instead of overriding one another. + +Select the engine through `OrmConfig.history.dialect` or `--database`. Directory +generation requires an engine and never connects to discover it. Single-file +programmatic generation without an engine retains engine-neutral metadata; select +PostgreSQL explicitly when generating its physical namespaces. + +With an engine selected, table ordering is deterministic. Splitting declarations +or renaming a file within one schema does not change the physical snapshot. +Moving a model to another schema changes its database identity and requires a +reviewed migration. Existing migration definitions and fingerprints stay frozen. + +PostgreSQL namespace identity is a breaking change. Older unqualified snapshots +are not automatically matched to `public` tables. Regenerate the client and +snapshot, then review the migration before applying it. A generated replacement +plan requires `--allow-destructive` and recreates the affected tables, deleting +their rows. Projects that need to retain data must author and review that data +migration explicitly. Historical migration files must not be rewritten. + +## SQL names and migration ownership + +The schema, table and column names are separate identifiers. The example above +queries `"auth"."Users"` and `"DisplayName"`. PostgreSQL generation also explicitly +qualifies default models with `"public"`. Querying these tables does not depend +on a later `SET search_path` or an identically named temporary table. + +Do not write `model('auth.Users', ...)`: dotted table names are rejected. +Manual `TableSchema` definitions apply the same rule to table names, namespaces +and foreign-key targets when constructed, and also reject empty names and NUL +characters. Column names use snake_case unless +their helper specifies `name:`. Explicit physical +names retain their spelling and quotes are escaped for the selected dialect; +quoting does not change the database's own rules for case equality. + +Generated PostgreSQL creation plans include the required `CREATE SCHEMA IF NOT +EXISTS` statements. Review these through the normal migration workflow and use a +role with the required privileges. Removing models never automatically drops a +schema or runs `DROP SCHEMA ... CASCADE`. Explicit table moves can use +`SchemaRenames.tables` with the before/after identities, for example +`{'auth.Users': 'archive.Users'}`; these keys match snapshot identities and are +not interpreted as arbitrary SQL. + +Migration locks cover one PostgreSQL database, including work spanning several +schemas. Catalog verification, foreign keys, query inspection, cursors and change +subscriptions distinguish schema-qualified tables. Namespace getters do not grant +database permissions. Raw SQL remains trusted application SQL and follows its +own explicit names and session state. + +MySQL/MariaDB cross-database models and SQLite attached-database models are outside +this layout feature. Their definition files continue to describe one database. diff --git a/lib/builder.dart b/lib/builder.dart index e58dc6d4..c7a77753 100644 --- a/lib/builder.dart +++ b/lib/builder.dart @@ -1,6 +1,7 @@ /// Optional build_runner factories for model and named SQL generation. /// -/// Select source libraries with `generate_for` in `build.yaml`. Applications +/// Select libraries with `generate_for`, or configure `schema` and `database` +/// options to collect a definition directory as one client. Applications /// import the generated files; these builders run only during development. /// /// {@category Tooling} diff --git a/lib/src/cli/commands.dart b/lib/src/cli/commands.dart index 954961b5..cb1d1b8c 100644 --- a/lib/src/cli/commands.dart +++ b/lib/src/cli/commands.dart @@ -10,11 +10,18 @@ import '../../drivers/sqlite.dart'; import '../../generate.dart'; import '../../migrate.dart'; import '../../runtime.dart'; +import '../generate/schema/layout.dart'; import '../sqlite/assets_io.dart'; import 'arguments.dart'; import 'output.dart'; -Future runExplicitCli(List arguments, {bool json = false}) async { +Future runExplicitCli( + List arguments, { + bool json = false, + SqlDialect? dialect, + String defaultSource = 'lib/schema.dart', + String? defaultOutput, +}) async { void report(Map value) => CliOutput(json).report(value); try { if (arguments.first == 'web-assets') { @@ -32,21 +39,39 @@ Future runExplicitCli(List arguments, {bool json = false}) async { return; } if (arguments.first == 'generate') { - if (arguments.length < 2 || - arguments.length > 3 || - arguments.skip(1).any((a) => a.startsWith('--'))) { + final args = arguments.skip(1).toList(); + final index = args.indexOf('--database'); + if (index >= 0) { + if (args.where((v) => v == '--database').length != 1 || + index + 1 >= args.length || + !SqlDialect.values.any((d) => d.name == args[index + 1])) { + throw const FormatException( + 'Use --database sqlite|postgres|mysql|mariadb once.', + ); + } + final selected = SqlDialect.values.byName(args[index + 1]); + if (dialect != null && selected != dialect) { + throw const FormatException( + 'Generation engine must match the project migration history.', + ); + } + dialect = selected; + args.removeRange(index, index + 2); + } + if (args.length > 2 || args.any((a) => a.startsWith('--'))) { throw const FormatException( - 'generate expects a schema and optional output path.', + 'generate expects a schema path, optional output and --database engine.', ); } - await writeGeneratedSchema( - arguments[1], - output: arguments.length == 3 ? arguments[2] : null, - ); + final source = args.isEmpty ? defaultSource : args.first; + final output = args.length == 2 + ? args.last + : args.isEmpty + ? defaultOutput + : null; + await writeGeneratedSchema(source, output: output, dialect: dialect); CliOutput(json).report({ - 'generated': arguments.length == 3 - ? arguments[2] - : p.setExtension(arguments[1], '.orm.dart'), + 'generated': output ?? '${SchemaLayout.stem(source)}.orm.dart', }); return; } diff --git a/lib/src/cli/config.dart b/lib/src/cli/config.dart index ec9c5321..b83c9f5d 100644 --- a/lib/src/cli/config.dart +++ b/lib/src/cli/config.dart @@ -6,7 +6,8 @@ import '../../migrate_cli.dart'; /// Pass this configuration to the project CLI. The history fixes one database /// engine; changing a connection URL does not translate migrations. final class OrmConfig { - /// Source declaration read by generation and migration creation. + /// Declaration file or directory root read by generation and migration creation. + /// The history engine selects the directory layout; generation stays offline. final String schema; /// Generated client path, or the schema basename with `.orm.dart` when null. diff --git a/lib/src/cli/init.dart b/lib/src/cli/init.dart index ff139ad6..0de0e307 100644 --- a/lib/src/cli/init.dart +++ b/lib/src/cli/init.dart @@ -53,10 +53,14 @@ Future initializeProject(List args, CliOutput output) async { try { await save(source, _initialSchema); - final generated = await generateSchema(source, outputPath: client); + final dialect = SqlDialect.values.byName(engine); + final generated = await generateSchema( + source, + outputPath: client, + dialect: dialect, + ); await save(client, generated.dart); await save(snapshot, generated.snapshotDart); - final dialect = SqlDialect.values.byName(engine); final formatter = DartFormatter( languageVersion: DartFormatter.latestLanguageVersion, ); diff --git a/lib/src/cli/runner.dart b/lib/src/cli/runner.dart index 3e492411..391d7a99 100644 --- a/lib/src/cli/runner.dart +++ b/lib/src/cli/runner.dart @@ -4,6 +4,7 @@ import 'package:path/path.dart' as p; import '../../generate.dart'; import '../../migrate_cli.dart'; +import '../generate/schema/layout.dart'; import 'commands.dart'; import 'config.dart'; import 'init.dart'; @@ -12,7 +13,7 @@ import 'output.dart'; const _help = { '': '''Usage: dart run orm [--json] [--config orm.config.dart] init --database sqlite|postgres|mysql|mariadb - generate [schema.dart] [output.orm.dart] + generate [schema-path] [output.orm.dart] [--database engine] migrate create|check|plan|apply|status|verify|baseline|record|inspect migration registry [--dialect engine] db inspect|import @@ -27,11 +28,12 @@ Creates lib/schema.dart, its generated client/snapshot, a static migration registry and orm.config.dart in an existing Dart project. Never replaces files, connects to a database or applies DDL. Server URLs are read from DATABASE_URL only when a connection command runs.''', - 'generate': - '''Usage: dart run orm generate [schema.dart] [output.orm.dart] [--json] + 'generate': '''Usage: dart run orm generate [schema-path] [output.orm.dart] [--database engine] [--json] Without a path, uses OrmConfig.schema or lib/schema.dart. Generates the client and standalone physical snapshot without connecting to a database. -Use an explicit schema path to recreate a deleted snapshot before loading config.''', +Directory roots accept lib/schema, lib/schema/ or lib/schema.dart. +The project history selects the engine; --database selects it without loading config. +To recreate a missing snapshot: generate lib/schema --database .''', 'migrate': '''Usage: dart run orm migrate [--json] create [--allow-destructive] Generate schema, then save a reviewed diff check Validate fixed Dart migration history @@ -118,7 +120,9 @@ Future runOrmCli(List arguments, {OrmConfig? config}) async { return; } final projectCommand = - args.first == 'migrate' || args.first == 'generate' && args.length == 1; + args.first == 'migrate' || + args.first == 'generate' && + (!args.contains('--database') || configPath != null); if (config == null && projectCommand) { final path = configPath ?? 'orm.config.dart'; if (await File(path).exists()) { @@ -136,18 +140,20 @@ Future runOrmCli(List arguments, {OrmConfig? config}) async { 'Missing $path. Run dart run orm init --database .', ); } - args.add('lib/schema.dart'); + if (args.length == 1) args.add('lib/schema.dart'); } else if (configPath != null) { throw const FormatException( - '--config applies to generate without a path and migrate commands.', + '--config applies to generate and migrate commands.', ); } - if (config != null && args.first == 'generate' && args.length == 1) { - await writeGeneratedSchema(config.schema, output: config.output); - output.report({ - 'generated': - config.output ?? p.setExtension(config.schema, '.orm.dart'), - }); + if (config != null && args.first == 'generate') { + await runExplicitCli( + args, + json: json, + dialect: config.history.dialect, + defaultSource: config.schema, + defaultOutput: config.output, + ); return; } if (config != null && args.first == 'migrate') { @@ -165,9 +171,10 @@ Future runOrmCli(List arguments, {OrmConfig? config}) async { final generated = await generateSchema( config.schema, outputPath: config.output, + dialect: config.history.dialect, ); final client = - config.output ?? p.setExtension(config.schema, '.orm.dart'); + config.output ?? '${SchemaLayout.stem(config.schema)}.orm.dart'; await File(client).parent.create(recursive: true); await File(client).writeAsString(generated.dart); final snapshotPath = client.endsWith('.orm.dart') diff --git a/lib/src/generate/build.dart b/lib/src/generate/build.dart index 29f1b929..8b7257cc 100644 --- a/lib/src/generate/build.dart +++ b/lib/src/generate/build.dart @@ -3,20 +3,43 @@ import 'package:analyzer/dart/analysis/session.dart'; import 'package:analyzer/dart/ast/ast.dart'; import 'package:analyzer/dart/element/element.dart'; import 'package:build/build.dart' as builder; +import 'package:glob/glob.dart'; import 'package:path/path.dart' as p; +import '../../driver.dart' show SqlDialect; import 'exception.dart'; import 'queries.dart'; import 'schema.dart'; +import 'schema/layout.dart'; /// Factory used by build_runner's build.yaml registration. builder.Builder ormBuilder(builder.BuilderOptions options) { - if (options.config.isNotEmpty) { + if (options.config.keys.any((key) => !{'database', 'schema'}.contains(key))) { throw ArgumentError( - 'ORM has no builder-specific options. Select schema roots with generate_for.', + 'Use database and schema builder options, or generate_for for individual libraries.', ); } - return const _OrmBuilder(); + final database = options.config['database']; + if (database != null && !SqlDialect.values.any((d) => d.name == database)) { + throw ArgumentError('database must be sqlite, postgres, mysql or mariadb.'); + } + final dialect = database == null + ? null + : SqlDialect.values.byName(database as String); + final schema = options.config['schema']; + if (schema != null) { + if (schema is! String || dialect == null) { + throw ArgumentError( + 'Directory generation needs schema: lib/schema and database: engine.', + ); + } + final root = SchemaLayout.stem(schema, paths: p.url); + if (!p.url.isWithin('lib', root)) { + throw ArgumentError('Put the schema root under lib/.'); + } + return _OrmDirectoryBuilder(root, dialect); + } + return _OrmBuilder(dialect: dialect); } /// Fixed SQL files are read as build assets so their edits invalidate output. @@ -64,7 +87,8 @@ final class _OrmQueryBuilder implements builder.Builder { } final class _OrmBuilder implements builder.Builder { - const _OrmBuilder(); + final SqlDialect? dialect; + const _OrmBuilder({this.dialect}); @override Map> get buildExtensions => const { '.dart': ['.orm.dart', '.snapshot.dart'], @@ -94,6 +118,9 @@ final class _OrmBuilder implements builder.Builder { } return p.url.relative(asset.path, from: p.url.dirname(output.path)); }, + dialect: dialect, + stableOrder: dialect != null, + namespaceOf: dialect == SqlDialect.postgres ? (_) => 'public' : null, resolve: (library) async { final node = await step.resolver.astNodeFor( library.firstFragment, @@ -122,10 +149,12 @@ final class _OrmBuilder implements builder.Builder { } Future<(CompilationUnit, LibraryElement)> _resolveSchema( - builder.BuildStep step, - ) async { + builder.BuildStep step, [ + builder.AssetId? source, + ]) async { + final input = source ?? step.inputId; for (var attempt = 0; ; attempt++) { - final library = await step.resolver.libraryFor(step.inputId); + final library = await step.resolver.libraryFor(input); final node = await step.resolver.astNodeFor( library.firstFragment, resolve: true, @@ -157,3 +186,101 @@ final class _OrmBuilder implements builder.Builder { } } } + +final class _OrmDirectoryBuilder(final String root, final SqlDialect dialect) + implements builder.Builder { + @override + Map> get buildExtensions => { + r'$lib$': [ + '${p.url.relative(root, from: 'lib')}.orm.dart', + '${p.url.relative(root, from: 'lib')}.snapshot.dart', + ], + }; + + @override + Future build(builder.BuildStep step) async { + final layout = SchemaLayout( + root, + dialect: dialect, + directory: true, + paths: p.url, + ); + final package = step.inputId.package; + final sources = []; + final file = builder.AssetId(package, layout.file); + if (await step.canRead(file)) sources.add(file); + final direct = await step.findAssets(Glob('$root/*.dart')).toList(); + if (dialect == SqlDialect.postgres && + direct.any((f) => SchemaLayout.declaration(f.path))) { + throw GenerationException( + 'Put PostgreSQL declarations in $root/{schema}/*.dart.', + ); + } + sources.addAll( + dialect == SqlDialect.postgres + ? await step.findAssets(Glob('$root/*/*.dart')).toList() + : direct, + ); + sources.removeWhere((f) => !SchemaLayout.declaration(f.path)); + sources.sort((a, b) => a.path.compareTo(b.path)); + if (sources.isEmpty) { + return; // Deleting the definition removes owned outputs. + } + final units = []; + final paths = {}; + final resolver = _OrmBuilder(dialect: dialect); + for (final source in sources) { + if (!await step.resolver.isLibrary(source)) { + throw GenerationException( + 'Use independent Dart schema libraries: $source', + ); + } + final (unit, library) = await resolver._resolveSchema(step, source); + units.add(unit); + paths[library] = source.path; + } + final output = builder.AssetId(package, layout.output); + final library = units.first.declaredFragment!.element; + final result = await generateResolvedSchema( + units.first, + library, + p.url.relative(sources.first.path, from: p.url.dirname(output.path)), + (uri) { + if (uri.scheme != 'asset') return uri.toString(); + final asset = builder.AssetId.resolve(uri); + return asset.package == package + ? p.url.relative(asset.path, from: p.url.dirname(output.path)) + : 'package:${asset.package}/${p.url.relative(asset.path, from: 'lib')}'; + }, + resolve: (owner) async { + final node = await step.resolver.astNodeFor( + owner.firstFragment, + resolve: true, + ); + if (node is! CompilationUnit) { + throw GenerationException('Cannot resolve ${owner.uri}.'); + } + return node; + }, + additionalRoots: units.skip(1).toList(), + namespaceOf: (variable) { + final owner = variable.declaredFragment!.element.library!; + final path = paths[owner]; + if (path == null) { + throw GenerationException( + 'Model declared outside the schema layout: ${owner.uri}', + ); + } + return layout.namespace(path); + }, + stableOrder: true, + dialect: dialect, + ); + final snapshot = result.snapshotDart; + await step.writeAsString(output, result.dart); + await step.writeAsString( + builder.AssetId(package, '$root.snapshot.dart'), + snapshot, + ); + } +} diff --git a/lib/src/generate/emitter.dart b/lib/src/generate/emitter.dart index 5e50335a..807cc4b1 100644 --- a/lib/src/generate/emitter.dart +++ b/lib/src/generate/emitter.dart @@ -31,13 +31,14 @@ String emitSchema(List schema, String import, DartNames names) { ); } b.writeln( - 'final ${entity.name}Schema = TableSchema(${dartLiteral(entity.table)}, ' + 'final ${entity.binding}Schema = TableSchema(${dartLiteral(entity.table)}, ' + '${entity.namespace == null ? '' : 'namespace: ${dartLiteral(entity.namespace!)}, '}' 'columns: [${entity.fields.map((f) => columnSymbol(entity, f)).join(', ')}], ' 'primaryKey: ${dartStringList(entity.columns(entity.primaryKey))}, ' 'uniqueKeys: [${entity.uniqueKeys.map((k) => dartStringList(entity.columns(k))).join(', ')}], ' 'indexes: [${entity.indexes.map((i) => 'IndexSchema(${dartLiteral(i.name)}, ${dartStringList(entity.columns(i.keys))}, unique: ${i.unique})').join(', ')}], ' '${entity.checks.isEmpty ? '' : 'checks: [${entity.checks.map((c) => 'CheckSchema.forDialects(${c.name == null ? 'null' : dartLiteral(c.name!)}, sqlite: ${dartLiteral(c.sqlite)}, postgres: ${dartLiteral(c.postgres)}, mysql: ${c.mysql == null ? 'null' : dartLiteral(c.mysql!)}, mariadb: ${c.mariadb == null ? 'null' : dartLiteral(c.mariadb!)})').join(', ')}], '}' - 'foreignKeys: [${entity.edges.where((e) => e.isForeignKey).map((e) => 'ForeignKey(${dartStringList(entity.columns(e.parentKeys))}, ${dartLiteral(e.target.table)}, ${dartStringList(e.target.columns(e.childKeys))}, onDelete: ${dartLiteral(e.onDelete!)})').join(', ')}]);', + 'foreignKeys: [${entity.edges.where((e) => e.isForeignKey).map((e) => 'ForeignKey(${dartStringList(entity.columns(e.parentKeys))}, ${dartLiteral(e.target.table)}, ${dartStringList(e.target.columns(e.childKeys))}, onDelete: ${dartLiteral(e.onDelete!)}${e.target.namespace == null ? '' : ', targetNamespace: ${dartLiteral(e.target.namespace!)}'})').join(', ')}]);', ); b.writeln( 'final class ${entity.fieldsType} extends Fields {\n ${entity.fieldsType}(super.table);', @@ -55,19 +56,19 @@ String emitSchema(List schema, String import, DartNames names) { } b.writeln( 'Relation<${edge.target.rowType}, ${edge.target.fieldsType}> get ${edge.name} => ' - 'Relation(${edge.target.name}Table, parent: [${edge.parentKeys.join(', ')}], ' + 'Relation(${edge.target.binding}Table, parent: [${edge.parentKeys.join(', ')}], ' 'child: (row) => [${edge.childKeys.map((k) => 'row.$k').join(', ')}]);', ); } b.writeln('}'); final selection = modelSelection(entity, 'row'); b.writeln( - 'final ${entity.name}Table = Table<${entity.rowType}, ${entity.fieldsType}>(' - '${entity.name}Schema, ${entity.fieldsType}.new, (row) => $selection);', + 'final ${entity.binding}Table = Table<${entity.rowType}, ${entity.fieldsType}>(' + '${entity.binding}Schema, ${entity.fieldsType}.new, (row) => $selection);', ); b.writeln( 'final class ${entity.setType} extends TableSet<${entity.rowType}, ${entity.fieldsType}> {' - '${entity.setType}(QueryContext db) : super(db, ${entity.name}Table) { db.registerSchema(appSchema); }', + '${entity.setType}(QueryContext db) : super(db, ${entity.binding}Table) { db.registerSchema(appSchema); }', ); var input = 'row'; final create = @@ -124,13 +125,35 @@ String emitSchema(List schema, String import, DartNames names) { } } b.writeln( - 'final appSchema = List.unmodifiable([${schema.map((e) => '${e.name}Schema').join(', ')}]);', + 'final appSchema = List.unmodifiable([${schema.map((e) => '${e.binding}Schema').join(', ')}]);', ); - b.writeln('extension AppTables on QueryContext {'); - for (final e in schema) { - b.writeln('${e.setType} get ${e.name} => ${e.setType}(this);'); + if (schema.first.grouped) { + final namespaces = schema.map((e) => e.namespace!).toSet().toList()..sort(); + for (final namespace in namespaces) { + final type = + '${namespace[0].toUpperCase()}${namespace.substring(1)}Tables'; + b.writeln( + 'final class $type { final QueryContext _context; $type(this._context);', + ); + for (final e in schema.where((e) => e.namespace == namespace)) { + b.writeln('${e.setType} get ${e.name} => ${e.setType}(_context);'); + } + b.writeln('}'); + } + b.writeln('extension AppTables on QueryContext {'); + for (final namespace in namespaces) { + final type = + '${namespace[0].toUpperCase()}${namespace.substring(1)}Tables'; + b.writeln('$type get $namespace => $type(this);'); + } + b.writeln('}'); + } else { + b.writeln('extension AppTables on QueryContext {'); + for (final e in schema) { + b.writeln('${e.setType} get ${e.name} => ${e.setType}(this);'); + } + b.writeln('}'); } - b.writeln('}'); return b.toString(); } diff --git a/lib/src/generate/model.dart b/lib/src/generate/model.dart index 83d36728..965cb0b3 100644 --- a/lib/src/generate/model.dart +++ b/lib/src/generate/model.dart @@ -150,6 +150,10 @@ final class ModelEntity { final String name; final String table; final String row; + final String? namespace; + final bool grouped; + String get binding => grouped ? '${namespace}_$name' : name; + String get identity => namespace == null ? table : '$namespace.$table'; final List fields; List primaryKey; @@ -157,16 +161,22 @@ final class ModelEntity { final List indexes = []; final List edges = []; final List checks = []; - ModelEntity(this.name, this.table, this.row, this.fields) - : primaryKey = [ - for (final f in fields) - if (f.id) f.name, - ], - uniqueKeys = [ - for (final f in fields) - if (f.unique) [f.name], - ]; - String get symbol => name[0].toUpperCase() + name.substring(1); + ModelEntity( + this.name, + this.table, + this.row, + this.fields, { + this.namespace, + this.grouped = false, + }) : primaryKey = [ + for (final f in fields) + if (f.id) f.name, + ], + uniqueKeys = [ + for (final f in fields) + if (f.unique) [f.name], + ]; + String get symbol => row; String get fieldsType => '${symbol}Fields'; String get setType => '${symbol}TableSet'; String get rowType => row; @@ -179,6 +189,7 @@ final class ModelEntity { ]; TableSchema snapshot() => TableSchema( table, + namespace: namespace, columns: [for (final f in fields) f.snapshot()], primaryKey: columns(primaryKey), uniqueKeys: [for (final key in uniqueKeys) columns(key)], @@ -194,6 +205,7 @@ final class ModelEntity { edge.target.table, edge.target.columns(edge.childKeys), onDelete: edge.onDelete!, + targetNamespace: edge.target.namespace, ), ], checks: checks, @@ -222,4 +234,4 @@ final class ModelRelation( } String columnSymbol(ModelEntity entity, ModelField field) => - '_${entity.name}${field.name[0].toUpperCase()}${field.name.substring(1)}'; + '_${entity.binding}${field.name[0].toUpperCase()}${field.name.substring(1)}'; diff --git a/lib/src/generate/schema.dart b/lib/src/generate/schema.dart index d3fbc460..af2f6f83 100644 --- a/lib/src/generate/schema.dart +++ b/lib/src/generate/schema.dart @@ -10,6 +10,7 @@ import 'package:path/path.dart' as p; import '../../migrate.dart'; import 'emitter.dart'; import 'exception.dart'; +import 'schema/layout.dart'; import 'schema/reader.dart'; import 'schema/sources.dart'; import 'source.dart'; @@ -33,66 +34,116 @@ final class GeneratedSchema { String get snapshotDart => formatMigration(schemaSource(snapshot)); } -/// Analyzes [sourcePath] and generates a client without writing files. +/// Analyzes one file or a definition directory without writing files. +/// +/// [dialect] selects the directory layout and physical namespace rules. PostgreSQL +/// uses `{root}/{schema}/*.dart`; other engines use `{root}/*.dart`. A sibling +/// `{root}.dart` contributes to the default namespace. Discovery is not recursive. +/// PostgreSQL generation records `public` explicitly for default models. +/// Single-file generation without a dialect retains engine-neutral metadata. /// /// [outputPath] determines relative imports and defaults to the source basename /// with an `.orm.dart` extension. Invalid declarations or output collisions throw -/// [GenerationException]. Application default factories are never executed. +/// [GenerationException]. Directory outputs must not be discovered as schema +/// inputs on later runs. Application default factories are never executed. Future generateSchema( String sourcePath, { String? outputPath, + SqlDialect? dialect, }) async { - final source = p.normalize(p.absolute(sourcePath)); - final output = p.normalize( - p.absolute(outputPath ?? p.setExtension(source, '.orm.dart')), + final input = p.normalize(p.absolute(sourcePath)); + final root = SchemaLayout.stem(input); + final layout = SchemaLayout( + input, + dialect: dialect, + directory: await Directory(root).exists(), ); + final output = p.normalize(p.absolute(outputPath ?? layout.output)); + if (layout.directory && + SchemaLayout.declaration(output) && + (layout.includes(output) || p.dirname(output) == root)) { + throw const GenerationException( + 'Generated outputs must not become schema inputs. ' + 'Choose a path outside the schema layout or use an .orm.dart filename.', + ); + } + final sources = []; + if (await File(layout.file).exists()) sources.add(layout.file); + if (layout.directory) { + await for (final entry in Directory(root).list(followLinks: false)) { + if (entry is File && SchemaLayout.declaration(entry.path)) { + if (dialect == SqlDialect.postgres) { + throw GenerationException( + 'Put PostgreSQL declarations in $root/{schema}/*.dart: ${entry.path}', + ); + } + sources.add(entry.path); + } else if (entry is Directory && dialect == SqlDialect.postgres) { + await for (final file in entry.list(followLinks: false)) { + if (file is File && SchemaLayout.declaration(file.path)) { + sources.add(file.path); + } + } + } + } + sources.sort(); + } + if (sources.isEmpty) { + throw GenerationException('No schema files found for $sourcePath.'); + } if (p.extension(output) != '.dart' || - source == output || - source == _schemaSnapshotPath(output)) { + sources.contains(output) || + sources.contains(_schemaSnapshotPath(output))) { throw const GenerationException( 'Client and snapshot outputs must be separate Dart files from the source.', ); } - final contexts = AnalysisContextCollection(includedPaths: [source]); + final contexts = AnalysisContextCollection(includedPaths: [p.dirname(root)]); try { - final resolved = await contexts - .contextFor(source) - .currentSession - .getResolvedUnit(source); - if (resolved is! ResolvedUnitResult) { - throw GenerationException('Cannot analyze $source.'); + Future resolvePath(String path) async { + final result = await contexts + .contextFor(path) + .currentSession + .getResolvedUnit(path); + if (result is! ResolvedUnitResult) { + throw GenerationException('Cannot analyze $path.'); + } + final errors = result.diagnostics.where( + (e) => e.severity.name.toLowerCase() == 'error', + ); + if (errors.isNotEmpty) throw GenerationException(errors.join('\n')); + return result; } - final errors = resolved.diagnostics.where( - (e) => e.severity.name.toLowerCase() == 'error', - ); - if (errors.isNotEmpty) { - throw GenerationException(errors.map((e) => e.toString()).join('\n')); + + final roots = []; + for (final path in sources) { + roots.add(await resolvePath(path)); } - final import = p - .relative(source, from: p.dirname(output)) - .replaceAll(r'\', '/'); + String importPath(Uri uri) => uri.scheme == 'file' + ? p + .relative(uri.toFilePath(), from: p.dirname(output)) + .replaceAll(r'\', '/') + : uri.toString(); + final resolved = roots.first; return await generateResolvedSchema( resolved.unit, resolved.libraryElement, - import, - (uri) => uri.scheme == 'file' - ? p - .relative(uri.toFilePath(), from: p.dirname(output)) - .replaceAll(r'\', '/') - : uri.toString(), - resolve: (library) async { - final result = await library.session.getResolvedUnit( - library.firstFragment.source.fullName, - ); - if (result is! ResolvedUnitResult) { - throw GenerationException('Cannot analyze ${library.uri}.'); - } - final errors = result.diagnostics.where( - (e) => e.severity.name.toLowerCase() == 'error', - ); - if (errors.isNotEmpty) throw GenerationException(errors.join('\n')); - return result.unit; - }, + importPath(Uri.file(sources.first)), + importPath, + resolve: (library) async => + (await resolvePath(library.firstFragment.source.fullName)).unit, + additionalRoots: [for (final result in roots.skip(1)) result.unit], + namespaceOf: (variable) => layout.namespace( + variable + .declaredFragment! + .element + .library! + .firstFragment + .source + .fullName, + ), + stableOrder: layout.directory || dialect != null, + dialect: dialect, ); } finally { await contexts.dispose(); @@ -105,18 +156,33 @@ Future generateResolvedSchema( String sourceImport, String Function(Uri) importUri, { required Future Function(LibraryElement) resolve, + List additionalRoots = const [], + String? Function(VariableDeclaration)? namespaceOf, + bool stableOrder = false, + SqlDialect? dialect, }) async { final names = DartNames(library.uri, importUri); final schema = SchemaReader( unit, library.typeSystem, names, - await schemaSources(unit, library, resolve), + await schemaSources( + unit, + library, + resolve, + additionalRoots: additionalRoots, + ), + namespaceOf: namespaceOf, + additionalRoots: additionalRoots, + stableOrder: stableOrder, ).read(); + final snapshot = SchemaSnapshot([ + for (final table in schema) table.snapshot(), + ]); return GeneratedSchema( DartFormatter(languageVersion: library.languageVersion.effective) .format(emitSchema(schema, sourceImport, names)), - SchemaSnapshot([for (final table in schema) table.snapshot()]), + dialect == null ? snapshot : snapshot.forDialect(dialect), ); } @@ -125,9 +191,17 @@ Future generateResolvedSchema( /// [output] defaults to the source basename with an `.orm.dart` extension. /// The snapshot uses the corresponding `.snapshot.dart` basename. Both are /// derived files and are replaced when generation succeeds. -Future writeGeneratedSchema(String source, {String? output}) async { - output ??= p.setExtension(source, '.orm.dart'); - final result = await generateSchema(source, outputPath: output); +Future writeGeneratedSchema( + String source, { + String? output, + SqlDialect? dialect, +}) async { + output ??= '${SchemaLayout.stem(source)}.orm.dart'; + final result = await generateSchema( + source, + outputPath: output, + dialect: dialect, + ); final file = File(output); await file.parent.create(recursive: true); final snapshot = result.snapshotDart; diff --git a/lib/src/generate/schema/layout.dart b/lib/src/generate/schema/layout.dart new file mode 100644 index 00000000..383a5f27 --- /dev/null +++ b/lib/src/generate/schema/layout.dart @@ -0,0 +1,65 @@ +import 'package:path/path.dart' as p; + +import '../../../driver.dart' show SqlDialect; +import '../exception.dart'; + +/// One generation root, shared by file discovery and build assets. +final class SchemaLayout { + final String root; + final SqlDialect? dialect; + final bool directory; + final p.Context _paths; + + SchemaLayout( + String input, { + required this.dialect, + required this.directory, + p.Context? paths, + }) : _paths = paths ?? p.context, + root = stem(input, paths: paths) { + if (directory && dialect == null) { + throw const GenerationException( + 'Directory generation requires a database engine. Use --database or the project configuration.', + ); + } + } + + static String stem(String input, {p.Context? paths}) { + paths ??= p.context; + final normalized = paths.normalize(input); + return normalized.endsWith('.dart') + ? paths.withoutExtension(normalized) + : normalized; + } + + String get file => '$root.dart'; + String get output => '$root.orm.dart'; + + static bool declaration(String path) => + path.endsWith('.dart') && + !path.endsWith('.orm.dart') && + !path.endsWith('.snapshot.dart'); + + bool includes(String path) { + path = _paths.normalize(path); + if (path == file) return true; + if (!directory || !_paths.isWithin(root, path) || !declaration(path)) { + return false; + } + final parts = _paths.split(_paths.relative(path, from: root)); + return parts.length == (dialect == SqlDialect.postgres ? 2 : 1); + } + + String? namespace(String path) { + path = _paths.normalize(path); + if (directory && !includes(path)) { + throw GenerationException( + 'Model declared outside the schema layout: $path. ' + 'Use ${dialect == SqlDialect.postgres ? '$root/{schema}/*.dart' : '$root/*.dart'}.', + ); + } + if (dialect != SqlDialect.postgres) return null; + if (path == file || !directory) return 'public'; + return _paths.split(_paths.relative(path, from: root)).first; + } +} diff --git a/lib/src/generate/schema/reader.dart b/lib/src/generate/schema/reader.dart index 48b7aa33..3a381b85 100644 --- a/lib/src/generate/schema/reader.dart +++ b/lib/src/generate/schema/reader.dart @@ -1,5 +1,6 @@ import 'package:analyzer/dart/ast/ast.dart'; import 'package:analyzer/dart/ast/visitor.dart'; +import 'package:analyzer/dart/ast/token.dart' show Keyword; import 'package:analyzer/dart/element/element.dart'; import 'package:analyzer/dart/element/type.dart'; import 'package:analyzer/dart/element/type_system.dart'; @@ -16,8 +17,11 @@ final class SchemaReader( final CompilationUnit unit, final TypeSystem typeSystem, final DartNames names, - final List declarations, -) { + final List declarations, { + final String? Function(VariableDeclaration)? namespaceOf, + final List additionalRoots = const [], + final bool stableOrder = false, +}) { final _models = {}; final _definitions = {}; final _fieldTypes = {}; @@ -26,6 +30,11 @@ final class SchemaReader( List read() { final consumed = {}; + final namespaces = { + for (final variable in declarations) + variable: namespaceOf?.call(variable), + }; + final grouped = namespaces.values.any((n) => n != null && n != 'public'); for (final variable in declarations) { final declaration = variable.parent as VariableDeclarationList; final element = variable.declaredFragment!.element; @@ -66,29 +75,48 @@ final class SchemaReader( fields.add(field); } final modelName = element.name!; - final row = modelName[0].toUpperCase() + modelName.substring(1); + final namespace = namespaces[variable]; + if (namespace != null && + (!RegExp(r'^[a-zA-Z][a-zA-Z0-9_]*$').hasMatch(namespace) || + databaseMembers.contains(namespace) || + Keyword.keywords[namespace]?.isReservedWord == true)) { + failAt( + variable, + 'NAMESPACE', + 'Schema $namespace cannot be used as a Dart database member.', + ); + } + final prefix = grouped + ? '${namespace![0].toUpperCase()}${namespace.substring(1)}' + : ''; + final row = + '$prefix${modelName[0].toUpperCase()}${modelName.substring(1)}'; final model = ModelEntity( modelName, stringValue(positionalArguments(expression).first), row, fields, + namespace: namespace, + grouped: grouped, ); _models[element] = model; _definitions[model] = expression; consumed.add(expression); } // A nested/list/function declaration must never disappear from the schema. - unit.accept( - _ModelCalls((call) { - if (!consumed.contains(call)) { - failAt( - call, - 'MODEL', - 'Place each model directly in a public final top-level variable.', - ); - } - }), - ); + for (final root in [unit, ...additionalRoots]) { + root.accept( + _ModelCalls((call) { + if (!consumed.contains(call)) { + failAt( + call, + 'MODEL', + 'Place each model directly in a public final top-level variable.', + ); + } + }), + ); + } if (_models.isEmpty) { failAt(unit, 'MODEL', 'No model declarations found.'); } @@ -118,6 +146,10 @@ final class SchemaReader( ); } _validate(); + if (stableOrder) { + return _models.values.toList() + ..sort((a, b) => a.identity.compareTo(b.identity)); + } // Preserve local declaration order and existing snapshots. External models // have no local source order; use physical identity so Dart renames cannot // reorder a barrel's migration snapshot. @@ -398,10 +430,26 @@ final class SchemaReader( void _validate() { final tables = {}, indexes = {}; final symbols = {'appSchema', 'AppTables', ...generatedTypeNames}; + if (_models.values.first.grouped) { + for (final namespace in _models.values.map((m) => m.namespace!).toSet()) { + final type = + '${namespace[0].toUpperCase()}${namespace.substring(1)}Tables'; + if (!symbols.add(type)) { + failAt(unit, 'NAME', 'Generated schema type $type is ambiguous.'); + } + } + } for (final model in _models.values) { final node = _definitions[model]!; - if (!tables.add(model.table)) { - failAt(node, 'DUPLICATE', 'Duplicate physical table ${model.table}.'); + if (!tables.add(model.identity)) { + final previous = _models.values.firstWhere( + (m) => m != model && m.identity == model.identity, + ); + failAt( + node, + 'DUPLICATE', + 'Duplicate physical table ${model.identity}; also declared in ${(_definitions[previous]!.root as CompilationUnit).declaredFragment!.element.uri}.', + ); } if (model.fields.map((f) => f.column).toSet().length != model.fields.length) { @@ -416,8 +464,8 @@ final class SchemaReader( model.fieldsType, model.setType, '${model.symbol}Updates', - '${model.name}Schema', - '${model.name}Table', + '${model.binding}Schema', + '${model.binding}Table', for (final field in model.fields) columnSymbol(model, field), ]) { if (!symbols.add(symbol)) { @@ -451,7 +499,7 @@ final class SchemaReader( } } for (final index in model.indexes) { - if (!indexes.add(index.name)) { + if (!indexes.add('${model.namespace ?? ''}.${index.name}')) { failAt(node, 'DUPLICATE', 'Duplicate index name ${index.name}.'); } } diff --git a/lib/src/generate/schema/sources.dart b/lib/src/generate/schema/sources.dart index 20d3631e..622cb4a6 100644 --- a/lib/src/generate/schema/sources.dart +++ b/lib/src/generate/schema/sources.dart @@ -27,18 +27,24 @@ VariableElement? modelVariable(Element? element) { Future> schemaSources( CompilationUnit root, LibraryElement library, - Future Function(LibraryElement) resolve, -) async { - final units = {library: root}; + Future Function(LibraryElement) resolve, { + List additionalRoots = const [], +}) async { + final roots = [root, ...additionalRoots]; + final units = { + for (final unit in roots) unit.declaredFragment!.element: unit, + }; final result = {}; - final exports = library.exportNamespace.definedNames2.values + final exports = units.keys + .expand((library) => library.exportNamespace.definedNames2.values) .map(modelVariable) .whereType() .toSet(); final pending = [ - for (final variable in topLevelVariables(root)) - if (isModel(variable.declaredFragment!.element.type)) - variable.declaredFragment!.element, + for (final unit in roots) + for (final variable in topLevelVariables(unit)) + if (isModel(variable.declaredFragment!.element.type)) + variable.declaredFragment!.element, ...exports, ]; for (var i = 0; i < pending.length; i++) { diff --git a/lib/src/migrate/backfill.dart b/lib/src/migrate/backfill.dart index e65e5e5b..3e1bacaf 100644 --- a/lib/src/migrate/backfill.dart +++ b/lib/src/migrate/backfill.dart @@ -11,7 +11,7 @@ import 'migration.dart' show Migration; import 'mysql_schema.dart' show isMysqlFamily, mysqlColumnType; import 'recovery.dart' show checkpoint, probe; import 'snapshot.dart' show SchemaSnapshot; -import 'sql_utils.dart' show quoteIdentifier; +import 'sql_utils.dart' show quoteIdentifier, quoteQualified; import 'step.dart' show Backfill; /// Durable counters and primary-key cursors. Keys use lossless strings (binary @@ -104,13 +104,14 @@ Future verifyBackfill(SqlDatabase db, Backfill step) async { if (db.dialect == SqlDialect.postgres) { final security = await db.execute( SqlCommand( - r'''SELECT row_security_active(c.oid), pg_table_is_visible(c.oid) -FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace -WHERE n.nspname = current_schema() AND c.relname = $1''', - [step.table.name], + r'''SELECT pg_catalog.row_security_active(c.oid), pg_catalog.pg_table_is_visible(c.oid) +FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace +WHERE n.nspname = coalesce($2::text, pg_catalog.current_schema()) AND c.relname = $1''', + [step.table.name, step.table.namespace], ), ); - if (security.rows.single[0] == true || security.rows.single[1] != true) { + if (security.rows.single[0] == true || + (step.table.namespace == null && security.rows.single[1] != true)) { throw const OrmException( 'MIGRATION.BACKFILL_SCOPE', 'The historical table is shadowed or has active row security. Backfill requires an unshadowed table and a role that sees all its rows.', @@ -151,7 +152,7 @@ Future backfillChunk( budget.check(); final cursor = _BackfillCursor(step); final names = step.table.primaryKey.map(quoteIdentifier).toList(), - table = quoteIdentifier(step.table.name); + table = quoteQualified(step.table.name, step.table.namespace); final columns = names.join(', '), key = _tuple(names); phase('scan'); var progress = saved; @@ -233,7 +234,7 @@ Future backfillChunk( 'UPDATE $table SET ${step.set.entries.map((e) => '${quoteIdentifier(e.key)} = ${e.value}').join(', ')} WHERE $predicate${isMysqlFamily(tx.dialect) ? '' : ' AND (${step.where}) RETURNING $columns'}', parameters, ), - changedTables: [step.table.name], + changedTables: [step.table.identity], ); final expected = keys.map(jsonEncode).toSet(), actual = (isMysqlFamily(tx.dialect) ? selected.rows : updated.rows) diff --git a/lib/src/migrate/catalog.dart b/lib/src/migrate/catalog.dart index 04590706..925518b0 100644 --- a/lib/src/migrate/catalog.dart +++ b/lib/src/migrate/catalog.dart @@ -1,6 +1,7 @@ import 'dart:convert' show jsonEncode; import '../../driver.dart' show Backend, SqlCommand, SqlDialect; +import '../../values.dart' show OrmException; import '../../runtime.dart' show SqlDatabase; import '../../schema_model.dart' show Column, ForeignKey, IndexSchema; import '../../values.dart' @@ -44,6 +45,9 @@ final class CatalogObject { /// [unmanaged] preserves descriptions of objects a migration must not silently /// replace, such as custom triggers or unsupported index definitions. final class TableInfo { + /// PostgreSQL schema explicitly selected for this inspection. + final String? namespace; + /// Physical table name used for this inspection. final String name; @@ -73,6 +77,7 @@ final class TableInfo { /// Supplied lists are retained; use [inspectTable] to obtain live metadata. const TableInfo({ required this.name, + this.namespace, required this.columns, required this.primaryKey, required this.uniqueKeys, @@ -99,9 +104,19 @@ final class SchemaVerification { } /// Reads one physical [table] without changing its schema or data. -Future inspectTable(SqlDatabase db, String table) async { +Future inspectTable( + SqlDatabase db, + String table, { + String? namespace, +}) async { + if (namespace != null && db.dialect != SqlDialect.postgres) { + throw const OrmException( + 'SCHEMA.NAMESPACE', + 'Database schemas require PostgreSQL.', + ); + } if (isMysqlFamily(db.dialect)) return mysqlTable(db, table); - final columns = await inspectColumns(db, table); + final columns = await inspectColumns(db, table, namespace: namespace); final primary = [], unique = >[], indexes = []; @@ -233,29 +248,31 @@ Future inspectTable(SqlDatabase db, String table) async { SqlCommand( r''' SELECT c.conname, c.contype::text, - ARRAY(SELECT a.attname::text FROM unnest(c.conkey) WITH ORDINALITY k(num, ord) - JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = k.num ORDER BY k.ord), + ARRAY(SELECT a.attname::text FROM pg_catalog.unnest(c.conkey) WITH ORDINALITY k(num, ord) + JOIN pg_catalog.pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = k.num ORDER BY k.ord), t.relname, - ARRAY(SELECT a.attname::text FROM unnest(c.confkey) WITH ORDINALITY k(num, ord) - JOIN pg_attribute a ON a.attrelid = c.confrelid AND a.attnum = k.num ORDER BY k.ord), + ARRAY(SELECT a.attname::text FROM pg_catalog.unnest(c.confkey) WITH ORDINALITY k(num, ord) + JOIN pg_catalog.pg_attribute a ON a.attrelid = c.confrelid AND a.attnum = k.num ORDER BY k.ord), c.confdeltype::text, c.confupdtype::text, c.confmatchtype::text, - c.condeferrable, c.convalidated, pg_get_constraintdef(c.oid), tn.nspname, - coalesce((to_jsonb(i)->>'indnullsnotdistinct')::boolean, false), - pg_get_expr(c.conbin, c.conrelid), c.connoinherit, c.conislocal, c.coninhcount, - coalesce((to_jsonb(c)->>'conenforced')::boolean, true) -FROM pg_constraint c JOIN pg_class r ON r.oid = c.conrelid -JOIN pg_namespace n ON n.oid = r.relnamespace -LEFT JOIN pg_class t ON t.oid = c.confrelid -LEFT JOIN pg_namespace tn ON tn.oid = t.relnamespace -LEFT JOIN pg_index i ON i.indexrelid = c.conindid -WHERE n.nspname = current_schema() AND r.relname = $1''', - [table], + c.condeferrable, c.convalidated, pg_catalog.pg_get_constraintdef(c.oid), tn.nspname, + coalesce((pg_catalog.to_jsonb(i)->>'indnullsnotdistinct')::boolean, false), + pg_catalog.pg_get_expr(c.conbin, c.conrelid), c.connoinherit, c.conislocal, c.coninhcount, + coalesce((pg_catalog.to_jsonb(c)->>'conenforced')::boolean, true) +FROM pg_catalog.pg_constraint c JOIN pg_catalog.pg_class r ON r.oid = c.conrelid +JOIN pg_catalog.pg_namespace n ON n.oid = r.relnamespace +LEFT JOIN pg_catalog.pg_class t ON t.oid = c.confrelid +LEFT JOIN pg_catalog.pg_namespace tn ON tn.oid = t.relnamespace +LEFT JOIN pg_catalog.pg_index i ON i.indexrelid = c.conindid +WHERE n.nspname = coalesce($2::text, pg_catalog.current_schema()) AND r.relname = $1''', + [table, namespace], ), ); - final schemaName = (await db.execute(SqlCommand('SELECT current_schema()'))) - .rows - .single - .single; + final schemaName = + namespace ?? + (await db.execute(SqlCommand('SELECT pg_catalog.current_schema()'))) + .rows + .single + .single; for (final row in constraints.rows) { final kind = row[1] as String, keys = (row[2] as List).cast(); @@ -276,12 +293,15 @@ WHERE n.nspname = current_schema() AND r.relname = $1''', row[7] == 's' && row[8] == false && row[9] == true && - row[11] == schemaName) { + (namespace != null || row[11] == schemaName)) { foreign.add( ForeignKey( keys, row[3] as String, (row[4] as List).cast(), + targetNamespace: namespace == null && row[11] == schemaName + ? null + : row[11] as String, onDelete: switch (row[5]) { 'a' => 'NO ACTION', 'r' => 'RESTRICT', @@ -309,22 +329,22 @@ WHERE n.nspname = current_schema() AND r.relname = $1''', SqlCommand( r''' SELECT ic.relname, i.indisunique, i.indisvalid AND i.indisready AND i.indislive, - ARRAY(SELECT a.attname::text FROM unnest(i.indkey) WITH ORDINALITY k(num, ord) - LEFT JOIN pg_attribute a ON a.attrelid = i.indrelid AND a.attnum = k.num ORDER BY k.ord), - pg_get_indexdef(i.indexrelid), i.indexprs IS NULL AND i.indpred IS NULL + ARRAY(SELECT a.attname::text FROM pg_catalog.unnest(i.indkey) WITH ORDINALITY k(num, ord) + LEFT JOIN pg_catalog.pg_attribute a ON a.attrelid = i.indrelid AND a.attnum = k.num ORDER BY k.ord), + pg_catalog.pg_get_indexdef(i.indexrelid), i.indexprs IS NULL AND i.indpred IS NULL AND i.indnatts = i.indnkeyatts AND am.amname = 'btree' AND ic.reloptions IS NULL - AND NOT coalesce((to_jsonb(i)->>'indnullsnotdistinct')::boolean, false) - AND NOT EXISTS(SELECT 1 FROM unnest(i.indoption) v WHERE v <> 0) - AND NOT EXISTS(SELECT 1 FROM unnest(i.indclass) v JOIN pg_opclass o ON o.oid = v WHERE NOT o.opcdefault) - AND NOT EXISTS(SELECT 1 FROM unnest(i.indkey, i.indcollation) k(num, collation_oid) - JOIN pg_attribute a ON a.attrelid = t.oid AND a.attnum = k.num WHERE k.collation_oid <> a.attcollation) -FROM pg_index i JOIN pg_class t ON t.oid = i.indrelid -JOIN pg_namespace n ON n.oid = t.relnamespace JOIN pg_class ic ON ic.oid = i.indexrelid -JOIN pg_am am ON am.oid = ic.relam -WHERE n.nspname = current_schema() AND t.relname = $1 -AND NOT EXISTS (SELECT 1 FROM pg_constraint c WHERE c.conindid = i.indexrelid AND c.contype IN ('p', 'u', 'x'))''', - [table], + AND NOT coalesce((pg_catalog.to_jsonb(i)->>'indnullsnotdistinct')::boolean, false) + AND NOT EXISTS(SELECT 1 FROM pg_catalog.unnest(i.indoption) v WHERE v <> 0) + AND NOT EXISTS(SELECT 1 FROM pg_catalog.unnest(i.indclass) v JOIN pg_catalog.pg_opclass o ON o.oid = v WHERE NOT o.opcdefault) + AND NOT EXISTS(SELECT 1 FROM ROWS FROM (pg_catalog.unnest(i.indkey), pg_catalog.unnest(i.indcollation)) k(num, collation_oid) + JOIN pg_catalog.pg_attribute a ON a.attrelid = t.oid AND a.attnum = k.num WHERE k.collation_oid <> a.attcollation) +FROM pg_catalog.pg_index i JOIN pg_catalog.pg_class t ON t.oid = i.indrelid +JOIN pg_catalog.pg_namespace n ON n.oid = t.relnamespace JOIN pg_catalog.pg_class ic ON ic.oid = i.indexrelid +JOIN pg_catalog.pg_am am ON am.oid = ic.relam +WHERE n.nspname = coalesce($2::text, pg_catalog.current_schema()) AND t.relname = $1 +AND NOT EXISTS (SELECT 1 FROM pg_catalog.pg_constraint c WHERE c.conindid = i.indexrelid AND c.contype IN ('p', 'u', 'x'))''', + [table, namespace], ), ); for (final row in list.rows) { @@ -346,20 +366,20 @@ AND NOT EXISTS (SELECT 1 FROM pg_constraint c WHERE c.conindid = i.indexrelid AN final extras = await db.execute( SqlCommand( r''' -SELECT 'trigger', t.tgname, pg_get_triggerdef(t.oid) -FROM pg_trigger t JOIN pg_class c ON c.oid = t.tgrelid JOIN pg_namespace n ON n.oid = c.relnamespace -WHERE c.relname = $1 AND n.nspname = current_schema() AND NOT t.tgisinternal +SELECT 'trigger', t.tgname, pg_catalog.pg_get_triggerdef(t.oid) +FROM pg_catalog.pg_trigger t JOIN pg_catalog.pg_class c ON c.oid = t.tgrelid JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace +WHERE c.relname = $1 AND n.nspname = coalesce($2::text, pg_catalog.current_schema()) AND NOT t.tgisinternal UNION ALL SELECT 'policy', policyname, - jsonb_build_object('permissive', permissive, 'roles', roles, 'command', cmd, + pg_catalog.jsonb_build_object('permissive', permissive, 'roles', roles, 'command', cmd, 'using', qual, 'withCheck', with_check)::text -FROM pg_policies WHERE schemaname = current_schema() AND tablename = $1 +FROM pg_catalog.pg_policies WHERE schemaname = coalesce($2::text, pg_catalog.current_schema()) AND tablename = $1 UNION ALL SELECT 'row_security', c.relname, - jsonb_build_object('enabled', c.relrowsecurity, 'forced', c.relforcerowsecurity)::text -FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace -WHERE c.relname = $1 AND n.nspname = current_schema() + pg_catalog.jsonb_build_object('enabled', c.relrowsecurity, 'forced', c.relforcerowsecurity)::text +FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace +WHERE c.relname = $1 AND n.nspname = coalesce($2::text, pg_catalog.current_schema()) AND (c.relrowsecurity OR c.relforcerowsecurity - OR EXISTS (SELECT 1 FROM pg_policy p WHERE p.polrelid = c.oid))''', - [table], + OR EXISTS (SELECT 1 FROM pg_catalog.pg_policy p WHERE p.polrelid = c.oid))''', + [table, namespace], ), ); for (final row in extras.rows) { @@ -370,6 +390,7 @@ AND (c.relrowsecurity OR c.relforcerowsecurity } return TableInfo( name: table, + namespace: namespace, columns: columns, primaryKey: primary, uniqueKeys: unique, @@ -391,13 +412,17 @@ Future verifySchema( if (isMysqlFamily(db.dialect)) return mysqlVerifySchema(db, expected); final differences = [], unmanaged = []; for (final table in expected.tables) { - final actual = await inspectTable(db, table.name); + final actual = await inspectTable( + db, + table.name, + namespace: table.namespace, + ); unmanaged.addAll(actual.unmanaged); final columns = {for (final c in actual.columns) c.name: c}; var checkContextMatches = true; for (final column in table.columns) { final found = columns.remove(column.name), - path = '${table.name}.${column.name}'; + path = '${table.identity}.${column.name}'; if (found == null) { checkContextMatches = false; differences.add('$path is missing'); @@ -445,7 +470,7 @@ Future verifySchema( } } for (final name in columns.keys) { - differences.add('${table.name}.$name is unmanaged'); + differences.add('${table.identity}.$name is unmanaged'); } differences.addAll( await verifyComputed( @@ -457,7 +482,7 @@ Future verifySchema( ); void compare(String kind, Object? desired, Object? found) { if (migrationHash(desired) != migrationHash(found)) { - differences.add('${table.name} $kind differs'); + differences.add('${table.identity} $kind differs'); } } @@ -469,16 +494,26 @@ Future verifySchema( // Expected SQL may no longer resolve when a referenced column/type drifted. // Report that schema drift instead of attempting an invalid EXPLAIN. final checkMatches = checkContextMatches - ? await matchChecks(db, table.name, table.checks, actual.checks) + ? await matchChecks( + db, + table.name, + table.checks, + actual.checks, + namespace: table.namespace, + ) : List.filled(table.checks.length, null); if (checkMatches.any((i) => i == null) || checkMatches.length != actual.checks.length) { - differences.add('${table.name} checks differs'); + differences.add('${table.identity} checks differs'); for (var i = 0; i < actual.checks.length; i++) { if (!checkMatches.contains(i)) { final c = actual.checks[i]; unmanaged.add( - CatalogObject('check', c.name ?? '${table.name}#$i', c.expression), + CatalogObject( + 'check', + c.name ?? '${table.identity}#$i', + c.expression, + ), ); } } diff --git a/lib/src/migrate/checks.dart b/lib/src/migrate/checks.dart index 169ab77b..7b107754 100644 --- a/lib/src/migrate/checks.dart +++ b/lib/src/migrate/checks.dart @@ -5,7 +5,7 @@ import '../../runtime.dart' show SqlDatabase; import '../../schema_model.dart' show CheckSchema, TableSchema; import '../../values.dart' show Codecs, OrmException; import 'catalog.dart' show inspectTable; -import 'sql_utils.dart' show quoteIdentifier; +import 'sql_utils.dart' show quoteIdentifier, quoteQualified; import 'sqlite_checks.dart' show sqliteName, sqliteTokens; /// An enforced, validated row CHECK read from this database's catalog. @@ -23,6 +23,7 @@ final class CheckInfo { TableSchema withChecks(TableSchema table, List checks) => TableSchema( table.name, + namespace: table.namespace, columns: table.columns, primaryKey: table.primaryKey, uniqueKeys: table.uniqueKeys, @@ -90,8 +91,9 @@ String _checkSignature(String expression) { Future> checkExpressions( SqlDatabase db, String table, - List expressions, -) async { + List expressions, { + String? namespace, +}) async { if (db.dialect == SqlDialect.sqlite) { return expressions.map(_checkSignature).toList(); } @@ -99,7 +101,7 @@ Future> checkExpressions( final result = await db.execute( SqlCommand( 'EXPLAIN (VERBOSE, FORMAT JSON, COSTS OFF) SELECT ' - '${expressions.map((e) => '($e\n)').join(', ')} FROM ONLY ${quoteIdentifier(table)}', + '${expressions.map((e) => '($e\n)').join(', ')} FROM ONLY ${quoteQualified(table, namespace)}', ), ); final json = Codecs.json.decode(result.rows.single.single) as List; @@ -117,13 +119,14 @@ Future> matchChecks( SqlDatabase db, String table, List expected, - List actual, -) async { + List actual, { + String? namespace, +}) async { if (expected.isEmpty) return const []; final signatures = await checkExpressions(db, table, [ ...expected.map((c) => c.expression(db.dialect)), ...actual.map((c) => c.expression), - ]); + ], namespace: namespace); final used = {}, matches = []; for (var i = 0; i < expected.length; i++) { int? found; @@ -151,10 +154,17 @@ Future> matchChecks( Future dropCheck( SqlDatabase db, String table, - CheckSchema check, -) async { - final actual = (await inspectTable(db, table)).checks; - final match = (await matchChecks(db, table, [check], actual)).single; + CheckSchema check, { + String? namespace, +}) async { + final actual = (await inspectTable(db, table, namespace: namespace)).checks; + final match = (await matchChecks( + db, + table, + [check], + actual, + namespace: namespace, + )).single; if (match == null || actual[match].name == null) { throw OrmException( 'MIGRATION.DRIFT', @@ -163,7 +173,7 @@ Future dropCheck( } await db.execute( SqlCommand( - 'ALTER TABLE ${quoteIdentifier(table)} DROP CONSTRAINT ${quoteIdentifier(actual[match].name!)}', + 'ALTER TABLE ${quoteQualified(table, namespace)} DROP CONSTRAINT ${quoteIdentifier(actual[match].name!)}', ), ); } diff --git a/lib/src/migrate/columns.dart b/lib/src/migrate/columns.dart index 9f5fa2c9..8891fba9 100644 --- a/lib/src/migrate/columns.dart +++ b/lib/src/migrate/columns.dart @@ -2,6 +2,7 @@ import '../../driver.dart' show Backend, SqlCommand, SqlDialect; import '../../runtime.dart' show SqlDatabase; +import '../../values.dart' show OrmException; import '../../schema_model.dart' show Column, ComputedColumn, ComputedStorage, TableSchema; import 'catalog.dart' show normalizeDefault; @@ -99,8 +100,15 @@ final class ColumnInfo { /// Reads storage types, nullability, defaults, and computed-column metadata. Future> inspectColumns( SqlDatabase db, - String table, -) async { + String table, { + String? namespace, +}) async { + if (namespace != null && db.dialect != SqlDialect.postgres) { + throw const OrmException( + 'SCHEMA.NAMESPACE', + 'Database schemas require PostgreSQL.', + ); + } if (isMysqlFamily(db.dialect)) return mysqlColumns(db, table); if (db.dialect == SqlDialect.sqlite) { final rows = await db.execute( @@ -164,15 +172,15 @@ Future> inspectColumns( SqlCommand( ''' SELECT a.attname, pg_catalog.format_type(a.atttypid, a.atttypmod), - NOT a.attnotnull, pg_get_expr(d.adbin, d.adrelid), + NOT a.attnotnull, pg_catalog.pg_get_expr(d.adbin, d.adrelid), a.attidentity <> '' OR a.attgenerated <> '', a.attgenerated::text FROM pg_catalog.pg_attribute a JOIN pg_catalog.pg_class c ON c.oid = a.attrelid JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace LEFT JOIN pg_catalog.pg_attrdef d ON d.adrelid = a.attrelid AND d.adnum = a.attnum -WHERE c.relname = \$1 AND n.nspname = current_schema() AND a.attnum > 0 AND NOT a.attisdropped +WHERE c.relname = \$1 AND n.nspname = coalesce(\$2::text, pg_catalog.current_schema()) AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum''', - [table], + [table, namespace], ), ); return [ @@ -212,12 +220,16 @@ Future> verifyColumns( ) async { final differences = []; for (final table in tables) { - final inspected = await inspectColumns(db, table.name); + final inspected = await inspectColumns( + db, + table.name, + namespace: table.namespace, + ); final actual = {for (final c in inspected) c.name: c}; var contextMatches = true; for (final expected in table.columns) { final column = actual.remove(expected.name); - final path = '${table.name}.${expected.name}'; + final path = '${table.identity}.${expected.name}'; if (column == null) { contextMatches = false; differences.add('$path is missing'); @@ -247,7 +259,7 @@ Future> verifyColumns( } } for (final extra in actual.keys) { - differences.add('${table.name}.$extra is unmanaged'); + differences.add('${table.identity}.$extra is unmanaged'); } differences.addAll( await verifyComputed( diff --git a/lib/src/migrate/computed.dart b/lib/src/migrate/computed.dart index 6609fbfc..e4a74dea 100644 --- a/lib/src/migrate/computed.dart +++ b/lib/src/migrate/computed.dart @@ -146,7 +146,7 @@ Future> verifyComputed( final found = actual.where((c) => c.name == column.name).firstOrNull; final expected = column.computed, observed = found?.computed; if (expected == null && observed == null) continue; - final path = '${table.name}.${column.name}'; + final path = '${table.identity}.${column.name}'; if (!contextMatches || expected == null || observed == null || @@ -170,7 +170,12 @@ Future> verifyComputed( : pair, ); } - final signatures = await checkExpressions(db, table.name, expressions); + final signatures = await checkExpressions( + db, + table.name, + expressions, + namespace: table.namespace, + ); for (var i = 0; i < paths.length; i++) { if (signatures[i * 2] != signatures[i * 2 + 1]) { differences.add('${paths[i]} computed expression or storage differs'); @@ -181,6 +186,7 @@ Future> verifyComputed( TableSchema materializedColumns(TableSchema table) => TableSchema( table.name, + namespace: table.namespace, columns: [ for (final c in table.columns) Column( diff --git a/lib/src/migrate/diff.dart b/lib/src/migrate/diff.dart index f5703552..d2f2c481 100644 --- a/lib/src/migrate/diff.dart +++ b/lib/src/migrate/diff.dart @@ -19,7 +19,7 @@ import 'schema.dart' sameStorage; import 'snapshot.dart' show SchemaSnapshot, columnJson, foreignKeyJson, indexJson; -import 'sql_utils.dart' show migrationHash, quoteIdentifier; +import 'sql_utils.dart' show migrationHash, quoteIdentifier, quoteQualified; import 'step.dart' show DropConstraint, DropTable, ExecuteSql, MigrationStep, RebuildTable; @@ -61,9 +61,32 @@ Migration diffSchema( using: using, ); } + final sourceTables = {for (final t in from.tables) t.identity: t}; + final targetTables = {for (final t in to.tables) t.identity: t}; + String tableName(String name) => renames.tables[name] ?? name; + String quoted(TableSchema table) => + quoteQualified(table.name, table.namespace); + String tableSql(String identity) => + quoted(targetTables[identity] ?? sourceTables[identity]!); final renameSql = []; - final oldNames = from.tables.map((t) => t.name).toSet(); - final newNames = to.tables.map((t) => t.name).toSet(); + if (dialect == SqlDialect.postgres) { + final previousNamespaces = from.tables.map((t) => t.namespace).toSet(); + final namespaces = + to.tables + .map((t) => t.namespace) + .nonNulls + .toSet() + .difference(previousNamespaces.nonNulls.toSet()) + .toList() + ..sort(); + for (final namespace in namespaces) { + renameSql.add( + ExecuteSql('CREATE SCHEMA IF NOT EXISTS ${quoteIdentifier(namespace)}'), + ); + } + } + final oldNames = from.tables.map((t) => t.identity).toSet(); + final newNames = to.tables.map((t) => t.identity).toSet(); void validateRenames( Map map, Set old, @@ -85,18 +108,36 @@ Migration diffSchema( validateRenames(renames.tables, oldNames, newNames); for (final entry in renames.tables.entries) { - renameSql.add( - ExecuteSql( - 'ALTER TABLE ${quoteIdentifier(entry.key)} RENAME TO ${quoteIdentifier(entry.value)}', - ), - ); + final old = sourceTables[entry.key]!, next = targetTables[entry.value]!; + var namespace = old.namespace; + if (old.namespace != next.namespace) { + if (dialect != SqlDialect.postgres || + old.namespace == null || + next.namespace == null) { + throw const OrmException( + 'MIGRATION.RENAME', + 'Schema moves require explicit PostgreSQL source and target namespaces.', + ); + } + renameSql.add( + ExecuteSql( + 'ALTER TABLE ${quoted(old)} SET SCHEMA ${quoteIdentifier(next.namespace!)}', + ), + ); + namespace = next.namespace; + } + if (old.name != next.name) { + renameSql.add( + ExecuteSql( + 'ALTER TABLE ${quoteQualified(old.name, namespace)} RENAME TO ${quoteIdentifier(next.name)}', + ), + ); + } } - final renamedTables = { - for (final t in from.tables) renames.tables[t.name] ?? t.name: t, - }; + final renamedTables = {for (final t in from.tables) tableName(t.identity): t}; for (final entry in renames.columns.entries) { final old = renamedTables[entry.key], - next = to.tables.where((t) => t.name == entry.key).firstOrNull; + next = to.tables.where((t) => t.identity == entry.key).firstOrNull; if (old == null || next == null) { throw const OrmException( 'MIGRATION.RENAME', @@ -111,20 +152,19 @@ Migration diffSchema( for (final rename in entry.value.entries) { renameSql.add( ExecuteSql( - 'ALTER TABLE ${quoteIdentifier(entry.key)} RENAME COLUMN ${quoteIdentifier(rename.key)} TO ${quoteIdentifier(rename.value)}', + 'ALTER TABLE ${tableSql(entry.key)} RENAME COLUMN ${quoteIdentifier(rename.key)} TO ${quoteIdentifier(rename.value)}', ), ); } } - String tableName(String name) => renames.tables[name] ?? name; String columnName(String table, String column) => renames.columns[tableName(table)]?[column] ?? column; final computedRenames = from.tables .where( (t) => t.columns.any((c) => c.computed != null) && - (renames.tables.containsKey(t.name) || - (renames.columns[tableName(t.name)]?.isNotEmpty ?? false)), + (renames.tables.containsKey(t.identity) || + (renames.columns[tableName(t.identity)]?.isNotEmpty ?? false)), ) .toList(); // CHECK SQL is deliberately not rewritten as text. Remove it before native @@ -133,19 +173,20 @@ Migration diffSchema( .where( (t) => t.checks.isNotEmpty && - (renames.tables.containsKey(t.name) || - (renames.columns[tableName(t.name)]?.isNotEmpty ?? false)), + (renames.tables.containsKey(t.identity) || + (renames.columns[tableName(t.identity)]?.isNotEmpty ?? false)), ) .toList(); final before = { for (final table in from.tables) - tableName(table.name): TableSchema( - tableName(table.name), + tableName(table.identity): TableSchema( + (targetTables[tableName(table.identity)] ?? table).name, + namespace: (targetTables[tableName(table.identity)] ?? table).namespace, checks: checkedRenames.contains(table) ? const [] : table.checks, columns: [ for (final c in table.columns) Column( - columnName(table.name, c.name), + columnName(table.identity, c.name), c.codec, nullable: c.nullable, generated: c.generated, @@ -158,32 +199,37 @@ Migration diffSchema( ), ], primaryKey: table.primaryKey - .map((c) => columnName(table.name, c)) + .map((c) => columnName(table.identity, c)) .toList(), uniqueKeys: [ for (final key in table.uniqueKeys) - key.map((c) => columnName(table.name, c)).toList(), + key.map((c) => columnName(table.identity, c)).toList(), ], indexes: [ for (final index in table.indexes) IndexSchema( index.name, - index.columns.map((c) => columnName(table.name, c)).toList(), + index.columns.map((c) => columnName(table.identity, c)).toList(), unique: index.unique, ), ], foreignKeys: [ for (final key in table.foreignKeys) ForeignKey( - key.columns.map((c) => columnName(table.name, c)).toList(), - tableName(key.target), - key.targetColumns.map((c) => columnName(key.target, c)).toList(), + key.columns.map((c) => columnName(table.identity, c)).toList(), + (targetTables[tableName(key.targetIdentity)]?.name ?? key.target), + targetNamespace: + targetTables[tableName(key.targetIdentity)]?.namespace ?? + key.targetNamespace, + key.targetColumns + .map((c) => columnName(key.targetIdentity, c)) + .toList(), onDelete: key.onDelete, ), ], ), }; - final after = {for (final t in to.tables) t.name: t}; + final after = targetTables; final removedTables = before.keys.toSet().difference(after.keys.toSet()); final addedTables = after.keys.toSet().difference(before.keys.toSet()); final shared = before.keys.toSet().intersection(after.keys.toSet()); @@ -299,7 +345,7 @@ Migration diffSchema( 'kind': 'c', 'name': check.name, 'expression': check.expression(dialect), - }), + }, namespace: table.namespace), ); } } @@ -333,37 +379,50 @@ Migration diffSchema( } for (final old in before.values) { - final next = after[old.name]; + final next = after[old.identity]; for (final key in old.foreignKeys) { final retained = next?.foreignKeys.any((k) => _sameForeignKey(k, key)) ?? false; if (!retained || - keysChanged(key.target) || - typesChanged(key.target) || - typesChanged(old.name)) { + keysChanged(key.targetIdentity) || + typesChanged(key.targetIdentity) || + typesChanged(old.identity)) { steps.add( - DropConstraint(old.name, {'kind': 'f', ...foreignKeyJson(key)}), + DropConstraint(old.name, { + 'kind': 'f', + ...foreignKeyJson(key), + }, namespace: old.namespace), ); - if (retained) addedForeignKeys.add((old.name, key)); + if (retained) addedForeignKeys.add((old.identity, key)); } } if (next != null) { for (final key in next.foreignKeys) { if (!old.foreignKeys.any((k) => _sameForeignKey(k, key))) { - addedForeignKeys.add((old.name, key)); + addedForeignKeys.add((old.identity, key)); } } } } } for (final name in removedTables) { - steps.add(DropTable(name)); + steps.add( + DropTable(before[name]!.name, namespace: before[name]!.namespace), + ); } for (final name in addedTables) { final table = after[name]!; steps.add(ExecuteSql(createTable(table, dialect))); for (final index in table.indexes) { - steps.add(ExecuteSql(createIndexSql(name, index))); + steps.add( + ExecuteSql( + createIndexSql( + after[name]!.name, + index, + namespace: after[name]!.namespace, + ), + ), + ); } if (dialect == SqlDialect.postgres) { addedForeignKeys.addAll(table.foreignKeys.map((k) => (name, k))); @@ -372,7 +431,7 @@ Migration diffSchema( for (final name in shared) { var old = before[name]!; if (dialect == SqlDialect.sqlite && - computedRenames.any((t) => tableName(t.name) == name)) { + computedRenames.any((t) => tableName(t.identity) == name)) { old = materializedColumns(old); } final next = after[name]!; @@ -426,24 +485,32 @@ Migration diffSchema( if (dialect == SqlDialect.postgres) { for (final check in checks.removed) { steps.add( - DropConstraint(name, { + DropConstraint(next.name, { 'kind': 'c', 'name': check.name, 'expression': check.expression(dialect), - }), + }, namespace: next.namespace), ); } if (migrationHash(old.primaryKey) != migrationHash(next.primaryKey) && old.primaryKey.isNotEmpty) { steps.add( - DropConstraint(name, {'kind': 'p', 'columns': old.primaryKey}), + DropConstraint(next.name, { + 'kind': 'p', + 'columns': old.primaryKey, + }, namespace: next.namespace), ); } for (final key in old.uniqueKeys) { if (!next.uniqueKeys.any( (k) => migrationHash(k) == migrationHash(key), )) { - steps.add(DropConstraint(name, {'kind': 'u', 'columns': key})); + steps.add( + DropConstraint(next.name, { + 'kind': 'u', + 'columns': key, + }, namespace: next.namespace), + ); } } } @@ -451,27 +518,31 @@ Migration diffSchema( if (!next.indexes.any( (i) => migrationHash(indexJson(i)) == migrationHash(indexJson(index)), )) { - steps.add(ExecuteSql('DROP INDEX ${quoteIdentifier(index.name)}')); + steps.add( + ExecuteSql( + 'DROP INDEX ${quoteQualified(index.name, next.namespace)}', + ), + ); } } for (final column in removed) { steps.add( ExecuteSql( - 'ALTER TABLE ${quoteIdentifier(name)} DROP COLUMN ${quoteIdentifier(column)}', + 'ALTER TABLE ${tableSql(name)} DROP COLUMN ${quoteIdentifier(column)}', ), ); } for (final column in added) { steps.add( ExecuteSql( - 'ALTER TABLE ${quoteIdentifier(name)} ADD COLUMN ${columnDefinition(newColumns[column]!, dialect)}', + 'ALTER TABLE ${tableSql(name)} ADD COLUMN ${columnDefinition(newColumns[column]!, dialect)}', ), ); } for (final column in changed) { final a = oldColumns[column]!, b = newColumns[column]!; final prefix = - 'ALTER TABLE ${quoteIdentifier(name)} ALTER COLUMN ${quoteIdentifier(column)}'; + 'ALTER TABLE ${tableSql(name)} ALTER COLUMN ${quoteIdentifier(column)}'; final typeChanged = !sameStorage(a, b); if (a.computed != null && b.computed == null) { steps.add(ExecuteSql('$prefix DROP EXPRESSION')); @@ -514,7 +585,7 @@ Migration diffSchema( if (checks.added.isNotEmpty) { steps.add( ExecuteSql( - 'ALTER TABLE ${quoteIdentifier(name)} ' + 'ALTER TABLE ${tableSql(name)} ' '${checks.added.map((c) => 'ADD ${checkDefinition(c, dialect)}').join(', ')}', ), ); @@ -523,7 +594,7 @@ Migration diffSchema( next.primaryKey.isNotEmpty) { steps.add( ExecuteSql( - 'ALTER TABLE ${quoteIdentifier(name)} ADD PRIMARY KEY (${next.primaryKey.map(quoteIdentifier).join(', ')})', + 'ALTER TABLE ${tableSql(name)} ADD PRIMARY KEY (${next.primaryKey.map(quoteIdentifier).join(', ')})', ), ); } @@ -533,7 +604,7 @@ Migration diffSchema( )) { steps.add( ExecuteSql( - 'ALTER TABLE ${quoteIdentifier(name)} ADD UNIQUE (${key.map(quoteIdentifier).join(', ')})', + 'ALTER TABLE ${tableSql(name)} ADD UNIQUE (${key.map(quoteIdentifier).join(', ')})', ), ); } @@ -543,13 +614,21 @@ Migration diffSchema( if (!old.indexes.any( (i) => migrationHash(indexJson(i)) == migrationHash(indexJson(index)), )) { - steps.add(ExecuteSql(createIndexSql(name, index))); + steps.add( + ExecuteSql( + createIndexSql( + after[name]!.name, + index, + namespace: after[name]!.namespace, + ), + ), + ); } } } for (final (name, key) in addedForeignKeys) { steps.add( - ExecuteSql('ALTER TABLE ${quoteIdentifier(name)} ADD ${foreignKey(key)}'), + ExecuteSql('ALTER TABLE ${tableSql(name)} ADD ${foreignKey(key)}'), ); } return Migration.steps( diff --git a/lib/src/migrate/execute.dart b/lib/src/migrate/execute.dart index 3932e889..6b3048b3 100644 --- a/lib/src/migrate/execute.dart +++ b/lib/src/migrate/execute.dart @@ -8,7 +8,7 @@ import 'catalog.dart' show inspectTable, verifySchema; import 'checks.dart' show dropCheck, matchChecks; import 'schema.dart' show coerceColumn, createIndexSql, createTable; import 'snapshot.dart' show SchemaSnapshot, indexJson; -import 'sql_utils.dart' show migrationHash, quoteIdentifier; +import 'sql_utils.dart' show migrationHash, quoteIdentifier, quoteQualified; import 'step.dart' show Backfill, @@ -30,7 +30,9 @@ Future executeStep(SqlDatabase db, MigrationStep step) async { case ExecuteSql(): await db.execute(SqlCommand(step.sql)); case DropTable(): - await db.execute(SqlCommand('DROP TABLE ${quoteIdentifier(step.table)}')); + await db.execute( + SqlCommand('DROP TABLE ${quoteQualified(step.table, step.namespace)}'), + ); case RebuildTable(): await _rebuild(db, step); case DropConstraint(): @@ -42,6 +44,7 @@ Future executeStep(SqlDatabase db, MigrationStep step) async { step.constraint['name'] as String?, step.constraint['expression'] as String, ), + namespace: step.namespace, ); return; } @@ -49,17 +52,18 @@ Future executeStep(SqlDatabase db, MigrationStep step) async { SqlCommand( r''' SELECT c.conname, c.contype::text, - ARRAY(SELECT a.attname::text FROM unnest(c.conkey) WITH ORDINALITY k(num, ord) - JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = k.num ORDER BY k.ord), + ARRAY(SELECT a.attname::text FROM pg_catalog.unnest(c.conkey) WITH ORDINALITY k(num, ord) + JOIN pg_catalog.pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = k.num ORDER BY k.ord), t.relname, - ARRAY(SELECT a.attname::text FROM unnest(c.confkey) WITH ORDINALITY k(num, ord) - JOIN pg_attribute a ON a.attrelid = c.confrelid AND a.attnum = k.num ORDER BY k.ord), - c.confdeltype::text -FROM pg_constraint c JOIN pg_class r ON r.oid = c.conrelid -JOIN pg_namespace n ON n.oid = r.relnamespace -LEFT JOIN pg_class t ON t.oid = c.confrelid -WHERE n.nspname = current_schema() AND r.relname = $1''', - [step.table], + ARRAY(SELECT a.attname::text FROM pg_catalog.unnest(c.confkey) WITH ORDINALITY k(num, ord) + JOIN pg_catalog.pg_attribute a ON a.attrelid = c.confrelid AND a.attnum = k.num ORDER BY k.ord), + c.confdeltype::text, tn.nspname +FROM pg_catalog.pg_constraint c JOIN pg_catalog.pg_class r ON r.oid = c.conrelid +JOIN pg_catalog.pg_namespace n ON n.oid = r.relnamespace +LEFT JOIN pg_catalog.pg_class t ON t.oid = c.confrelid +LEFT JOIN pg_catalog.pg_namespace tn ON tn.oid = t.relnamespace +WHERE n.nspname = coalesce($2::text, pg_catalog.current_schema()) AND r.relname = $1''', + [step.table, step.namespace], ), ); final matches = rows.rows @@ -70,6 +74,8 @@ WHERE n.nspname = current_schema() AND r.relname = $1''', 'columns': row[2], if (row[1] == 'f') ...{ 'target': row[3], + if (step.constraint.containsKey('targetNamespace')) + 'targetNamespace': row[6], 'targetColumns': row[4], 'onDelete': switch (row[5]) { 'a' => 'NO ACTION', @@ -92,7 +98,7 @@ WHERE n.nspname = current_schema() AND r.relname = $1''', } await db.execute( SqlCommand( - 'ALTER TABLE ${quoteIdentifier(step.table)} DROP CONSTRAINT ${quoteIdentifier(matches.single.first as String)}', + 'ALTER TABLE ${quoteQualified(step.table, step.namespace)} DROP CONSTRAINT ${quoteIdentifier(matches.single.first as String)}', ), ); } diff --git a/lib/src/migrate/mysql_schema.dart b/lib/src/migrate/mysql_schema.dart index 86b0e162..dcc8bdd6 100644 --- a/lib/src/migrate/mysql_schema.dart +++ b/lib/src/migrate/mysql_schema.dart @@ -35,6 +35,7 @@ TableSchema mysqlCopy( List? checks, }) => TableSchema( name ?? table.name, + namespace: table.namespace, columns: columns ?? table.columns, primaryKey: primaryKey ?? table.primaryKey, uniqueKeys: uniqueKeys ?? table.uniqueKeys, diff --git a/lib/src/migrate/recovery.dart b/lib/src/migrate/recovery.dart index 30cb0e6a..f845fa7f 100644 --- a/lib/src/migrate/recovery.dart +++ b/lib/src/migrate/recovery.dart @@ -203,18 +203,9 @@ Future _withMigrationLock( Duration timeout, Future Function() action, ) async { - final schemaKey = - (await session.execute(SqlCommand('SELECT hashtext(current_schema())'))) - .rows - .single - .single - as int?; - if (schemaKey == null) { - throw const OrmException( - 'MIGRATION.SCHEMA', - 'A current PostgreSQL schema is required.', - ); - } + // PostgreSQL advisory locks already belong to one database. Serialize its + // migration histories because different histories may touch the same schema. + const schemaKey = 0; final watch = Stopwatch()..start(); var delay = 10; while (true) { @@ -222,9 +213,10 @@ Future _withMigrationLock( try { locked = (await session.execute( - SqlCommand(r'SELECT pg_try_advisory_lock(182983479, $1)', [ - schemaKey, - ]), + SqlCommand( + r'SELECT pg_catalog.pg_try_advisory_lock(182983479, $1)', + [schemaKey], + ), )).rows.single.single as bool; } catch (_) { @@ -253,7 +245,9 @@ Future _withMigrationLock( } finally { try { final unlocked = await session.execute( - SqlCommand(r'SELECT pg_advisory_unlock(182983479, $1)', [schemaKey]), + SqlCommand(r'SELECT pg_catalog.pg_advisory_unlock(182983479, $1)', [ + schemaKey, + ]), ); if (unlocked.rows.single.single != true) { throw const OrmException( @@ -711,9 +705,9 @@ Future hasMigrationTable(SqlDatabase db, String table) async { final rows = await db.execute( SqlCommand( r'''SELECT - EXISTS(SELECT 1 FROM pg_catalog.pg_tables WHERE schemaname = current_schema() AND tablename::text = $1), - coalesce((SELECT n.nspname = current_schema() AND c.relkind IN ('r', 'p') FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace - WHERE c.oid = to_regclass(quote_ident($1))), true)''', + EXISTS(SELECT 1 FROM pg_catalog.pg_tables WHERE schemaname = pg_catalog.current_schema() AND tablename::text = $1), + coalesce((SELECT n.nspname = pg_catalog.current_schema() AND c.relkind IN ('r', 'p') FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace + WHERE c.oid = pg_catalog.to_regclass(pg_catalog.quote_ident($1))), true)''', [table], ), ); diff --git a/lib/src/migrate/schema.dart b/lib/src/migrate/schema.dart index d17f963a..e2242b53 100644 --- a/lib/src/migrate/schema.dart +++ b/lib/src/migrate/schema.dart @@ -21,7 +21,7 @@ import 'mysql_schema.dart' mysqlCreateSchema, mysqlStorageType, validateMysqlSchema; -import 'sql_utils.dart' show quoteIdentifier; +import 'sql_utils.dart' show quoteIdentifier, quoteQualified; import 'sqlite_checks.dart' show sqliteName; /// Creates a new schema. Applications should execute the resulting SQL through @@ -30,6 +30,15 @@ List createSchema(List tables, SqlDialect dialect) { if (isMysqlFamily(dialect)) return mysqlCreateSchema(tables, dialect); final commands = []; validateSchema(tables, dialect); + if (dialect == SqlDialect.postgres) { + final namespaces = tables.map((t) => t.namespace).nonNulls.toSet().toList() + ..sort(); + for (final namespace in namespaces) { + commands.add( + SqlCommand('CREATE SCHEMA IF NOT EXISTS ${quoteIdentifier(namespace)}'), + ); + } + } for (final table in tables) { commands.add(SqlCommand(createTable(table, dialect))); } @@ -39,7 +48,7 @@ List createSchema(List tables, SqlDialect dialect) { for (final key in table.foreignKeys) { commands.add( SqlCommand( - 'ALTER TABLE ${quoteIdentifier(table.name)} ADD ${foreignKey(key)}', + 'ALTER TABLE ${quoteQualified(table.name, table.namespace)} ADD ${foreignKey(key)}', ), ); } @@ -50,7 +59,7 @@ List createSchema(List tables, SqlDialect dialect) { commands.add( SqlCommand( 'CREATE ${index.unique ? 'UNIQUE ' : ''}INDEX ${quoteIdentifier(index.name)} ' - 'ON ${quoteIdentifier(table.name)} (${index.columns.map(quoteIdentifier).join(', ')})', + 'ON ${quoteQualified(table.name, table.namespace)} (${index.columns.map(quoteIdentifier).join(', ')})', ), ); } @@ -76,8 +85,22 @@ void validateSchema(List tables, [SqlDialect? dialect]) { final names = {}; final indexes = {}; + String identity(String name, String? namespace) { + identifier(name); + if (namespace != null) { + identifier(namespace); + if (dialect != null && dialect != SqlDialect.postgres) { + throw const OrmException( + 'SCHEMA.NAMESPACE', + 'Database schemas require PostgreSQL.', + ); + } + } + return namespace == null ? identifier(name) : '$namespace.$name'; + } + for (final table in tables) { - if (!names.add(identifier(table.name))) { + if (!names.add(identity(table.name, table.namespace))) { throw const OrmException('SCHEMA.DUPLICATE', 'Duplicate table name.'); } } @@ -193,7 +216,10 @@ void validateSchema(List tables, [SqlDialect? dialect]) { ); } for (final index in table.indexes) { - final name = identifier(index.name); + final component = identifier(index.name); + final name = table.namespace == null + ? component + : '${table.namespace}.$component'; if (names.contains(name) || !indexes.add(name)) { throw const OrmException( 'SCHEMA.DUPLICATE', @@ -203,7 +229,7 @@ void validateSchema(List tables, [SqlDialect? dialect]) { } for (final key in table.foreignKeys) { foreignKey(key); - identifier(key.target); + identity(key.target, key.targetNamespace); if (key.columns.toSet().length != key.columns.length || key.targetColumns.toSet().length != key.targetColumns.length || key.columns.any( @@ -257,7 +283,7 @@ String createTable(TableSchema table, SqlDialect dialect, {String? name}) { } } - return 'CREATE TABLE ${quoteIdentifier(name ?? table.name)} (${definitions.join(', ')})'; + return 'CREATE TABLE ${quoteQualified(name ?? table.name, table.namespace)} (${definitions.join(', ')})'; } String checkDefinition(CheckSchema check, SqlDialect dialect) => @@ -308,9 +334,9 @@ String columnDefinition(Column c, SqlDialect dialect) { return b.toString(); } -String createIndexSql(String table, IndexSchema index) => +String createIndexSql(String table, IndexSchema index, {String? namespace}) => 'CREATE ${index.unique ? 'UNIQUE ' : ''}INDEX ${quoteIdentifier(index.name)} ' - 'ON ${quoteIdentifier(table)} (${index.columns.map(quoteIdentifier).join(', ')})'; + 'ON ${quoteQualified(table, namespace)} (${index.columns.map(quoteIdentifier).join(', ')})'; String foreignKey(ForeignKey key) { if (key.columns.isEmpty || @@ -327,7 +353,7 @@ String foreignKey(ForeignKey key) { 'Invalid foreign key declaration.', ); } - return 'FOREIGN KEY (${key.columns.map(quoteIdentifier).join(', ')}) REFERENCES ${quoteIdentifier(key.target)} ' + return 'FOREIGN KEY (${key.columns.map(quoteIdentifier).join(', ')}) REFERENCES ${quoteQualified(key.target, key.targetNamespace)} ' '(${key.targetColumns.map(quoteIdentifier).join(', ')}) ON DELETE ${key.onDelete}'; } diff --git a/lib/src/migrate/snapshot.dart b/lib/src/migrate/snapshot.dart index 368ae5a2..9c67db0a 100644 --- a/lib/src/migrate/snapshot.dart +++ b/lib/src/migrate/snapshot.dart @@ -45,6 +45,7 @@ final class SchemaSnapshot { TableSchema targetTable(TableSchema table, SqlDialect dialect) { final target = TableSchema( table.name, + namespace: table.namespace, columns: [ for (final c in table.columns) Column( @@ -112,11 +113,13 @@ Map indexJson(IndexSchema index) => { Map foreignKeyJson(ForeignKey key) => { 'columns': key.columns, 'target': key.target, + if (key.targetNamespace != null) 'targetNamespace': key.targetNamespace, 'targetColumns': key.targetColumns, 'onDelete': key.onDelete, }; Map tableJson(TableSchema table) => { 'name': table.name, + if (table.namespace != null) 'namespace': table.namespace, 'columns': [for (final column in table.columns) columnJson(column)], 'primaryKey': table.primaryKey, 'uniqueKeys': table.uniqueKeys, diff --git a/lib/src/migrate/source.dart b/lib/src/migrate/source.dart index fd1bedc4..7ac58da7 100644 --- a/lib/src/migrate/source.dart +++ b/lib/src/migrate/source.dart @@ -105,12 +105,12 @@ String _snapshotSource(SchemaSnapshot schema) => String _tableSource(TableSchema table) => '''TableSchema( - ${_dartValue(table.name)}, + ${_dartValue(table.name)},${table.namespace == null ? '' : '\n namespace: ${_dartValue(table.namespace)},'} columns: [${table.columns.map(_columnSource).join(', ')}], primaryKey: ${_dartValue(table.primaryKey)}, ${table.uniqueKeys.isEmpty ? '' : 'uniqueKeys: ${_dartValue(table.uniqueKeys)},'} ${table.indexes.isEmpty ? '' : 'indexes: [${table.indexes.map((i) => 'IndexSchema(${_dartValue(i.name)}, ${_dartValue(i.columns)}, unique: ${i.unique})').join(', ')}],'} - ${table.foreignKeys.isEmpty ? '' : 'foreignKeys: [${table.foreignKeys.map((k) => 'ForeignKey(${_dartValue(k.columns)}, ${_dartValue(k.target)}, ${_dartValue(k.targetColumns)}, onDelete: ${_dartValue(k.onDelete)})').join(', ')}],'} + ${table.foreignKeys.isEmpty ? '' : 'foreignKeys: [${table.foreignKeys.map((k) => 'ForeignKey(${_dartValue(k.columns)}, ${_dartValue(k.target)}, ${_dartValue(k.targetColumns)}, onDelete: ${_dartValue(k.onDelete)}${k.targetNamespace == null ? '' : ', targetNamespace: ${_dartValue(k.targetNamespace)}'})').join(', ')}],'} ${table.checks.isEmpty ? '' : 'checks: [${table.checks.map(_checkSource).join(', ')}],'} )'''; @@ -164,9 +164,10 @@ String _stepSource(MigrationStep step) => switch (step) { ExecuteSql() => 'ExecuteSql(${_dartValue(step.sql)})', CheckedTableSql() => 'CheckedTableSql(${_dartValue(step.sql)}, before: ${step.before == null ? 'null' : _tableSource(step.before!)}, after: ${step.after == null ? 'null' : _tableSource(step.after!)})', - DropTable() => 'DropTable(${_dartValue(step.table)})', + DropTable() => + 'DropTable(${_dartValue(step.table)}${step.namespace == null ? '' : ', namespace: ${_dartValue(step.namespace)}'})', DropConstraint() => - 'DropConstraint(${_dartValue(step.table)}, ${_dartValue(step.constraint)})', + 'DropConstraint(${_dartValue(step.table)}, ${_dartValue(step.constraint)}${step.namespace == null ? '' : ', namespace: ${_dartValue(step.namespace)}'})', RebuildTable() => 'RebuildTable(${_tableSource(step.before)}, ${_tableSource(step.after)}, copy: ${_dartValue(step.copy)})', CheckedSql() => diff --git a/lib/src/migrate/sql_utils.dart b/lib/src/migrate/sql_utils.dart index 24346ac7..dd4e7218 100644 --- a/lib/src/migrate/sql_utils.dart +++ b/lib/src/migrate/sql_utils.dart @@ -7,6 +7,10 @@ import 'package:crypto/crypto.dart' show sha256; String quoteIdentifier(String identifier) => '"${identifier.replaceAll('"', '""')}"'; +String quoteQualified(String name, String? namespace) => namespace == null + ? quoteIdentifier(name) + : '${quoteIdentifier(namespace)}.${quoteIdentifier(name)}'; + String postgresLiteral(String value) { var tag = r'$orm$'; while (value.contains(tag)) { diff --git a/lib/src/migrate/step.dart b/lib/src/migrate/step.dart index 75025a51..c75b5744 100644 --- a/lib/src/migrate/step.dart +++ b/lib/src/migrate/step.dart @@ -30,13 +30,20 @@ final class ExecuteSql( /// A reviewed table removal. The SQLite runner checks foreign keys before commit. final class DropTable( /// Physical table name to remove; the runner quotes it as an identifier. - final String table, -) extends MigrationStep { + final String table, { + + /// PostgreSQL schema containing the table; null preserves historical scope. + final String? namespace, +}) extends MigrationStep { /// Records a reviewed table removal without executing it. this; @override - Map toJson() => {'kind': 'dropTable', 'table': table}; + Map toJson() => { + 'kind': 'dropTable', + 'table': table, + if (namespace != null) 'namespace': namespace, + }; } /// SQLite's copy-and-replace operation. Expressions are trusted migration SQL. @@ -83,16 +90,20 @@ final class DropConstraint extends MigrationStep { /// Physical PostgreSQL table whose matching constraint will be removed. final String table; + /// PostgreSQL schema containing the table; null preserves historical scope. + final String? namespace; + /// Immutable constraint signature used to find its actual catalog name. final Map constraint; /// Freezes a reviewed signature; execution requires exactly one catalog match. - DropConstraint(this.table, Map constraint) + DropConstraint(this.table, Map constraint, {this.namespace}) : constraint = freezeMigrationValue(constraint) as Map; @override Map toJson() => { 'kind': 'dropConstraint', 'table': table, + if (namespace != null) 'namespace': namespace, 'constraint': constraint, }; } @@ -114,37 +125,45 @@ final class CheckedSql extends MigrationStep { /// A concurrent, ascending B-tree index with default collation/opclasses. /// The completion check compares its definition as well as ready/valid state. - factory CheckedSql.createIndex(String table, IndexSchema index) { + factory CheckedSql.createIndex( + String table, + IndexSchema index, { + String? namespace, + }) { if (index.columns.isEmpty) throw ArgumentError('An index needs columns.'); final source = postgresLiteral(table), name = postgresLiteral(index.name); + final scope = namespace == null + ? 'pg_catalog.current_schema()' + : postgresLiteral(namespace); final columns = 'ARRAY[${index.columns.map(postgresLiteral).join(', ')}]::text[]'; return CheckedSql( createIndexSql( table, index, + namespace: namespace, ).replaceFirst('INDEX ', 'INDEX CONCURRENTLY '), readyWhen: '''SELECT NOT EXISTS ( -SELECT 1 FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace -WHERE n.nspname = current_schema() AND c.relname = $name)''', +SELECT 1 FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace +WHERE n.nspname = $scope AND c.relname = $name)''', doneWhen: '''SELECT EXISTS ( -SELECT 1 FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid -JOIN pg_namespace n ON n.oid = c.relnamespace JOIN pg_class t ON t.oid = i.indrelid -JOIN pg_am am ON am.oid = c.relam -WHERE n.nspname = current_schema() AND c.relname = $name AND t.relname = $source +SELECT 1 FROM pg_catalog.pg_index i JOIN pg_catalog.pg_class c ON c.oid = i.indexrelid +JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace JOIN pg_catalog.pg_class t ON t.oid = i.indrelid +JOIN pg_catalog.pg_am am ON am.oid = c.relam +WHERE n.nspname = $scope AND c.relname = $name AND t.relname = $source AND t.relnamespace = n.oid AND i.indisvalid AND i.indisready AND i.indislive AND i.indisunique = ${index.unique} AND NOT i.indisprimary AND NOT i.indisexclusion AND i.indexprs IS NULL AND i.indpred IS NULL AND i.indnatts = i.indnkeyatts AND am.amname = 'btree' AND c.reloptions IS NULL -AND NOT EXISTS (SELECT 1 FROM pg_constraint constraint_row WHERE constraint_row.conindid = i.indexrelid AND constraint_row.contype IN ('p', 'u', 'x')) -AND NOT coalesce((to_jsonb(i)->>'indnullsnotdistinct')::boolean, false) -AND NOT EXISTS (SELECT 1 FROM unnest(i.indoption) v WHERE v <> 0) -AND NOT EXISTS (SELECT 1 FROM unnest(i.indclass) v JOIN pg_opclass o ON o.oid = v WHERE NOT o.opcdefault) -AND NOT EXISTS (SELECT 1 FROM unnest(i.indkey, i.indcollation) k(num, collation_oid) - JOIN pg_attribute a ON a.attrelid = t.oid AND a.attnum = k.num WHERE k.collation_oid <> a.attcollation) -AND ARRAY(SELECT a.attname::text FROM unnest(i.indkey) WITH ORDINALITY k(num, ord) - JOIN pg_attribute a ON a.attrelid = t.oid AND a.attnum = k.num ORDER BY k.ord) = $columns)''', +AND NOT EXISTS (SELECT 1 FROM pg_catalog.pg_constraint constraint_row WHERE constraint_row.conindid = i.indexrelid AND constraint_row.contype IN ('p', 'u', 'x')) +AND NOT coalesce((pg_catalog.to_jsonb(i)->>'indnullsnotdistinct')::boolean, false) +AND NOT EXISTS (SELECT 1 FROM pg_catalog.unnest(i.indoption) v WHERE v <> 0) +AND NOT EXISTS (SELECT 1 FROM pg_catalog.unnest(i.indclass) v JOIN pg_catalog.pg_opclass o ON o.oid = v WHERE NOT o.opcdefault) +AND NOT EXISTS (SELECT 1 FROM ROWS FROM (pg_catalog.unnest(i.indkey), pg_catalog.unnest(i.indcollation)) k(num, collation_oid) + JOIN pg_catalog.pg_attribute a ON a.attrelid = t.oid AND a.attnum = k.num WHERE k.collation_oid <> a.attcollation) +AND ARRAY(SELECT a.attname::text FROM pg_catalog.unnest(i.indkey) WITH ORDINALITY k(num, ord) + JOIN pg_catalog.pg_attribute a ON a.attrelid = t.oid AND a.attnum = k.num ORDER BY k.ord) = $columns)''', ); } @override diff --git a/lib/src/migrate/validation.dart b/lib/src/migrate/validation.dart index 1b4523a2..4e0c2fbf 100644 --- a/lib/src/migrate/validation.dart +++ b/lib/src/migrate/validation.dart @@ -103,6 +103,12 @@ void validateMigrations( continue; } if (step is DropTable) { + if (step.namespace != null && dialect != SqlDialect.postgres) { + throw const OrmException( + 'MIGRATION.TARGET', + 'Database schemas require PostgreSQL.', + ); + } if (isMysqlFamily(dialect)) { throw const OrmException( 'MIGRATION.RECOVERY', diff --git a/lib/src/orm/changes.dart b/lib/src/orm/changes.dart index 7f13958e..de239181 100644 --- a/lib/src/orm/changes.dart +++ b/lib/src/orm/changes.dart @@ -47,23 +47,23 @@ final class ChangeHub { } void registerTable(TableSchema table) { - if (closed || identical(_tables[table.name], table)) return; - final previous = _tables[table.name]; + if (closed || identical(_tables[table.identity], table)) return; + final previous = _tables[table.identity]; if (previous != null) { for (final fk in previous.foreignKeys) { - final effects = _deleteEffects[fk.target]; - effects?.remove(table.name); - if (effects?.isEmpty ?? false) _deleteEffects.remove(fk.target); + final effects = _deleteEffects[fk.targetIdentity]; + effects?.remove(table.identity); + if (effects?.isEmpty ?? false) _deleteEffects.remove(fk.targetIdentity); } } - _tables[table.name] = table; + _tables[table.identity] = table; for (final fk in table.foreignKeys) { if (!{'CASCADE', 'SET NULL', 'SET DEFAULT'}.contains(fk.onDelete)) { continue; } - final effects = _deleteEffects.putIfAbsent(fk.target, () => {}); - effects[table.name] = - (effects[table.name] ?? false) || fk.onDelete == 'CASCADE'; + final effects = _deleteEffects.putIfAbsent(fk.targetIdentity, () => {}); + effects[table.identity] = + (effects[table.identity] ?? false) || fk.onDelete == 'CASCADE'; } } diff --git a/lib/src/orm/database.dart b/lib/src/orm/database.dart index 175c041e..61aad815 100644 --- a/lib/src/orm/database.dart +++ b/lib/src/orm/database.dart @@ -101,7 +101,7 @@ class Database extends QueryContext { final changes = {}; for (final table in tables) { _changes.registerTable(table); - changes[table.name] = cascade; + changes[table.identity] = cascade; } if (!inTransaction) { _changes.publish(changes); diff --git a/lib/src/orm/watch.dart b/lib/src/orm/watch.dart index c0d7fe37..eefa5012 100644 --- a/lib/src/orm/watch.dart +++ b/lib/src/orm/watch.dart @@ -72,7 +72,7 @@ final class _QueryWatch implements ChangeSubscription { ); } for (final table in [...reads.tables, ...extraReads]) { - tables.add(table.name); + tables.add(table.identity); changesFor(db.driver).registerTable(table); } // Subscribe before the initial read so concurrent commits cannot be lost. diff --git a/lib/src/postgres/options.dart b/lib/src/postgres/options.dart index 27d156f2..3ad3a22f 100644 --- a/lib/src/postgres/options.dart +++ b/lib/src/postgres/options.dart @@ -36,7 +36,9 @@ final class PostgresOptions { /// Label exposed by PostgreSQL in connection and activity diagnostics. final String applicationName; - /// Optional, quoted schema used as the connection search path. + /// Optional, quoted search path for raw SQL and historical unqualified metadata. + /// PostgreSQL-generated models retain their declared namespace (default `public`) + /// independently of this setting. final String? schema; /// Creates pool configuration; opening the driver validates its values. diff --git a/lib/src/query/cursor.dart b/lib/src/query/cursor.dart index a146951d..05cc08b4 100644 --- a/lib/src/query/cursor.dart +++ b/lib/src/query/cursor.dart @@ -97,7 +97,7 @@ extension KeysetQuery on Query { _validateCursor(terms.map((t) => t.order).toList()); final payload = { 'version': 1, - 'table': queryState.source.schema.name, + 'table': queryState.source.schema.identity, 'order': _cursorShape(terms.map((t) => t.order).toList()), 'values': [ for (final term in terms) _encodeCursorValue(term.assignedValue), @@ -122,7 +122,7 @@ extension KeysetQuery on Query { utf8.decode(base64Url.decode(token)), ) as Map; if (payload['version'] != 1 || - payload['table'] != queryState.source.schema.name || + payload['table'] != queryState.source.schema.identity || jsonEncode(payload['order']) != jsonEncode(_cursorShape(order))) { throw const FormatException( 'Cursor belongs to another schema or sort order.', diff --git a/lib/src/query/mutation.dart b/lib/src/query/mutation.dart index 80dfcca3..498864a6 100644 --- a/lib/src/query/mutation.dart +++ b/lib/src/query/mutation.dart @@ -318,8 +318,8 @@ final class Mutation { } final table = w.mysql && mutationKind == MutationKind.insert - ? w.quote(queryState.source.schema.name) - : '${w.quote(queryState.source.schema.name)} AS ${w.quote('t0')}'; + ? w.table(queryState.source.schema) + : '${w.table(queryState.source.schema)} AS ${w.quote('t0')}'; if (w.mysql && mutationKind == MutationKind.insert) { w.unqualifiedTable = queryState.source; } diff --git a/lib/src/query/nodes.dart b/lib/src/query/nodes.dart index 741a5bcf..3cdc11d6 100644 --- a/lib/src/query/nodes.dart +++ b/lib/src/query/nodes.dart @@ -1,6 +1,7 @@ import 'package:meta/meta.dart'; import '../../driver.dart'; +import '../../schema_model.dart' show TableSchema; import 'context.dart'; import 'expression.dart'; import 'joins.dart'; @@ -200,6 +201,18 @@ final class SqlWriter { : '"${name.replaceAll('"', '""')}"'; } + String table(TableSchema table) { + final namespace = table.namespace; + if (namespace == null) return quote(table.name); + if (dialect != SqlDialect.postgres) { + throw const OrmException( + 'SCHEMA.NAMESPACE', + 'Database schemas require PostgreSQL.', + ); + } + return '${quote(namespace)}.${quote(table.name)}'; + } + String parameter(Object? value, {String? storageType}) { // Codec storage, not string pattern guessing, determines temporal binding. if (mysql && value is String && storageType == 'instant') { @@ -291,7 +304,7 @@ final class RelationSubqueryNode( p.expressionNode.write(w), ]; final query = - 'SELECT ${count ? 'COUNT(*)' : '1'} FROM ${w.quote(source.schema.name)} AS ${w.quote(alias)} WHERE ${predicates.join(' AND ')}'; + 'SELECT ${count ? 'COUNT(*)' : '1'} FROM ${w.table(source.schema)} AS ${w.quote(alias)} WHERE ${predicates.join(' AND ')}'; return count ? '($query)' : 'EXISTS ($query)'; } finally { if (previous == null) { diff --git a/lib/src/query/plan.dart b/lib/src/query/plan.dart index 8a36f35b..22372301 100644 --- a/lib/src/query/plan.dart +++ b/lib/src/query/plan.dart @@ -65,7 +65,7 @@ final class QueryPlan { joins = List.unmodifiable(joins), loads = List.unmodifiable(loads), reads = List.unmodifiable( - reads.tables.map((t) => t.name).toSet().toList()..sort(), + reads.tables.map((t) => t.identity).toSet().toList()..sort(), ), opaqueReads = reads.opaque; @@ -192,9 +192,9 @@ QueryPlan _describeStatement( for (final (i, expression) in plan.columns.indexed) { final node = unwrapStorage(expression.expressionNode); final (table, column) = switch (node) { - ColumnNode() => (node.table.schema.name, node.name), + ColumnNode() => (node.table.schema.identity, node.name), PresenceNode() => ( - node.alias.fields.table.schema.name, + node.alias.fields.table.schema.identity, node.alias.presenceMarker, ), _ => (null, null), @@ -214,13 +214,13 @@ QueryPlan _describeStatement( [ for (final join in state.joins) ( - table: join.alias.fields.table.schema.name, + table: join.alias.fields.table.schema.identity, left: join.left, relation: false, ), for (final join in plan.joins) ( - table: join.alias.fields.table.schema.name, + table: join.alias.fields.table.schema.identity, left: join.left, relation: true, ), diff --git a/lib/src/query/query.dart b/lib/src/query/query.dart index 5a4183db..f103938f 100644 --- a/lib/src/query/query.dart +++ b/lib/src/query/query.dart @@ -266,7 +266,9 @@ class Query { final joins = [...queryState.joins, ...plan.joins]; if (queryState.union == null && !queryState.ctes.any( - (cte) => cte.name == queryState.source.schema.name, + (cte) => + queryState.source.schema.namespace == null && + cte.name == queryState.source.schema.name, )) { w.reads?.tables.add(queryState.source.schema); } @@ -388,7 +390,7 @@ class Query { stage == null ? columns.join(', ') : stage.inputs.join(', '), ); buffer.write( - ' FROM ${queryState.union == null ? w.quote(queryState.source.schema.name) : '(${queryState.union!.write(w)})'} AS ${w.quote(rootAlias)}', + ' FROM ${queryState.union == null ? w.table(queryState.source.schema) : '(${queryState.union!.write(w)})'} AS ${w.quote(rootAlias)}', ); final visible = {...saved, queryState.source: rootAlias}; for (final join in joins) { @@ -403,7 +405,7 @@ class Query { temporal: w.temporal, )..leftJoins.addAll(w.leftJoins.where(visible.containsKey)); join.on.expressionNode.write(check); - final table = w.quote(ref.schema.name); + final table = w.table(ref.schema); final source = join.left ? '(SELECT *, 1 AS ${w.quote(join.alias.presenceMarker)} FROM $table)' : table; diff --git a/lib/src/schema/model.dart b/lib/src/schema/model.dart index 1ac74c33..23f75c65 100644 --- a/lib/src/schema/model.dart +++ b/lib/src/schema/model.dart @@ -134,6 +134,14 @@ final class ForeignKey { /// Physical name of the referenced table. final String target; + /// PostgreSQL namespace of the target, independent of its table name. + /// Null retains an unqualified historical target. + final String? targetNamespace; + + /// Stable physical identity for matching references, never parsed as SQL. + String get targetIdentity => + targetNamespace == null ? target : '$targetNamespace.$target'; + /// Target key columns in the corresponding source-column order. final List targetColumns; @@ -146,6 +154,7 @@ final class ForeignKey { this.target, this.targetColumns, { this.onDelete = 'RESTRICT', + this.targetNamespace, }); } @@ -219,7 +228,9 @@ final class IndexSchema { /// Immutable physical table metadata, independent of Dart model identity. /// /// The constructor copies collection inputs, including nested key and index -/// column lists. Schema consumers validate engine support before executing DDL. +/// column lists, and rejects empty table/schema names and names containing dots +/// or NUL, including foreign-key targets. Schema consumers validate engine +/// support before executing DDL. /// /// ```dart /// final accounts = TableSchema( @@ -232,9 +243,17 @@ final class IndexSchema { /// /// {@category Schema} final class TableSchema { - /// Physical table name. + /// Non-empty physical table name, without a schema prefix, dots or NUL. final String name; + /// PostgreSQL namespace. Generated PostgreSQL models always specify this, + /// including `public`; other engines reject explicit namespaces. + /// Null also preserves the meaning and fingerprints of historical metadata. + final String? namespace; + + /// Stable physical identity for maps and diagnostics, never parsed as SQL. + String get identity => namespace == null ? name : '$namespace.$name'; + /// Ordered columns used to render the physical schema. final List> columns; @@ -257,8 +276,13 @@ final class TableSchema { final List> clientDefaults; /// Copies schema collections without opening or altering a database. + /// + /// Throws `SCHEMA.IDENTIFIER` for empty table/schema names or names containing + /// dots or NUL, including foreign-key targets, before the metadata can be + /// bound to queries or change tracking. TableSchema( this.name, { + this.namespace, required List> columns, List primaryKey = const [], List> uniqueKeys = const [], @@ -280,6 +304,7 @@ final class TableSchema { key.target, List.unmodifiable(key.targetColumns), onDelete: key.onDelete, + targetNamespace: key.targetNamespace, ), ]), checks = List.unmodifiable(checks), @@ -290,5 +315,21 @@ final class TableSchema { List.unmodifiable(index.columns), unique: index.unique, ), - ]); + ]) { + void checkIdentifier(String name) { + if (name.isEmpty || name.contains('.') || name.contains('\u0000')) { + throw const OrmException( + 'SCHEMA.IDENTIFIER', + 'Table and schema names must be non-empty identifiers, without dots or NUL.', + ); + } + } + + checkIdentifier(name); + if (namespace != null) checkIdentifier(namespace!); + for (final key in this.foreignKeys) { + checkIdentifier(key.target); + if (key.targetNamespace != null) checkIdentifier(key.targetNamespace!); + } + } } diff --git a/lib/src/sqlite/web_build.dart b/lib/src/sqlite/web_build.dart index 6fbec54d..a841ae35 100644 --- a/lib/src/sqlite/web_build.dart +++ b/lib/src/sqlite/web_build.dart @@ -1,11 +1,11 @@ // Generated by tool/build_sqlite_web.dart. Do not edit. const sqliteWebProtocol = 1; const sqliteWorkerBuild = - '1ffcc1c93a300242a8c4ca7badc270240d0550d23254799eda18232f8bd3312d'; + '1a12f6159d3f3b9a21811c2431bd7cd0d7986d0d8530f6085246adfbff6d049e'; const sqliteWorkerFile = - 'worker-4360a5ed33bec6997af5514e04d393eb8fae00e331f2532a696ce34b0b9e44a1.js'; + 'worker-a49ab41eb8d74a1dba9d1dd8062e938daa4ab55ec26178f47a83c129806012f4.js'; const sqliteWorkerSha256 = - '4360a5ed33bec6997af5514e04d393eb8fae00e331f2532a696ce34b0b9e44a1'; + 'a49ab41eb8d74a1dba9d1dd8062e938daa4ab55ec26178f47a83c129806012f4'; const sqliteWasmFile = 'sqlite3-13d3f11d05b39ba0618a7115fb41640a5d48b6300f5d3f325f554b42bd6688a4.wasm'; const sqliteWasmSha256 = diff --git a/pubspec.lock b/pubspec.lock index b654d5b0..c5b8d39f 100644 --- a/pubspec.lock +++ b/pubspec.lock @@ -218,7 +218,7 @@ packages: source: hosted version: "4.0.0" glob: - dependency: transitive + dependency: "direct main" description: name: glob sha256: "218aeb56050c714f62a3182775320dfa04602b55074873e24e31bbd39bda96fb" diff --git a/pubspec.yaml b/pubspec.yaml index 48dbe327..4ea43028 100644 --- a/pubspec.yaml +++ b/pubspec.yaml @@ -17,6 +17,7 @@ dependencies: build: ^4.0.11 crypto: ^3.0.7 dart_style: ^3.1.13 + glob: ^2.2.0 meta: ^1.19.0 mysql_client_plus: ^0.1.3 path: ^1.9.1 diff --git a/test/migration_recovery_test.dart b/test/migration_recovery_test.dart index bda004de..acceeae5 100644 --- a/test/migration_recovery_test.dart +++ b/test/migration_recovery_test.dart @@ -296,9 +296,7 @@ void main() { try { await blocker.session((held) async { await held.execute( - SqlCommand( - 'SELECT pg_advisory_lock(182983479, hashtext(current_schema()))', - ), + SqlCommand('SELECT pg_advisory_lock(182983479, 0)'), ); try { final migrator = Migrator( @@ -326,9 +324,7 @@ void main() { ); } finally { await held.execute( - SqlCommand( - 'SELECT pg_advisory_unlock(182983479, hashtext(current_schema()))', - ), + SqlCommand('SELECT pg_advisory_unlock(182983479, 0)'), ); } }); diff --git a/test/namespace_generation_test.dart b/test/namespace_generation_test.dart new file mode 100644 index 00000000..6886feed --- /dev/null +++ b/test/namespace_generation_test.dart @@ -0,0 +1,101 @@ +import 'dart:io'; + +import 'package:orm/postgres.dart'; +import 'package:test/test.dart'; + +import '../tool/src/build_fixture.dart'; + +void main() { + final url = Platform.environment['ORM_TEST_POSTGRES']; + test( + 'generated namespace clients execute cross-schema relations on PostgreSQL', + () async { + final fixture = await BuildFixture.create( + ormPath: Directory.current.path, + ); + final admin = postgres( + PostgresOptions(url: Uri.parse(url!), tls: .disable), + ); + final name = + 'orm_namespace_${pid}_${DateTime.now().microsecondsSinceEpoch}'; + var created = false; + try { + await admin.execute(SqlCommand('CREATE DATABASE "$name"')); + created = true; + await fixture.file('lib/schema.dart').delete(); + await fixture.write('lib/schema/auth/users.dart', ''' +import 'package:orm/schema.dart'; +final user = model('Users', (id: identity(), name: text(name: 'DisplayName'))); +'''); + await fixture.write('lib/schema/public/users.dart', ''' +import 'package:orm/schema.dart'; +import '../auth/users.dart' as auth; +final user = model('Users', (id: identity(), name: text(), accountId: integer()), + relations: (u) => (account: references(u.accountId, () => auth.user),)); +'''); + await fixture.run([ + 'run', + 'orm', + 'generate', + 'lib/schema', + '--database', + 'postgres', + ]); + await fixture.write('bin/namespaces.dart', r''' +import 'dart:io'; +import 'package:orm/postgres.dart'; +import 'package:orm/migrate.dart'; +import '../lib/schema.orm.dart'; + +Future main() async { + final db = postgres(PostgresOptions(url: Uri.parse(Platform.environment['ORM_NAMESPACE_TEST_URL']!), tls: .disable)); + try { + final initial = Migration.create('0001_initial', appSchema, dialect: .postgres); + await Migrator(db.sql).apply([initial]); + final AuthUser account = await db.auth.user.create(name: 'Alice'); + final PublicUser profile = await db.public.user.create(name: 'Profile', accountId: account.id); + final owner = await db.public.user.byId(profile.id).select((u) => u.account.select((a) => a.name).required()).single(); + if (owner != 'Alice') throw StateError('Wrong relationship target: $owner'); + await db.session((session) async { + await session.execute(SqlCommand('SET search_path TO auth')); + await session.execute(SqlCommand('CREATE TEMP TABLE "Users" (id bigint, name text)')); + final rows = await session.public.user.get(); + if (rows.single.name != 'Profile') throw StateError('Session state changed table target.'); + final verification = await verifySchema(session.sql, initial.snapshot!); + if (!verification.matches) throw StateError(verification.differences.join('\n')); + }); + if ((await db.auth.user.get()).single.name != 'Alice') throw StateError('Wrong namespace.'); + print('namespace-client-ok'); + } finally { await db.close(); } +} +'''); + final result = await Process.run( + Platform.resolvedExecutable, + ['run', 'bin/namespaces.dart'], + workingDirectory: fixture.directory.path, + environment: { + 'ORM_NAMESPACE_TEST_URL': Uri.parse(url) + .replace(path: '/$name') + .toString(), + }, + ); + expect( + result.exitCode, + 0, + reason: '${result.stdout}\n${result.stderr}', + ); + expect(result.stdout, contains('namespace-client-ok')); + } finally { + if (created) { + await admin.execute(SqlCommand('DROP DATABASE "$name" WITH (FORCE)')); + } + await admin.close(); + await fixture.dispose(); + } + }, + skip: url == null + ? 'Set ORM_TEST_POSTGRES to a disposable PostgreSQL database.' + : false, + timeout: const Timeout(Duration(minutes: 3)), + ); +} diff --git a/test/namespace_test.dart b/test/namespace_test.dart new file mode 100644 index 00000000..48b3fac4 --- /dev/null +++ b/test/namespace_test.dart @@ -0,0 +1,482 @@ +import 'dart:async'; +import 'dart:io'; + +import 'package:orm/migrate.dart'; +import 'package:orm/postgres.dart'; +import 'package:orm/sqlite.dart'; +import 'package:test/test.dart'; + +final _id = Column('Id', Codecs.integer, generated: true); +final _name = Column('DisplayName', Codecs.text); + +final class _UserFields extends Fields { + _UserFields(super.table); + late final id = column(_id); + late final name = column(_name); +} + +Table<({int id, String name}), _UserFields> _users(String namespace) => Table( + _userSchema(namespace), + _UserFields.new, + (u) => (u.id, u.name).map((id, name) => (id: id, name: name)), +); + +TableSchema _userSchema( + String namespace, { + String name = 'Users', + bool check = false, +}) => TableSchema( + name, + namespace: namespace, + columns: [_id, _name], + primaryKey: ['Id'], + indexes: [ + IndexSchema('Names.Index', ['DisplayName']), + ], + checks: check ? [CheckSchema('NameLength', 'length("DisplayName") > 0')] : [], +); + +TableSchema _messages( + String namespace, + String target, { + String action = 'RESTRICT', +}) => TableSchema( + 'Messages', + namespace: namespace, + columns: [Column('Id', Codecs.integer), Column('AuthorId', Codecs.integer)], + primaryKey: ['Id'], + foreignKeys: [ + ForeignKey( + ['AuthorId'], + 'Users', + ['Id'], + targetNamespace: target, + onDelete: action, + ), + ], +); + +void main() { + test( + 'manual tables reject invalid physical identity components before binding', + () { + final sql = SqlBuilder(.postgres); + Table<({int id, String name}), _UserFields> definition( + String name, { + String? namespace, + }) => Table( + TableSchema(name, namespace: namespace, columns: [_id, _name]), + _UserFields.new, + (u) => (u.id, u.name).map((id, name) => (id: id, name: name)), + ); + final invalid = throwsA( + isA().having((e) => e.code, 'code', 'SCHEMA.IDENTIFIER'), + ); + for (final name in ['', 'auth.Users', 'bad\u0000name']) { + expect(() => sql.table(definition(name)).compile(), invalid); + expect( + () => sql.table(definition('Users', namespace: name)).compile(), + invalid, + ); + for (final key in [ + ForeignKey(['Id'], name, ['Id']), + ForeignKey(['Id'], 'Users', ['Id'], targetNamespace: name), + ]) { + expect( + () => TableSchema('Reports', columns: [_id], foreignKeys: [key]), + invalid, + ); + } + } + expect( + sql.table(definition('Users', namespace: 'auth')).compile().sql, + contains('"auth"."Users"'), + ); + }, + ); + + test( + 'explicit namespaces reject unsupported query and migration engines', + () async { + final db = await sqlite(const SqliteOptions.memory()); + addTearDown(db.close); + expect( + () => db.table(_users('auth')).compile(), + throwsA(isA()), + ); + for (final dialect in [ + SqlDialect.sqlite, + SqlDialect.mysql, + SqlDialect.mariadb, + ]) { + expect( + () => SchemaSnapshot([_userSchema('auth')]).forDialect(dialect), + throwsA(isA()), + ); + } + }, + ); + + test('namespace metadata survives source emission without changing old fingerprints', () { + final plain = TableSchema('Users', columns: [_id], primaryKey: ['Id']); + expect(SchemaSnapshot([plain]).toJson()['tables'], [ + { + 'name': 'Users', + 'columns': [ + { + 'name': 'Id', + 'type': 'integer', + 'nullable': false, + 'generated': true, + }, + ], + 'primaryKey': ['Id'], + 'uniqueKeys': [], + 'indexes': [], + 'foreignKeys': [], + }, + ]); + final migration = Migration.steps('0001_drop', [ + DropTable('Users', namespace: 'auth'), + DropConstraint('Users', { + 'kind': 'p', + 'columns': ['Id'], + }, namespace: 'auth'), + ], dialect: .postgres); + expect(migrationSource(migration), contains("namespace: \"auth\"")); + expect(migration.steps.first.toJson()['namespace'], 'auth'); + }); + + final url = Platform.environment['ORM_TEST_POSTGRES']; + group( + 'PostgreSQL namespaces', + () { + late Database db; + late String first, second, history; + var serial = 0; + setUp(() async { + final prefix = 'orm_ns_${pid}_${serial++}'; + first = '${prefix}_A'; + second = '${prefix}_B'; + history = '${prefix}_history'; + db = postgres( + PostgresOptions(url: Uri.parse(url!), tls: .disable, schema: history), + ); + await db.execute(SqlCommand('CREATE SCHEMA "$history"')); + }); + tearDown(() async { + for (final namespace in [first, second, history]) { + await db.execute( + SqlCommand('DROP SCHEMA IF EXISTS "$namespace" CASCADE'), + ); + } + await db.close(); + }); + + test( + 'unqualified snapshots require explicit destructive replacement', + () async { + final old = TableSchema( + 'Users', + columns: [_id, _name], + primaryKey: ['Id'], + ); + final initial = Migration.create('0001_initial', [ + old, + ], dialect: .postgres); + await Migrator(db.sql).apply([initial]); + await db.execute( + SqlCommand( + 'INSERT INTO "Users" ("DisplayName") VALUES (\'Old row\')', + ), + ); + final target = SchemaSnapshot([_userSchema(first)]); + expect( + () => Migration.diff( + '0002_replace', + dialect: .postgres, + from: initial.snapshot!, + to: target, + ), + throwsA(isA()), + ); + final replace = Migration.diff( + '0002_replace', + dialect: .postgres, + from: initial.snapshot!, + to: target, + previous: initial.checksum, + allowDestructive: true, + ); + await Migrator(db.sql).apply([initial, replace]); + expect((await verifySchema(db.sql, target)).matches, true); + expect(await db.table(_users(first)).get(), isEmpty); + expect( + (await inspectTable(db.sql, 'Users', namespace: history)).columns, + isEmpty, + ); + }, + ); + + test('catalog probes ignore shadow system tables and helper functions', () async { + final a = _users(first), b = _users(second); + final initial = Migration.create('0001_initial', [ + a.schema, + b.schema, + ], dialect: .postgres); + await Migrator(db.sql).apply([initial]); + await db.table(a).createRow((u) => [u.name.set('Alice')]); + await db.execute( + SqlCommand( + 'CREATE TABLE "$second".pg_class AS SELECT * FROM pg_catalog.pg_class WHERE false', + ), + ); + await db.execute( + SqlCommand( + 'CREATE FUNCTION "$second".row_security_active(oid) RETURNS boolean LANGUAGE plpgsql AS \$\$ BEGIN RAISE EXCEPTION \'shadow helper called\'; END \$\$', + ), + ); + await db.session((session) async { + await session.execute( + SqlCommand('SET search_path TO "$history", "$second", pg_catalog'), + ); + expect( + (await verifySchema(session.sql, initial.snapshot!)).differences, + isEmpty, + ); + final index = CheckedSql.createIndex( + 'Users', + a.schema.indexes.single, + namespace: first, + ); + expect( + (await session.execute(SqlCommand(index.readyWhen))) + .rows + .single + .single, + false, + ); + expect( + (await session.execute(SqlCommand(index.doneWhen))) + .rows + .single + .single, + true, + ); + final fill = Migration.steps( + '0002_fill', + [ + Backfill( + a.schema, + set: {'DisplayName': "'Updated'"}, + where: '"DisplayName" <> \'Updated\'', + doneWhen: + 'SELECT NOT EXISTS (SELECT 1 FROM "$first"."Users" WHERE "DisplayName" <> \'Updated\')', + ), + ], + dialect: .postgres, + snapshot: initial.snapshot, + previous: initial.checksum, + ); + await Migrator(session.sql).apply([initial, fill]); + expect((await session.table(a).get()).single.name, 'Updated'); + expect( + await verifyColumns(session.sql, [ + TableSchema( + 'Users', + namespace: first, + columns: [_id, Column('Missing', Codecs.text)], + primaryKey: ['Id'], + ), + ]), + [ + '$first.Users.Missing is missing', + '$first.Users.DisplayName is unmanaged', + ], + ); + }); + }); + + test('same-named tables, SQL scope, cursors, joins and watch remain distinct', () async { + final a = _users(first), b = _users(second); + await Migrator(db.sql).apply([ + Migration.create('0001_initial', [ + a.schema, + b.schema, + ], dialect: .postgres), + ]); + await db.table(a).createRow((u) => [u.name.set('Alice')]); + await db.table(b).createRow((u) => [u.name.set('Bob')]); + final token = db.table(a).cursorToken((u) => [u.id.cursor(1)]); + expect( + () => db.table(b).seekToken(token, orderBy: (u) => [u.id.asc()]), + throwsA(isA()), + ); + final plan = db.table(a).inspect(); + expect(plan.reads, [a.schema.identity]); + expect(plan.sql, contains('"$first"."Users"')); + await db.session((session) async { + await session.execute(SqlCommand('SET search_path TO "$second"')); + await session.execute( + SqlCommand( + 'CREATE TEMPORARY TABLE "Users" ("Id" bigint, "DisplayName" text)', + ), + ); + expect((await session.table(a).get()).single.name, 'Alice'); + expect((await session.table(b).get()).single.name, 'Bob'); + final cte = session.table(b).select((u) => u.id).asCte('Users'); + expect(await cte.query.get(), [1]); + final alias = b.alias(); + final joined = await session + .table(a) + .join(alias, on: (left, right) => left.id.equals(right.id)) + .select( + (left) => (left.name, alias.fields.name).map((a, b) => (a, b)), + ) + .get(); + expect(joined, [('Alice', 'Bob')]); + }); + final emissions = >[]; + final initial = Completer(), changed = Completer(); + final subscription = db.table(a).watch().listen((rows) { + emissions.add(rows); + if (!initial.isCompleted) initial.complete(); + if (rows.single.name == 'Updated' && !changed.isCompleted) { + changed.complete(); + } + }); + try { + await initial.future; + await db + .table(b) + .where((u) => u.id.eq(1)) + .update((u) => [u.name.set('Other')]) + .execute(); + await db + .table(a) + .where((u) => u.id.eq(1)) + .update((u) => [u.name.set('Updated')]) + .execute(); + await changed.future.timeout(const Duration(seconds: 5)); + expect(emissions.map((rows) => rows.single.name), [ + 'Alice', + 'Updated', + ]); + } finally { + await subscription.cancel(); + } + }); + + test( + 'cross-schema foreign keys, checks and qualified diff verification', + () async { + final start = Migration.create('0001_initial', [ + _userSchema(first), + _userSchema(second), + _messages(second, first), + ], dialect: .postgres); + await Migrator(db.sql).apply([start]); + expect( + (await verifySchema(db.sql, start.snapshot!)).differences, + isEmpty, + ); + await db.execute( + SqlCommand( + 'INSERT INTO "$first"."Users" ("DisplayName") VALUES (\'Alice\')', + ), + ); + await db.execute( + SqlCommand('INSERT INTO "$second"."Messages" VALUES (1, 1)'), + ); + final next = SchemaSnapshot([ + _userSchema(first, check: true), + _userSchema(second), + _messages(second, first, action: 'CASCADE'), + ]); + final change = Migration.diff( + '0002_constraints', + dialect: .postgres, + from: start.snapshot!, + to: next, + previous: start.checksum, + ); + await Migrator(db.sql).apply([start, change]); + expect((await verifySchema(db.sql, next)).differences, isEmpty); + await db.table(_users(first)).delete().execute(); + expect( + (await db.execute( + SqlCommand('SELECT count(*) FROM "$second"."Messages"'), + )).rows.single.single, + 0, + ); + final dropCheck = Migration.diff( + '0003_check', + dialect: .postgres, + from: next, + to: SchemaSnapshot([ + _userSchema(first), + _userSchema(second), + _messages(second, first, action: 'CASCADE'), + ]), + previous: change.checksum, + ); + await Migrator(db.sql).apply([start, change, dropCheck]); + expect( + (await verifySchema(db.sql, dropCheck.snapshot!)).differences, + isEmpty, + ); + }, + ); + + test('explicit schema moves preserve rows and never infer a destructive rename', () async { + final start = Migration.create('0001_initial', [ + _userSchema(first), + ], dialect: .postgres); + await Migrator(db.sql).apply([start]); + await db.table(_users(first)).createRow((u) => [u.name.set('Alice')]); + final target = SchemaSnapshot([_userSchema(second, name: 'Members')]); + expect( + () => Migration.diff( + '0002_move', + dialect: .postgres, + from: start.snapshot!, + to: target, + ), + throwsA(isA()), + ); + final move = Migration.diff( + '0002_move', + dialect: .postgres, + from: start.snapshot!, + to: target, + renames: SchemaRenames(tables: {'$first.Users': '$second.Members'}), + previous: start.checksum, + ); + await Migrator(db.sql).apply([start, move]); + expect((await verifySchema(db.sql, target)).differences, isEmpty); + expect( + (await db.execute( + SqlCommand('SELECT "DisplayName" FROM "$second"."Members"'), + )).rows.single.single, + 'Alice', + ); + final removal = Migration.diff( + '0003_remove', + dialect: .postgres, + from: target, + to: SchemaSnapshot([]), + allowDestructive: true, + previous: move.checksum, + ); + await Migrator(db.sql).apply([start, move, removal]); + expect( + (await inspectTable(db.sql, 'Members', namespace: second)).columns, + isEmpty, + ); + }); + }, + skip: url == null + ? 'Set ORM_TEST_POSTGRES to a disposable PostgreSQL database.' + : false, + ); +} diff --git a/test/schema_boundary_test.dart b/test/schema_boundary_test.dart index 29bade04..07041b02 100644 --- a/test/schema_boundary_test.dart +++ b/test/schema_boundary_test.dart @@ -11,9 +11,13 @@ void main() { 'schema rejects malformed names, keys and local FK definitions offline', () { final id = Column('id', Codecs.integer); + for (final name in ['', 'bad\u0000name']) { + expect( + () => TableSchema(name, columns: [id]), + throwsA(isA()), + ); + } for (final table in [ - TableSchema('', columns: [id]), - TableSchema('bad\u0000name', columns: [id]), TableSchema('items', columns: [Column('', Codecs.integer)]), TableSchema('items', columns: [id], uniqueKeys: [[]]), TableSchema('items', columns: [id], primaryKey: ['id', 'id']), diff --git a/test/schema_builder_test.dart b/test/schema_builder_test.dart new file mode 100644 index 00000000..e89241e0 --- /dev/null +++ b/test/schema_builder_test.dart @@ -0,0 +1,104 @@ +import 'dart:io'; + +import 'package:build/build.dart'; +import 'package:build_test/build_test.dart'; +import 'package:orm/builder.dart'; +import 'package:test/test.dart'; + +import '../tool/src/build_fixture.dart'; + +void main() { + test( + 'package builder collects namespaces without a root Dart file', + () async { + final files = TestReaderWriter(rootPackage: 'orm', flattenOutput: true); + await files.testing.loadIsolateSources(); + final result = await testBuilder( + ormBuilder( + BuilderOptions({ + 'schema': 'lib/fixture/./schema.dart', + 'database': 'postgres', + }), + ), + { + 'orm|lib/fixture/schema/auth/users.dart': "import 'package:orm/schema.dart'; final user = model('users', (id: identity(),));", + 'orm|lib/fixture/schema/public/users.dart': "import 'package:orm/schema.dart'; final user = model('users', (id: identity(),));", + }, + rootPackage: 'orm', + readerWriter: files, + flattenOutput: true, + ); + expect(result.succeeded, true, reason: result.errors.toString()); + final client = files.testing.readString( + AssetId('orm', 'lib/fixture/schema.orm.dart'), + ); + expect(client, contains('final class AuthUser(')); + expect(client, contains('final class PublicUser(')); + }, + ); + + test('real directory build/watch tracks addition, removal, imports and CLI parity', () async { + final fixture = await BuildFixture.create(ormPath: Directory.current.path); + BuildWatch? watcher; + try { + await fixture.file('lib/schema.dart').delete(); + await fixture.write('build.yaml', ''' +targets: + \$default: + builders: + orm:orm: + enabled: true + options: + schema: lib/fixture/schema + database: postgres +'''); + await fixture.write( + 'lib/fixture/schema/public/users.dart', + "import 'package:orm/schema.dart'; final user = model('users', (id: identity(),));", + ); + await fixture.run(['run', 'build_runner', 'build']); + final client = fixture.file('lib/fixture/schema.orm.dart'); + final snapshot = fixture.file('lib/fixture/schema.snapshot.dart'); + expect(await client.readAsString(), contains('get user =>')); + final generated = await client.readAsString(), + frozen = await snapshot.readAsString(); + await fixture.run([ + 'run', + 'orm', + 'generate', + 'lib/fixture/schema', + '--database', + 'postgres', + ]); + expect(await client.readAsString(), generated); + expect(await snapshot.readAsString(), frozen); + watcher = await fixture.watch(); + await watcher.next(); + await fixture.write( + 'lib/fixture/schema/auth/users.dart', + "import 'package:orm/schema.dart'; final user = model('users', (id: identity(),));", + ); + await watcher.next(); + expect(await client.readAsString(), contains('AuthUser')); + expect(await client.readAsString(), contains('PublicUser')); + final beforeMove = await snapshot.readAsString(); + await fixture + .file('lib/fixture/schema/auth/users.dart') + .rename(fixture.file('lib/fixture/schema/auth/accounts.dart').path); + await watcher.next(); + expect(await snapshot.readAsString(), beforeMove); + await fixture.file('lib/fixture/schema/auth/accounts.dart').delete(); + await watcher.next(); + expect(await client.readAsString(), isNot(contains('AuthUser'))); + expect(await client.readAsString(), contains('get user =>')); + await fixture.write('lib/unrelated.dart', 'const unused = 1;'); + await watcher.quiet(); + await watcher.close(); + watcher = null; + await fixture.run(['analyze', 'lib/fixture/schema.orm.dart']); + } finally { + await watcher?.close(); + await fixture.dispose(); + } + }, timeout: const Timeout(Duration(minutes: 4))); +} diff --git a/test/schema_layout_test.dart b/test/schema_layout_test.dart new file mode 100644 index 00000000..756a6178 --- /dev/null +++ b/test/schema_layout_test.dart @@ -0,0 +1,390 @@ +import 'dart:io'; + +import 'package:orm/generate.dart'; +import 'package:orm/migrate.dart'; +import 'package:orm/src/generate/schema/layout.dart'; +import 'package:path/path.dart' as p; +import 'package:test/test.dart'; + +void main() { + test('layout respects native Windows and URL asset path contexts', () { + for (final paths in [p.Context(style: p.Style.windows), p.url]) { + final layout = SchemaLayout( + 'lib/fixture/./schema.dart', + dialect: .postgres, + directory: true, + paths: paths, + ); + expect(layout.root, paths.join('lib', 'fixture', 'schema')); + expect(layout.includes('lib/fixture/schema/auth/users.dart'), isTrue); + expect(layout.namespace('lib/fixture/schema/auth/users.dart'), 'auth'); + expect(layout.namespace('lib/fixture/schema.dart'), 'public'); + expect( + layout.includes('lib/fixture/schema/auth/deep/users.dart'), + isFalse, + ); + expect(layout.includes('lib/other/auth/users.dart'), isFalse); + } + }); + + late Directory project; + setUp(() async { + project = await Directory('.dart_tool').createTemp('schema-layout-'); + }); + tearDown(() => project.delete(recursive: true)); + + Future source(String path, String body) async { + final file = File('${project.path}/$path'); + await file.parent.create(recursive: true); + await file.writeAsString("import 'package:orm/schema.dart';\n$body"); + } + + for (final dialect in SqlDialect.values) { + test( + '${dialect.name} rejects outputs discovered as schema inputs', + () async { + final directory = dialect == SqlDialect.postgres + ? 'schema/public' + : 'schema'; + await source( + '$directory/users.dart', + "final user = model('users', (id: identity(),));", + ); + for (final output in { + 'schema.dart', + '$directory/client.dart', + 'schema/client.dart', + }) { + await expectLater( + writeGeneratedSchema( + '${project.path}/schema', + output: '${project.path}/$output', + dialect: dialect, + ), + throwsA(isA()), + ); + expect(File('${project.path}/$output').existsSync(), isFalse); + expect( + File(p.setExtension('${project.path}/$output', '.snapshot.dart')) + .existsSync(), + isFalse, + ); + } + final output = '${project.path}/$directory/client.orm.dart'; + await writeGeneratedSchema( + '${project.path}/schema', + output: output, + dialect: dialect, + ); + final before = await File(output).readAsString(); + await writeGeneratedSchema( + '${project.path}/schema', + output: output, + dialect: dialect, + ); + expect(await File(output).readAsString(), before); + }, + ); + } + + test( + 'PG directory namespaces, repeated names and cross-schema references', + () async { + await source( + 'schema/auth/users.dart', + "final user = model('Users', (id: identity(), displayName: text(name: 'DisplayName')));", + ); + await source( + 'schema/public/users.dart', + "import '../auth/users.dart' as auth;\nfinal user = model('Users', (id: identity(), accountId: integer()), relations: (u) => (account: references(u.accountId, () => auth.user),));", + ); + final result = await generateSchema( + '${project.path}/schema', + dialect: .postgres, + ); + expect(result.snapshot.tables.map((t) => t.identity), [ + 'auth.Users', + 'public.Users', + ]); + expect( + result.snapshot.tables.last.foreignKeys.single.targetNamespace, + 'auth', + ); + expect(result.dart, contains('final class AuthUser(')); + expect(result.dart, contains('final class PublicUser(')); + expect(result.dart, contains('get auth =>')); + expect(result.dart, contains('get public =>')); + await writeGeneratedSchema('${project.path}/schema/', dialect: .postgres); + final analysis = await Process.run(Platform.resolvedExecutable, [ + 'analyze', + project.path, + ]); + expect( + analysis.exitCode, + 0, + reason: '${analysis.stdout}\n${analysis.stderr}', + ); + final sql = Migration.create( + '0001_initial', + result.snapshot.tables, + dialect: .postgres, + ).steps.cast().map((s) => s.sql).join('\n'); + expect(sql, contains('CREATE TABLE "auth"."Users"')); + expect(sql, contains('REFERENCES "auth"."Users"')); + }, + ); + + test( + 'all entry spellings merge the default file and directory once', + () async { + await source( + 'schema.dart', + "export 'schema/public/users.dart';\nfinal post = model('posts', (id: identity(),));", + ); + await source( + 'schema/public/users.dart', + "final user = model('users', (id: identity(),));", + ); + final checksums = {}; + for (final suffix in ['schema', 'schema/', 'schema.dart']) { + final result = await generateSchema( + '${project.path}/$suffix', + dialect: .postgres, + ); + checksums.add(result.snapshot.checksum); + expect(result.snapshot.tables.map((t) => t.identity), [ + 'public.posts', + 'public.users', + ]); + expect(result.dart, contains('get user =>')); + expect(result.dart, isNot(contains('get public =>'))); + } + expect(checksums, hasLength(1)); + }, + ); + + test('directory generation requires an offline engine selection', () async { + await source( + 'schema/users.dart', + "final user = model('users', (id: identity(),));", + ); + await expectLater( + generateSchema('${project.path}/schema'), + throwsA(isA()), + ); + }); + + test( + 'splitting a targeted file into public preserves the snapshot', + () async { + const posts = "final post = model('posts', (id: identity(),));"; + const users = "final user = model('users', (id: identity(),));"; + await source('schema.dart', '$users\n$posts'); + final before = await generateSchema( + '${project.path}/schema', + dialect: .postgres, + ); + await source('schema.dart', users); + await source('schema/public/posts.dart', posts); + final after = await generateSchema( + '${project.path}/schema', + dialect: .postgres, + ); + expect(after.snapshot.checksum, before.snapshot.checksum); + expect( + Migration.diff( + '0002_split', + dialect: .postgres, + from: before.snapshot, + to: after.snapshot, + ).steps, + isEmpty, + ); + }, + ); + + test( + 'replacing unqualified PostgreSQL snapshots requires destructive opt-in', + () async { + await source('schema.dart', ''' +final user = model('users', (id: identity(),)); +final post = model('posts', (id: identity(), authorId: integer()), + relations: (p) => (author: references(p.authorId, () => user),)); +'''); + final legacy = await generateSchema('${project.path}/schema'); + final current = await generateSchema( + '${project.path}/schema', + dialect: .postgres, + ); + expect( + () => Migration.diff( + '0002_qualified', + dialect: .postgres, + from: legacy.snapshot, + to: current.snapshot, + ), + throwsA( + isA().having( + (e) => e.code, + 'code', + 'MIGRATION.DESTRUCTIVE', + ), + ), + ); + expect( + () => Migration.diff( + '0002_renamed', + dialect: .postgres, + from: legacy.snapshot, + to: current.snapshot, + renames: SchemaRenames( + tables: {'users': 'public.users', 'posts': 'public.posts'}, + ), + ), + throwsA( + isA().having((e) => e.code, 'code', 'MIGRATION.RENAME'), + ), + ); + final change = Migration.diff( + '0002_qualified', + dialect: .postgres, + from: legacy.snapshot, + to: current.snapshot, + allowDestructive: true, + ); + expect(change.steps.whereType().map((s) => s.table).toSet(), { + 'users', + 'posts', + }); + expect( + change.steps.whereType().map((s) => s.sql).join('\n'), + contains('CREATE TABLE "public"."users"'), + ); + expect( + change.snapshot!.tables.every((t) => t.namespace == 'public'), + true, + ); + }, + ); + + for (final dialect in [ + SqlDialect.sqlite, + SqlDialect.mysql, + SqlDialect.mariadb, + ]) { + test( + '${dialect.name} collects direct files without namespaces or recursion', + () async { + await source( + 'schema/users.dart', + "final user = model('users', (id: identity(),));", + ); + await source( + 'schema/nested/posts.dart', + "final post = model('posts', (id: identity(),));", + ); + final result = await generateSchema( + '${project.path}/schema', + dialect: dialect, + ); + expect(result.snapshot.tables.map((t) => t.name), ['users']); + expect(result.snapshot.tables.single.namespace, isNull); + }, + ); + } + + test('references cannot pull models through deeper directories', () async { + await source( + 'schema/auth/users.dart', + "import 'nested/accounts.dart';\nfinal user = model('users', (id: identity(), accountId: integer()), relations: (u) => (account: references(u.accountId, () => account),));", + ); + await source( + 'schema/auth/nested/accounts.dart', + "final account = model('accounts', (id: identity(),));", + ); + await expectLater( + generateSchema('${project.path}/schema', dialect: .postgres), + throwsA( + isA().having( + (e) => e.toString(), + 'diagnostic', + contains('outside the schema layout'), + ), + ), + ); + }); + + test( + 'moving declarations within a namespace preserves physical identity', + () async { + await source( + 'schema/auth/users.dart', + "final user = model('users', (id: identity(),));", + ); + final before = await generateSchema( + '${project.path}/schema', + dialect: .postgres, + ); + await File('${project.path}/schema/auth/users.dart') + .rename('${project.path}/schema/auth/accounts.dart'); + final after = await generateSchema( + '${project.path}/schema', + dialect: .postgres, + ); + expect(after.snapshot.checksum, before.snapshot.checksum); + }, + ); + + test( + 'dotted table names fail instead of being interpreted as qualification', + () async { + await source( + 'schema.dart', + "final user = model('auth.users', (id: identity(),));", + ); + await expectLater( + generateSchema('${project.path}/schema.dart', dialect: .postgres), + throwsA( + isA().having( + (e) => e.code, + 'code', + 'SCHEMA.IDENTIFIER', + ), + ), + ); + }, + ); + + test( + 'PG rejects direct directory files and conflicting physical declarations', + () async { + await source( + 'schema/users.dart', + "final user = model('users', (id: identity(),));", + ); + await expectLater( + generateSchema('${project.path}/schema', dialect: .postgres), + throwsA(isA()), + ); + await File('${project.path}/schema/users.dart').delete(); + await source( + 'schema.dart', + "final user = model('users', (id: identity(),));", + ); + await source( + 'schema/public/users.dart', + "final user = model('users', (id: identity(),));", + ); + await expectLater( + generateSchema('${project.path}/schema', dialect: .postgres), + throwsA( + isA().having( + (e) => e.toString(), + 'diagnostic', + contains('also declared'), + ), + ), + ); + }, + ); +}