Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
c65ab6d
Complete SQL entity source-anchor bounds protocol
deem0n Sep 20, 2026
d86ed12
Give SQL catalog entities qualified Moose names
deem0n Sep 20, 2026
9b5b5c6
Adapt SQL source definitions to the standard Moose browser
deem0n Sep 20, 2026
0a4909b
Add SQL relation extraction for standard CoUsage maps
deem0n Sep 20, 2026
3e86c85
Provide a conservative SQL dead-code analysis preset
deem0n Sep 20, 2026
5e2ec45
Load pinned archives independently of prebuilt Moose Iceberg checkouts
deem0n Sep 20, 2026
0139e85
Load pinned archives independently of prebuilt Moose Iceberg checkouts
deem0n Sep 20, 2026
0ad44a4
Load pinned archives independently of prebuilt Moose Iceberg checkouts
deem0n Sep 20, 2026
472e707
Add SQL-aware duplication cleaning and standard browser preset
deem0n Sep 20, 2026
4b2ecbb
Remove stale Iceberg registrations from disposable Moose CI image
deem0n Sep 20, 2026
4aee4a9
Provide tested Roassal 3 SQL views and retire LegacyUI load group
deem0n Sep 20, 2026
857df6b
Use Iceberg registry API to unregister prebuilt repositories
deem0n Sep 20, 2026
76fb2e9
Integrate qualified names for v3 qualification
deem0n Sep 20, 2026
5ea0c59
Integrate source anchor and browser fixes for v3 qualification
deem0n Sep 20, 2026
60cdc8c
Integrate SQL CoUsage for v3 qualification
deem0n Sep 20, 2026
f1da8f1
Integrate conservative SQL dead-code preset for v3 qualification
deem0n Sep 20, 2026
7bdee10
Integrate SQL duplication for v3 qualification
deem0n Sep 20, 2026
26a0391
Integrate Roassal 3 SQL views for v3 qualification
deem0n Sep 20, 2026
620cd14
Add v3 qualification roadmap, combined dependency pins and SQL naviga…
deem0n Sep 20, 2026
a871283
Qualify pinned MooseIDE packages against prebuilt Moose dependencies
deem0n Sep 20, 2026
0b6dadb
Require new generic regression classes in combined Moose CI
deem0n Sep 20, 2026
198e74b
Load pinned visualization package alongside its required regression t…
deem0n Sep 20, 2026
187ae92
Preserve prior namespace and Fame fixes in the final qualification pin
deem0n Sep 20, 2026
4a3e0dc
Pin reviewed namespace workflow cleanup for v3 qualification
deem0n Sep 20, 2026
a2da310
Point review snippet at the complete v3 qualification candidate
deem0n Sep 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,11 @@ jobs:
strategy:
fail-fast: false
matrix:
image: [Pharo64-13, Moose64-13]
include:
- image: Pharo64-13
config: .smalltalk.ston
- image: Moose64-13
config: .smalltalk.moose.ston
services:
postgres:
image: postgres:15
Expand All @@ -43,7 +47,7 @@ jobs:
with:
smalltalk-image: ${{ matrix.image }}
- name: Load and test in a clean image
run: smalltalkci -s '${{ matrix.image }}' .smalltalk.ston
run: smalltalkci -s '${{ matrix.image }}' '${{ matrix.config }}'
- name: Upload JUnit results
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
Expand Down
22 changes: 22 additions & 0 deletions .smalltalk.moose.ston
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
SmalltalkCISpec {
#name : 'FAMIXNGSQL-2-Pharo13-Moose13',
#preLoading : 'scripts/ci/prepare-moose.st',
#loading : [
SCIMetacelloLoadSpec {
#baseline : 'FAMIXNGSQL',
#directory : 'src',
#load : [ 'Tests', 'GeneratorTests', 'MooseIDETests' ],
#onConflict : #useIncoming,
#onUpgrade : #useIncoming,
#platforms : [ #pharo ]
}
],
#postLoading : 'scripts/ci/load-famix-regressions.st',
#preTesting : 'scripts/ci/configure-database.st',
#testing : {
#packages : [ 'FamixNGSQL-Importer-Tests', 'FAMIXNGSQLMetamodelGenerator-Tests', 'PgMetadata-tests', 'FamixNGSQL-MooseIDE-Tests' ],
#classes : [ #MiMetaBrowserRegressionTest, #MiModelReportRegressionTest, #MiLayerApplicabilityTest, #MiApplicationTest, #MiButterflyMapModelTest, #MiButterflyMapBrowserTest, #MooseGroupNameCloudTest, #MQNavigationContainmentTest, #FamixNamespaceGroupHierarchyTest, #MooseGroupTest ],
#defaultTimeout : 120,
#failOnZeroTests : true
}
}
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,10 @@ immutable Moose 13 release. smalltalkCI selects it with `Moose64-13`.
| `Tests` | Core, importer tests and PgMetadata scenario-test support |
| `Generator` | Core and the metamodel generator |
| `GeneratorTests` | Generator plus its global-regeneration selection regression test |
| `LegacyUI` | Historical GT Inspector, analysis, Telescope and connection-manager packages |
| `MooseIDE` | Optional standard Moose tool adapters and Roassal 3 SQL views (requires Moose 13) |
| `MooseIDETests` | Optional tool and visualization regression tests |

`LegacyUI` has not been ported or validated on the current image. Core usage does
The unsupported `LegacyUI` load group is removed for v3. Historical sources remain archived in `src`; use the [Roassal 3 SQL views](docs/mooseide-integration.md) instead. Core usage does
not require those UI packages or a Genie MCP server. Loading modern Genie in a
historical Pharo 7 image is not part of the installation procedure.

Expand Down Expand Up @@ -487,3 +488,7 @@ export can be parsed.
Distributed under the [MIT license](LICENSE). This fork builds on
[Julien Delplanque's FAMIXNGSQL](https://github.com/juliendelplanque/FAMIXNGSQL),
with updated metadata extraction, SQL parsing and Moose metamodel integration.

## v3.0.0 qualification

See the [release plan and issue/PR map](docs/v3-release-plan.md) for the compatibility audit, separate fixes, breaking changes and release gates. This branch is an integration candidate; v3.0.0 is not released.
334 changes: 334 additions & 0 deletions docs/moose-compatibility-audit.md

Large diffs are not rendered by default.

84 changes: 84 additions & 0 deletions docs/mooseide-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Optional Moose 13 integration

In a Moose 13 / Pharo 13 image with MooseIDE already installed, load the `MooseIDE`
group of FAMIXNGSQL. `MooseIDETests` adds its regression tests. The plain `Core`
group does not load these packages or require the Moose UI. The snippet pins the
review candidate; use `master` only after its prerequisite PRs are merged.

```smalltalk
Metacello new
baseline: 'FAMIXNGSQL';
repository: 'github://deem0n/FAMIXNGSQL:4a3e0dcb11e7ce06684ba19d4addb61976b96498/src';
load: 'MooseIDE'.
```

The Source Text browser automatically selects the SQL adapter. Routine and view
sources display without filesystem anchors. AST highlights are translated from
owner-relative intervals into the displayed fragment; absent, invalid, unrelated
or outside-fragment intervals produce no highlight. Definitions are not modified.

The dependency pin includes the cardinality-aware MooseQuery fix from
`deem0n/Famix` PR #2, required for views with scalar query containment. This is
fork-first v3 development work, not a published v3 release.

GitHub CI loads these optional tests in the ready-made Moose image. The plain
Pharo job continues to test Core without GUI dependencies.

## SQL CoUsage

Select **SQL relation references** as the inner-box extractor in the standard
CoUsage map settings, or use:

```smalltalk
FmxSQLCoUsageExtractor openOn: (model allWithType: FmxSQLStoredProcedure).
```

Containers are routines, views or tables; inner boxes are referenced database
tables/views. Column references are projected to their owning relation, and
foreign keys contribute referenced relations. Repeated references retain usage
counts. Empty results mean no modeled references, not proof that dynamic SQL has
no dependencies. The helper rejects mixed/unsupported input before opening.

## Dead-code candidates

Use `FmxSQLDeadCodeRule openOn: routines` to open the standard Dead Code browser
with the SQL refuting rule selected, or select that rule explicitly in an existing
browser before computing. SQL routines are externally callable, so this SQL preset
protects all of them until a closed-world candidate set is explicitly supplied.
The generic OO preset alone is not a SQL dead-code analysis.
Trigger/constraint bindings and missing or incomplete analysis remain protected.
For an explicitly bounded analysis, configure its selected `FmxSQLDeadCodeRule`:

```smalltalk
rule closedWorldCandidates: knownClosedWorldRoutines.
rule externalEntryPoints: knownExternalRoots.
```

The remaining results are candidates, not proof that routines can be dropped.
This integration does not execute database DDL or remove definitions. Existing
non-SQL dead-code rules retain their behavior.

## SQL duplication analysis

Choose **PostgreSQL** as the source cleaner in the standard Duplication Browser settings, or open it with the preset:

```smalltalk
FmxSQLReplicationCleaner openOn: (model entities select: [ :entity |
entity hasSourceAnchor and: [ entity sourceAnchor hasSourceText ] ]).
```

The cleaner preserves SQL literals, quoted identifiers and dollar-quoted bodies; removes line and nested block comments; and keeps original line numbers. Whitespace inside literals stays significant. It assumes PostgreSQL `standard_conforming_strings = on`; explicit `E` strings support escaped quotes. Unterminated lexical input raises an error instead of silently changing its meaning. Dollar-quoted bodies are compared verbatim, without recursively interpreting their language. This is lexical clone detection, not semantic SQL equivalence. Thresholds remain configurable in the standard browser.

## Roassal 3 SQL views

The builders return unopened `RSCanvas` objects. Use any imported SQL model; there is no global `mi` dependency or implicit schema filter. For large databases pass a deliberately selected model, since all matching entities are drawn.

```smalltalk
(FmxSQLRoassalViews schemaTablesIn: model) openWithTitle: 'Schemas and tables'.
(FmxSQLRoassalViews foreignKeysIn: model) openWithTitle: 'Foreign keys'.
(FmxSQLRoassalViews routineCallsIn: model) openWithTitle: 'Routine call candidates'.
```

Schema boxes contain table boxes, like class/method nesting examples. Foreign-key arrows point from the referencing table to the referenced table; composite references collapse to one edge. Call arrows point from the enclosing routine to every candidate routine, including trigger routines. Candidate edges are static-analysis evidence, not proof of runtime calls. Disconnected entities remain visible; endpoints outside the supplied model are omitted. Edges are placed above nodes, and no entities or relationships are changed. Empty models yield empty canvases.

The historical Roassal 2/GT/Telescope sources remain in the repository for migration reference, but the unsupported `LegacyUI` load group is retired in v3. These Roassal 3 builders use current association APIs and replace the advertised visualization entry points.
29 changes: 29 additions & 0 deletions docs/sql-moose-names.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# SQL Moose names for v3.0.0

Short `name` values remain PostgreSQL identifiers. `mooseName` is now a derived,
qualified identifier for navigation and lookup. It includes the SQL entity kind,
quoted ownership path, and input type signature for routines. For example:

- `Table:"app"."users"`
- `Column:"app"."users"."id"`
- `Type:"app"."users"` (a distinct naming domain from the table)
- `Routine:"app"."lookup"("pg_catalog"."int4")`

Double quotes inside identifiers are doubled. OUT/TABLE parameters do not identify
an overload; IN, INOUT and variadic parameters do. Trigger routines share the
routine naming domain. Parameter positions preserve signature order.

Names are computed from current ownership and types, avoiding stale cached names
when a schema, owner or type is renamed or a member is reparented. `name` and
PostgreSQL OIDs are unchanged. MSE stores the original names and relations; reloading
reconstructs the same qualified names.

This is a breaking change for scripts using `entityNamed:` with short strings or
persisting old Moose names. Pass an entity's new `mooseName`, or explicitly select
by `name` and schema when intentionally making a contextual lookup. Do not use
short labels as global identifiers.

Unscoped entities, local AST names and synthetic not-null constraints do not claim
a globally unique Moose name. An incomplete routine signature also cannot claim
uniqueness. Duplicate catalog entities should still be diagnosed at import time;
these names do not silently merge objects.
55 changes: 55 additions & 0 deletions docs/v3-release-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# FAMIXNGSQL v3.0.0 qualification

Status: review and integration candidate, not a published release. Track all work in the [v3.0.0 milestone](https://github.com/deem0n/FAMIXNGSQL/milestone/1). The [installed-image audit](moose-compatibility-audit.md) records the original failures and limits of coverage.

## Separate fixes

| Issue | Change | Implementation PR |
|---|---|---|
| [#11](https://github.com/deem0n/FAMIXNGSQL/issues/11) | Qualified SQL names and overloads | [SQL #26](https://github.com/deem0n/FAMIXNGSQL/pull/26) |
| [#12](https://github.com/deem0n/FAMIXNGSQL/issues/12) | Source-anchor protocol | [SQL #25](https://github.com/deem0n/FAMIXNGSQL/pull/25) |
| [#13](https://github.com/deem0n/FAMIXNGSQL/issues/13) | Scalar containment traversal | [Famix #2](https://github.com/deem0n/Famix/pull/2) |
| [#14](https://github.com/deem0n/FAMIXNGSQL/issues/14) | Standard Source Text browser | [SQL #27](https://github.com/deem0n/FAMIXNGSQL/pull/27) |
| [#15](https://github.com/deem0n/FAMIXNGSQL/issues/15) | SQL CoUsage preset | [SQL #28](https://github.com/deem0n/FAMIXNGSQL/pull/28) |
| [#16](https://github.com/deem0n/FAMIXNGSQL/issues/16) | Conservative dead-code preset | [SQL #29](https://github.com/deem0n/FAMIXNGSQL/pull/29) |
| [#17](https://github.com/deem0n/FAMIXNGSQL/issues/17) | SQL duplication cleaner | [SQL #30](https://github.com/deem0n/FAMIXNGSQL/pull/30) |
| [#18](https://github.com/deem0n/FAMIXNGSQL/issues/18) | Reject unsupported Layer Visualization inputs | [MooseIDE #4](https://github.com/deem0n/MooseIDE/pull/4) |
| [#19](https://github.com/deem0n/FAMIXNGSQL/issues/19) | Non-OO model report | [MooseIDE #2](https://github.com/deem0n/MooseIDE/pull/2) |
| [#20](https://github.com/deem0n/FAMIXNGSQL/issues/20) | Meta Browser Spec properties | [MooseIDE #1](https://github.com/deem0n/MooseIDE/pull/1) |
| [#21](https://github.com/deem0n/FAMIXNGSQL/issues/21) | Name Cloud | [Famix #1](https://github.com/deem0n/Famix/pull/1) |
| [#22](https://github.com/deem0n/FAMIXNGSQL/issues/22) | Roassal 3 SQL views | [SQL #31](https://github.com/deem0n/FAMIXNGSQL/pull/31) |
| [#23](https://github.com/deem0n/FAMIXNGSQL/issues/23) | Correct provider direction | [MooseIDE #3](https://github.com/deem0n/MooseIDE/pull/3) |
| [#24](https://github.com/deem0n/FAMIXNGSQL/issues/24) | Combined CI, immutable pins and release gate | This integration PR |

## Dependencies and merge order

All PRs target deem0n forks first. Famix #1/#2 plus the pre-existing namespace hierarchy and Fame fixes, and MooseIDE #1–#4 are independent review units. The draft [Famix #3](https://github.com/deem0n/Famix/pull/3) and [MooseIDE #5](https://github.com/deem0n/MooseIDE/pull/5) combine them solely for qualification; merge individual fixes first. SQL #25 and #26 can be reviewed independently. SQL #27 is stacked on #25, and #28–#31 on #27; retarget each to master after its prerequisite is merged.

This branch combines the SQL fixes and pins Famix `a4c7ab04868fc924bb0ca470142c60d8576f6a17`. The Moose CI preparation script additionally pins complete changed MooseIDE packages at `0078da66db3575346e3a25402a90684686234f07`. These are immutable qualification commits containing the separate PRs, not moving upstream development branches. After review, ensure the same fixes are reachable from the fork default branches and refresh pins if review changes them.

## Validation and release gates

The CI matrix is clean Pharo 13 core/importer/generator tests and prebuilt Moose 13 with optional UI tests, both against PostgreSQL 15. CI unregisters stale Iceberg checkout registrations in the disposable prebuilt image and uses archive downloads, so absent build-machine directories cannot override pinned forks. MooseIDE packages are loaded into prebuilt Moose 13 without reloading unrelated moving baseline dependencies; the fork is also tested from clean Pharo using its full baseline. MooseIDE fork CI needs repository variable `MOOSE_PHARO_VERSIONS={"Moose-latest":["Pharo64-13"]}` because upstream organization variables are not inherited by a fork.

Combined isolated validation: 172 passed, 2 skipped, zero failures/errors across SQL and generic-tool regression suites. The two existing Butterfly Map skips remain visible; no tests were disabled to obtain this result. Model Report HTML export emits an existing undeclared MicHTMLVisitor warning and is not qualified by the text-report fix. Individual SQL PRs #25–#31 passed both CI image jobs after correcting the prebuilt-image loading configuration. Consult each current commit's checks; this is a dated validation record, not a promise about future commits.

- [ ] Review and merge every separate implementation PR.
- [ ] All milestone issues, including existing regeneration issue #8, meet their acceptance criteria.
- [ ] Combined clean Pharo 13 / Moose 13 / PostgreSQL 15 CI passes at the final dependency pins.
- [ ] Full metamodel regeneration and MSE tests pass without unexpected generated changes.
- [ ] Reimport all application schemas of local `mi` and repeat the installed-tool acceptance matrix at final pins; verify identities, relationships and parse-analysis reports.
- [ ] Check the supported tool paths interactively, including source highlights, selected clone display and graph navigation; record rejected/inapplicable tools honestly.
- [ ] Confirm no database artifacts or credentials are tracked.
- [ ] Publish v3.0.0 only after the above gates; do not reuse v1.0.0/v2.0.0 tags.

The issue/PR set implements discovered incompatibilities. It does not imply universal compatibility with all Moose plugins, semantic SQL clone detection, complete parsing of dynamic SQL, or invented OO metrics/layer semantics. The installed-tool audit and final acceptance run define the evidence.

## Breaking changes and loading

Qualified Moose names change lookup/cache/export identifiers; see [naming migration](sql-moose-names.md). The unsupported LegacyUI load group is retired. Core stays independent of GUI packages; optional standard-tool adapters and Roassal 3 views use `MooseIDE`/`MooseIDETests` groups and require MooseIDE already installed. SQL CoUsage, dead-code and duplication use explicit SQL presets; see [working snippets](mooseide-integration.md). The generic OO defaults must not be assumed to carry SQL semantics.

Qualification scope is Pharo 13/Moose 13/PostgreSQL 15. Do not advertise Pharo 7–12, Moose 12, Pharo 14 or other PostgreSQL versions as supported by v3 until their own clean matrix is green. Famix's broader fork matrix is separate evidence, not full-stack SQL qualification.

The two pre-existing Famix fixes are also required: upstream [#1150](https://github.com/moosetechnology/Famix/pull/1150) (namespace hierarchy) and [#1149](https://github.com/moosetechnology/Famix/pull/1149) (Fame group inspection). Their existing branches are included in the qualification pin, with `FamixNamespaceGroupHierarchyTest` and `MooseGroupTest` required in combined CI. The original audit image already had these fixes; omitting them from a clean installation would regress it.

Fork preservation PRs: [Famix #4](https://github.com/deem0n/Famix/pull/4) for namespace hierarchy and [Famix #5](https://github.com/deem0n/Famix/pull/5) for the already upstream-merged Fame fix. As requested in upstream #1150 review, the extra namespace workflow/configuration is removed while the seven regression tests are retained; the existing combined Moose CI runs them.
13 changes: 13 additions & 0 deletions scripts/ci/load-famix-regressions.st
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
"Load regression classes from the same immutable Famix commit as the baseline.
The prebuilt image contains their Java/Smalltalk test model dependencies."
| repository loader |
repository := MCGitHubRepository location:
'github://deem0n/Famix:a4c7ab04868fc924bb0ca470142c60d8576f6a17/src'.
loader := MCVersionLoader new.
#('Famix-Visualizations' 'Famix-Visualizations-Tests' 'Moose-Core-Tests' 'Moose-Query-Test') do: [ :packageName |
loader addVersion: (repository packageNamed: packageName) ].
loader load.
#(MiMetaBrowserRegressionTest MiModelReportRegressionTest MiLayerApplicabilityTest
MiApplicationTest MiButterflyMapModelTest MiButterflyMapBrowserTest
MooseGroupNameCloudTest MQNavigationContainmentTest FamixNamespaceGroupHierarchyTest MooseGroupTest) do: [ :name |
Smalltalk at: name ifAbsent: [ self error: 'Missing required regression class: ', name ] ].
17 changes: 17 additions & 0 deletions scripts/ci/prepare-moose.st
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
"Use immutable archive dependencies, independent of prebuilt Iceberg checkouts.
Only affects the disposable CI image."
Iceberg enableMetacelloIntegration: false.
IceRepository registry copy do: [ :repository |
IceRepository unregisterRepository: repository ifAbsent: [ ] ].

"Moose 13 already supplies these tools and their dependencies. Load complete
packages from the pinned fork, avoiding unrelated moving baseline dependencies.
The MooseIDE fork itself is separately tested from clean Pharo by its full CI."
[ | repository loader |
repository := MCGitHubRepository location:
'github://deem0n/MooseIDE:0078da66db3575346e3a25402a90684686234f07/src'.
loader := MCVersionLoader new.
#('MooseIDE-Core' 'MooseIDE-Core-Reporter' 'MooseIDE-LayerVisualization'
'MooseIDE-Meta' 'MooseIDE-Tests') do: [ :packageName |
loader addVersion: (repository packageNamed: packageName) ].
loader load ] value.
Loading