refactor: extract the schema language into worktable_dsl - #79
Conversation
A schema is written down once, in a `worktable!` invocation, and the parser that understands it lived in `worktable_codegen`, which is `proc-macro = true`. A proc-macro crate can export nothing but macros, so every type describing a schema — the columns, the primary key, the indexes, the queries — was unreachable from any other crate however public it was declared. `mod common` was not public at that crate's root either. So anything wanting to *read* a declaration had two options: re-implement the grammar and drift from it, or do without. A diagram, a migration tool, a documentation generator and an editor all want to read one. `lib.rs` has carried `// TODO: Refactor this codegen stuff because it's now too strange.` `model` and `parser` move to `worktable_dsl`, a plain library. Nothing in them changed; the dependencies are the five they already used, none added, none dropped. `worktable_codegen` now depends on it, so there is one grammar rather than a copy that can disagree with the compiler about what a schema means. `name_generator` stays in codegen. It invents Rust identifiers for generated code, which is not the schema language, and generators here define inherent `impl`s on `WorktableNameGenerator` — the orphan rule allows that only in the crate owning the type. I had it in the extracted crate first and the compiler made the same argument the design does. `crate::common::` still resolves, through a thin module that re-exports the new crate, so the 127 paths across 67 files are untouched and the diff stays a move rather than a sweep. An integration test reads a declaration from outside, which is the claim worth pinning: it compiles as its own crate, so it stops building if this ever becomes a proc-macro crate again. It also records a property no caller existed to depend on before. `Columns::columns_map` is a `std::collections::HashMap`, whose iteration order Rust randomises per process; two runs of the same input gave `["answered", "project_id", "id"]` and `["project_id", "answered", "id"]`. The macro never cared, and the parser's own tests collect it into another `HashMap` and assert membership, so nothing noticed. A consumer rendering columns in that order draws a different table every run. `field_positions` already carries the declaration order and is the field to sort by; the test asserts that, and says so, so the next consumer learns it here rather than by shipping the bug.
`pub use worktable_dsl::{Parser, *}` failed clippy under `-D warnings` with
"unused import: `*`". `worktable_codegen` is a proc-macro crate, so its
`pub use` re-exports are not reachable from outside it -- the glob was only
ever visible within this crate, and nothing here needed what it brought in
beyond the `model` and `parser` modules the next line already re-exports.
Naming `Parser` alone keeps every `crate::common::` path working.
|
Green now. The three failing checks were one cause:
Coordination note. There is an unpushed local branch |
|
Superseded by #87, which carries this extraction as its first two commits ( |
1 participant
The problem
A WorkTable schema is written down exactly once, in a
worktable!invocation. The parser that understands it — and understands it exactly as the compiler does — lived inworktable_codegen, which is declaredproc-macro = true.A proc-macro crate can export nothing but macros. So
Columns,Index,PrimaryKey,Persistence,PartitionKey,Queriesand the parser that produces them were unreachable from any other crate no matter howpubthey were.mod commonwas not even public at the crate root.Anything wanting to read a schema therefore had two options: re-implement the grammar and drift from it, or do without. A diagram, a migration tool, a documentation generator and a visual editor all want to read one. I started re-implementing it before finding this, which is the evidence that the trap is real.
codegen/src/lib.rshas been carrying// TODO: Refactor this codegen stuff because it's now too strange.The change
modelandparsermove to a new plain library,worktable_dsl.Nothing in them changed. Same code, same five dependencies — none added, none dropped.
worktable_codegendepends on the new crate, so there is one grammar rather than a copy that can disagree with the compiler about what a schema means.crate::common::still resolves, through a thin module that re-exports the new crate. The 127 such paths across 67 files are untouched, so this reads as a move rather than a sweep.What stayed behind, and why
name_generatorremains in codegen. It invents Rust identifiers for generated code, which is not part of the schema language — and generators here define inherentimpls onWorktableNameGenerator, which the orphan rule permits only in the crate that owns the type.I had it in the extracted crate first and the build refused. The compiler makes the same argument the design does about where it belongs.
A property that had no owner until now
The integration test records something worth knowing before anyone builds on this.
Columns::columns_mapis astd::collections::HashMap, whose iteration order Rust randomises per process. Two runs over the same input gave:Nothing had noticed, and nothing needed to: the macro does not care what order it sees columns in, and the parser's own tests collect
columns_mapinto anotherHashMapand assert membership. The property was never specified because no caller existed to depend on it.A consumer does. A diagram or documentation page iterating that map renders a different table on every run and looks like its own bug.
field_positionsalready carries declaration order and is the field to sort by. The test asserts that and explains why, so the next consumer finds out here instead of by shipping it.I have not changed the map type or insertion order. That changes what generated code sees, and it belongs to whoever owns the macro — not to a drive-by inside an extraction whose entire claim is that nothing changed.
Verification
cargo fmt --all --checkclean