feat: read WorkTable's DSL and draw the store schema - #204
Open
pathscale wants to merge 3 commits into
Open
Conversation
added 3 commits
August 31, 2026 20:58
A reader crate for `worktable!` declarations: columns with their types, primary key, generators and index backends, the indexes, the generated queries, `persist`, `version`, `partition_by` and `page_size`. The grammar is walked here rather than borrowed. WorkTable's own parser lives in its proc-macro crate, which returns generated code rather than a description of what it read, and `syn` stops at the token stream inside the parentheses because a `macro_rules!` invocation has no typed AST. So the grammar is mirrored in one file, kept small, and unknown sections are an error: a reader that shrugs at what it does not understand describes a schema that is quietly missing something. Column comments come back too. They are `//`, not `///`, so the lexer drops them before any parser sees them, and they are recovered by line number. In this codebase that prose is most of what a diagram has worth showing. The DSL declares no relationships, so edges are proposed from naming and every one carries where it came from. What the convention cannot resolve is reported with suggestions instead of dropped, and `schema.overlay.json` carries the corrections a person makes, including the two edges inference structurally cannot see, where the primary key itself is the reference. Read-only, and it stays that way. Editing a `worktable!` means driving the schema fingerprint and a migration in the same breath, because rows are persisted positionally with rkyv and nothing on disk records which columns wrote them. The tests check the parse against `db/fingerprint.rs`, which states every table's column list, in order, and is maintained by hand rather than generated from these files. A missed column, a reordered one or a skipped file each break it by name.
A Schema tab beside Analytics: every `worktable!` in `apps/gui/src/db/schema` as a box of columns, laid out by reference depth, so the two tables that reference nothing sit in the first column and everything that points at them fans out to the right. Fifteen of seventeen tables carry a `project_id`, and a drawing that does not commit to a direction turns that into a hairball. Clicking a table opens what the source says about it: the module header, every column with the comment above it, the indexes, the generated queries, and the references in both directions. That prose is the reason the reader keeps `//` comments at all, being the only text on the screen nobody invented for the diagram. Inferred and declared edges are drawn differently, dashed for the overlay, because they are not the same claim. An inferred edge is a naming convention holding up; an overlay edge is a person's decision. A faded edge is one the source documents as often empty, which in a store with no nullable columns means an empty string rather than an absent value. The data is a committed JSON file, not a Tauri command. The diagram therefore renders in the browser fixture on :3010 with no build and no instance, and a schema change reaches review as a diff of the drawing's input rather than as something only a running app can show. `scripts/generate-schema.sh` rebuilds it, and biome leaves that file alone: its formatter is serde_json's, so formatting it here would make every regeneration produce a whitespace diff that the next `--write` undoes again. Read-only, and the tab says so in its own module doc. Editing a `worktable!` means driving `SCHEMA_FINGERPRINT` and a migration in the same breath.
The drawing is ps-qa's to judge, but everything that makes it readable is arithmetic and can be checked here: no two boxes overlap in either density, every relationship is routed, every edge runs right to left so direction means something, and a compact box never hides a column an edge starts from. Against the generated schema rather than a fixture, so a table added to the store is a table these assertions cover.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reads every
worktable!inapps/gui/src/db/schemaand draws it: a Schema tabbeside Analytics, with the tables laid out by reference depth so the two that
reference nothing sit in the first column and the fifteen that carry a
project_idfan out to the right.The reader
crates/wt-schemaparses the DSL into data: columns with types, primary key,generators,
optionalandusing <backend>, plus indexes, theupdate/delete/in_placequeries,persist,version,partition_byandpage_size. That is WorkTable's whole grammar, not the subset this app happensto use, so the reader also works on a schema file it has never seen.
The grammar is mirrored rather than borrowed: WorkTable's parser lives in its
proc-macro crate and returns generated code, not a description of what it read,
and
synstops at the token stream inside the parentheses. Unknown sections arean error, because a reader that shrugs at what it does not understand describes
a schema that is quietly missing something.
Column comments come back too. They are
//, not///, so the lexer drops thembefore any parser sees them; they are recovered by line number. In this codebase
that prose is most of what a diagram has worth showing.
Where the edges come from
The DSL declares no foreign keys.
project_idis aStringlike every othercolumn, so the edges are proposed from naming and corrected by hand in
schema.overlay.json, and the two are drawn differently: solid for inferred,dashed for declared, faded for a reference the source documents as often empty.
Rendering them identically would claim a certainty nobody has.
Anything the convention cannot resolve is reported with suggestions rather than
dropped. The overlay currently closes all nine of those, and two of them are why
the mechanism exists at all:
item_completion.idandusage_session.idareprimary keys that are themselves references, which no naming rule can see.
Read-only, deliberately
WorkTable persists rows with rkyv, positionally, and nothing on disk records
which columns produced them, so a table whose columns change reads every
existing row through the new layout with no error anywhere. A designer that
writes a
worktable!has to driveSCHEMA_FINGERPRINTand the migration in thesame breath, or it is a data-loss tool with a nice diagram. This parses and
describes; it never writes Rust.
Verification
The parser is checked against
db/fingerprint.rs, which states every table'scolumn list in order and is maintained by hand rather than generated from these
files. Reconstructing it from the parse is therefore a real independent check: a
missed column, a reordered one or a skipped file each break it by name, and it
will also fail the day someone adds a column and forgets the fingerprint bump.
Green: 8 Rust tests, 6 layout tests,
tsc --noEmit, biome, and the i18n audit.Not verified: how it looks. No frontend
node_modulesexists in any checkouton this machine, so I borrowed one rather than installing anything. With it the
fixture app crashes at boot with
REACTIVITY_HALTED ... reading 'models'beforeany of this code runs. I reproduced the identical crash on
master, so it is notthis change, but I cannot tell whether it is a real master bug or an artifact of
the borrowed dependency set. The diagram wants a look, and ps-qa outcomes, before
anyone calls it finished.
Note
The diagram reads a committed JSON file rather than a Tauri command, so it
renders in the browser fixture with no build and no instance, and a schema change
reaches review as a diff of the drawing's input.
scripts/generate-schema.shrebuilds it. Biome is told to leave that file alone: its formatter is
serde_json's, so formatting it here would make every regeneration produce a
whitespace diff that the next
--writeundoes.