Skip to content

feat: read WorkTable's DSL and draw the store schema - #204

Open
pathscale wants to merge 3 commits into
masterfrom
feat/schema-visual-editor
Open

feat: read WorkTable's DSL and draw the store schema#204
pathscale wants to merge 3 commits into
masterfrom
feat/schema-visual-editor

Conversation

@pathscale

Copy link
Copy Markdown
Owner

Reads every worktable! in apps/gui/src/db/schema and draws it: a Schema tab
beside Analytics, with the tables laid out by reference depth so the two that
reference nothing sit in the first column and the fifteen that carry a
project_id fan out to the right.

The reader

crates/wt-schema parses the DSL into data: columns with types, primary key,
generators, optional and using <backend>, plus indexes, the
update/delete/in_place queries, persist, version, partition_by and
page_size. That is WorkTable's whole grammar, not the subset this app happens
to use, so the reader also works on a schema file it has never seen.

The grammar is mirrored rather than borrowed: WorkTable's parser lives in its
proc-macro crate and returns generated code, not a description of what it read,
and syn stops at the token stream inside the parentheses. Unknown sections are
an error, because a reader that shrugs at what it does not understand describes
a schema that is quietly missing something.

Column comments come back too. They are //, not ///, so the lexer drops them
before any parser sees them; they are recovered by line number. In this codebase
that prose is most of what a diagram has worth showing.

Where the edges come from

The DSL declares no foreign keys. project_id is a String like every other
column, so the edges are proposed from naming and corrected by hand in
schema.overlay.json, and the two are drawn differently: solid for inferred,
dashed for declared, faded for a reference the source documents as often empty.
Rendering them identically would claim a certainty nobody has.

Anything the convention cannot resolve is reported with suggestions rather than
dropped. The overlay currently closes all nine of those, and two of them are why
the mechanism exists at all: item_completion.id and usage_session.id are
primary keys that are themselves references, which no naming rule can see.

Read-only, deliberately

WorkTable persists rows with rkyv, positionally, and nothing on disk records
which columns produced them, so a table whose columns change reads every
existing row through the new layout with no error anywhere. A designer that
writes a worktable! has to drive SCHEMA_FINGERPRINT and the migration in the
same breath, or it is a data-loss tool with a nice diagram. This parses and
describes; it never writes Rust.

Verification

The parser is checked against db/fingerprint.rs, which states every table's
column list in order and is maintained by hand rather than generated from these
files. Reconstructing it from the parse is therefore a real independent check: a
missed column, a reordered one or a skipped file each break it by name, and it
will also fail the day someone adds a column and forgets the fingerprint bump.

Green: 8 Rust tests, 6 layout tests, tsc --noEmit, biome, and the i18n audit.

Not verified: how it looks. No frontend node_modules exists in any checkout
on this machine, so I borrowed one rather than installing anything. With it the
fixture app crashes at boot with REACTIVITY_HALTED ... reading 'models' before
any of this code runs. I reproduced the identical crash on master, so it is not
this change, but I cannot tell whether it is a real master bug or an artifact of
the borrowed dependency set. The diagram wants a look, and ps-qa outcomes, before
anyone calls it finished.

Note

The diagram reads a committed JSON file rather than a Tauri command, so it
renders in the browser fixture with no build and no instance, and a schema change
reaches review as a diff of the drawing's input. scripts/generate-schema.sh
rebuilds it. Biome is told to leave that file alone: its formatter is
serde_json's, so formatting it here would make every regeneration produce a
whitespace diff that the next --write undoes.

meh added 3 commits August 31, 2026 20:58
A reader crate for `worktable!` declarations: columns with their types,
primary key, generators and index backends, the indexes, the generated
queries, `persist`, `version`, `partition_by` and `page_size`.

The grammar is walked here rather than borrowed. WorkTable's own parser lives
in its proc-macro crate, which returns generated code rather than a
description of what it read, and `syn` stops at the token stream inside the
parentheses because a `macro_rules!` invocation has no typed AST. So the
grammar is mirrored in one file, kept small, and unknown sections are an
error: a reader that shrugs at what it does not understand describes a schema
that is quietly missing something.

Column comments come back too. They are `//`, not `///`, so the lexer drops
them before any parser sees them, and they are recovered by line number. In
this codebase that prose is most of what a diagram has worth showing.

The DSL declares no relationships, so edges are proposed from naming and every
one carries where it came from. What the convention cannot resolve is reported
with suggestions instead of dropped, and `schema.overlay.json` carries the
corrections a person makes, including the two edges inference structurally
cannot see, where the primary key itself is the reference.

Read-only, and it stays that way. Editing a `worktable!` means driving the
schema fingerprint and a migration in the same breath, because rows are
persisted positionally with rkyv and nothing on disk records which columns
wrote them.

The tests check the parse against `db/fingerprint.rs`, which states every
table's column list, in order, and is maintained by hand rather than generated
from these files. A missed column, a reordered one or a skipped file each
break it by name.
A Schema tab beside Analytics: every `worktable!` in `apps/gui/src/db/schema`
as a box of columns, laid out by reference depth, so the two tables that
reference nothing sit in the first column and everything that points at them
fans out to the right. Fifteen of seventeen tables carry a `project_id`, and a
drawing that does not commit to a direction turns that into a hairball.

Clicking a table opens what the source says about it: the module header, every
column with the comment above it, the indexes, the generated queries, and the
references in both directions. That prose is the reason the reader keeps `//`
comments at all, being the only text on the screen nobody invented for the
diagram.

Inferred and declared edges are drawn differently, dashed for the overlay,
because they are not the same claim. An inferred edge is a naming convention
holding up; an overlay edge is a person's decision. A faded edge is one the
source documents as often empty, which in a store with no nullable columns
means an empty string rather than an absent value.

The data is a committed JSON file, not a Tauri command. The diagram therefore
renders in the browser fixture on :3010 with no build and no instance, and a
schema change reaches review as a diff of the drawing's input rather than as
something only a running app can show. `scripts/generate-schema.sh` rebuilds
it, and biome leaves that file alone: its formatter is serde_json's, so
formatting it here would make every regeneration produce a whitespace diff
that the next `--write` undoes again.

Read-only, and the tab says so in its own module doc. Editing a `worktable!`
means driving `SCHEMA_FINGERPRINT` and a migration in the same breath.
The drawing is ps-qa's to judge, but everything that makes it readable is
arithmetic and can be checked here: no two boxes overlap in either density,
every relationship is routed, every edge runs right to left so direction means
something, and a compact box never hides a column an edge starts from.

Against the generated schema rather than a fixture, so a table added to the
store is a table these assertions cover.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant