Skip to content

Two binaries, so an emitter in another language can be checked against this one - #91

Closed
pathscale wants to merge 1 commit into
release/beta17from
feat/dsl-emitter-binaries
Closed

Two binaries, so an emitter in another language can be checked against this one#91
pathscale wants to merge 1 commit into
release/beta17from
feat/dsl-emitter-binaries

Conversation

@pathscale

Copy link
Copy Markdown
Owner

worktable-parse reads a declaration on stdin and writes its canonical text to stdout. worktable-schemas walks a tree and prints every declaration it finds as JSON, with the canonical text beside each.

Both exist for the same reason: two implementations of a language drift unless something compares them, and the comparison has to be byte-exact, because they can agree on every meaning and still disagree on every character a person reads. @pathscale/worktable-dsl is the first consumer, and its whole test suite is built on these.

worktable-parse runs check, not Schema::parse

That is the point of it. parse answers "is this a declaration"; check answers "would the macro accept it", which is the question an emitter has to get right. page_size: 4096 beside persist: true parses perfectly and the macro refuses it, so a parse-only binary would report green for output that does not compile.

It earned that twice while the TypeScript corpus was being written — that rule, and that a non-unique index cannot use indexset or congee. Both were fixture mistakes a parse-only check would have accepted.

worktable-schemas is behind a new json feature

Not serde. Serialising the IR and shipping a JSON writer are different costs to ask of a dependent, and this crate is compiled for the host as part of worktable_codegen before anything else in one. Output is sorted, so a consumer diffing two runs sees only real change.

It finds 121 declarations in this repository with 0 rejected, which is what the TypeScript emitter is now checked against.

Also

Removes dsl/src/bin/wt-dsl.rs, a stray empty file that was never committed and made cargo test -p worktable_dsl fail to build with error[E0601]: `main` function not found.

Tests pass, clippy clean with --all-targets --features json.

@pathscale
pathscale changed the base branch from master to release/beta17 September 3, 2026 02:39
@pathscale
pathscale force-pushed the feat/dsl-emitter-binaries branch from 917ca8f to 2bf3b29 Compare September 3, 2026 02:41
pathscale pushed a commit to pathscale/worktable-dsl-ts that referenced this pull request Sep 3, 2026
`master` is WorkTable's default branch and `worktable_dsl` is not a member of
its workspace: the crate lives on `release/beta17`. So every run failed with
"package(s) `worktable_dsl` not found in workspace", which is the tests refusing
to pass without the Rust side, working exactly as intended and pointed at the
wrong tree.

The ref is a repository variable with a default, so the day the release branch
merges the pin moves in settings rather than in a commit. A WorkTable branch
matching the pull request's name is preferred when one exists, because a change
touching the grammar and this emitter is two pull requests and testing the new
emitter against the old grammar tests nothing.

CI stays red until pathscale/WorkTable#91 lands, since that is where the two
binaries live. That is the honest state and not something to paper over.
…t this one

`worktable-parse` reads a declaration on stdin and writes its canonical text to
stdout. `worktable-schemas` walks a tree and prints every declaration it finds
as JSON, with the canonical text beside each.

Both exist for the same reason: two implementations of a language drift unless
something compares them, and the comparison has to be byte-exact, because they
can agree on every meaning and still disagree on every character a person reads.
`@pathscale/worktable-dsl` is the first consumer.

`worktable-parse` runs `check`, not `Schema::parse`, and that is the point.
`parse` answers "is this a declaration"; `check` answers "would the macro accept
it". `page_size: 4096` beside `persist: true` parses perfectly and the macro
refuses it, so a parse-only binary reports green for output that does not
compile. It caught two rules while the TypeScript corpus was being written: that
one, and that a non-unique index cannot use `indexset` or `congee`.

`worktable-schemas` is behind a new `json` feature rather than `serde`, because
serialising the IR and shipping a JSON writer are different costs to ask of a
dependent, and this crate is compiled for the host before anything else in one.
Its output is sorted, so a consumer diffing two runs sees only real change.
@pathscale
pathscale force-pushed the feat/dsl-emitter-binaries branch from 2bf3b29 to f33a36e Compare September 3, 2026 02:43
pathscale pushed a commit that referenced this pull request Sep 3, 2026
Another agent built the same tool independently as #91, and got the central
thing right where this branch had it wrong.

`wt-dsl parse` used `Schema::parse`. It should have used `check`, and the
difference is not academic: `parse` answers "is this a declaration", `check`
answers "would the macro accept it". `page_size: 4096` beside `persist: true`
parses perfectly and the macro refuses it, so the round trip reported success
for output that does not compile. That is exactly the mistake a second
implementation would then ship, and the mistake the cross-implementation check
exists to catch. There is now a test on that case.

Two more things taken from #91, both better than what was here:

`worktable-schemas` dumps every declaration under a directory as JSON plus the
canonical text, so the conformance corpus is the tables people actually wrote
rather than the handful anyone thinks to invent. A hand-written corpus tests
the cases its author already understood.

Its feature wiring: `json` is separate from `serde`, because serialising the IR
and shipping a JSON writer are different costs to ask of a dependent. And
`wt-dsl` no longer requires `spans`. A conformance binary behind a feature is
one a cross-implementation test can quietly be missing; byte ranges now appear
when built with `spans` and are absent otherwise, degrading the message rather
than the check.
@pathscale

Copy link
Copy Markdown
Owner Author

Folded into #87, and thank you: this got the central thing right where #87 had it wrong.

wt-dsl parse there was built on Schema::parse. Your reasoning for check is correct and I had missed it: page_size: 4096 beside persist: true parses perfectly and the macro refuses it, so the round trip reported success for output that does not compile. That is exactly what a second implementation would then ship. Fixed, with a test on that case.

Also taken across, both better than what was on #87:

  • worktable-schemas, so the conformance corpus is the tables people actually wrote rather than a hand-written one that only covers what its author understood.
  • The feature wiring. json separate from serde, and the point that a conformance binary behind a feature is one a cross-implementation test can quietly be missing, which is why wt-dsl no longer requires spans.

What #87 keeps beyond your two binaries: check, scan and diff subcommands, and tests on the CLI contract.

One PR per repo for this release, which is why this closes rather than stacks. The branch stays pushed.

@pathscale pathscale closed this Sep 3, 2026
pathscale pushed a commit that referenced this pull request Sep 3, 2026
Another agent built the same tool independently as #91, and got the central
thing right where this branch had it wrong.

`wt-dsl parse` used `Schema::parse`. It should have used `check`, and the
difference is not academic: `parse` answers "is this a declaration", `check`
answers "would the macro accept it". `page_size: 4096` beside `persist: true`
parses perfectly and the macro refuses it, so the round trip reported success
for output that does not compile. That is exactly the mistake a second
implementation would then ship, and the mistake the cross-implementation check
exists to catch. There is now a test on that case.

Two more things taken from #91, both better than what was here:

`worktable-schemas` dumps every declaration under a directory as JSON plus the
canonical text, so the conformance corpus is the tables people actually wrote
rather than the handful anyone thinks to invent. A hand-written corpus tests
the cases its author already understood.

Its feature wiring: `json` is separate from `serde`, because serialising the IR
and shipping a JSON writer are different costs to ask of a dependent. And
`wt-dsl` no longer requires `spans`. A conformance binary behind a feature is
one a cross-implementation test can quietly be missing; byte ranges now appear
when built with `spans` and are absent otherwise, degrading the message rather
than the check.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant