Two binaries, so an emitter in another language can be checked against this one - #91
Closed
pathscale wants to merge 1 commit into
Closed
Two binaries, so an emitter in another language can be checked against this one#91pathscale wants to merge 1 commit into
pathscale wants to merge 1 commit into
Conversation
pathscale
force-pushed
the
feat/dsl-emitter-binaries
branch
from
September 3, 2026 02:41
917ca8f to
2bf3b29
Compare
pathscale
pushed a commit
to pathscale/worktable-dsl-ts
that referenced
this pull request
Sep 3, 2026
`master` is WorkTable's default branch and `worktable_dsl` is not a member of its workspace: the crate lives on `release/beta17`. So every run failed with "package(s) `worktable_dsl` not found in workspace", which is the tests refusing to pass without the Rust side, working exactly as intended and pointed at the wrong tree. The ref is a repository variable with a default, so the day the release branch merges the pin moves in settings rather than in a commit. A WorkTable branch matching the pull request's name is preferred when one exists, because a change touching the grammar and this emitter is two pull requests and testing the new emitter against the old grammar tests nothing. CI stays red until pathscale/WorkTable#91 lands, since that is where the two binaries live. That is the honest state and not something to paper over.
…t this one `worktable-parse` reads a declaration on stdin and writes its canonical text to stdout. `worktable-schemas` walks a tree and prints every declaration it finds as JSON, with the canonical text beside each. Both exist for the same reason: two implementations of a language drift unless something compares them, and the comparison has to be byte-exact, because they can agree on every meaning and still disagree on every character a person reads. `@pathscale/worktable-dsl` is the first consumer. `worktable-parse` runs `check`, not `Schema::parse`, and that is the point. `parse` answers "is this a declaration"; `check` answers "would the macro accept it". `page_size: 4096` beside `persist: true` parses perfectly and the macro refuses it, so a parse-only binary reports green for output that does not compile. It caught two rules while the TypeScript corpus was being written: that one, and that a non-unique index cannot use `indexset` or `congee`. `worktable-schemas` is behind a new `json` feature rather than `serde`, because serialising the IR and shipping a JSON writer are different costs to ask of a dependent, and this crate is compiled for the host before anything else in one. Its output is sorted, so a consumer diffing two runs sees only real change.
pathscale
force-pushed
the
feat/dsl-emitter-binaries
branch
from
September 3, 2026 02:43
2bf3b29 to
f33a36e
Compare
pathscale
pushed a commit
that referenced
this pull request
Sep 3, 2026
Another agent built the same tool independently as #91, and got the central thing right where this branch had it wrong. `wt-dsl parse` used `Schema::parse`. It should have used `check`, and the difference is not academic: `parse` answers "is this a declaration", `check` answers "would the macro accept it". `page_size: 4096` beside `persist: true` parses perfectly and the macro refuses it, so the round trip reported success for output that does not compile. That is exactly the mistake a second implementation would then ship, and the mistake the cross-implementation check exists to catch. There is now a test on that case. Two more things taken from #91, both better than what was here: `worktable-schemas` dumps every declaration under a directory as JSON plus the canonical text, so the conformance corpus is the tables people actually wrote rather than the handful anyone thinks to invent. A hand-written corpus tests the cases its author already understood. Its feature wiring: `json` is separate from `serde`, because serialising the IR and shipping a JSON writer are different costs to ask of a dependent. And `wt-dsl` no longer requires `spans`. A conformance binary behind a feature is one a cross-implementation test can quietly be missing; byte ranges now appear when built with `spans` and are absent otherwise, degrading the message rather than the check.
Owner
Author
|
Folded into #87, and thank you: this got the central thing right where #87 had it wrong.
Also taken across, both better than what was on #87:
What #87 keeps beyond your two binaries: One PR per repo for this release, which is why this closes rather than stacks. The branch stays pushed. |
pathscale
pushed a commit
that referenced
this pull request
Sep 3, 2026
Another agent built the same tool independently as #91, and got the central thing right where this branch had it wrong. `wt-dsl parse` used `Schema::parse`. It should have used `check`, and the difference is not academic: `parse` answers "is this a declaration", `check` answers "would the macro accept it". `page_size: 4096` beside `persist: true` parses perfectly and the macro refuses it, so the round trip reported success for output that does not compile. That is exactly the mistake a second implementation would then ship, and the mistake the cross-implementation check exists to catch. There is now a test on that case. Two more things taken from #91, both better than what was here: `worktable-schemas` dumps every declaration under a directory as JSON plus the canonical text, so the conformance corpus is the tables people actually wrote rather than the handful anyone thinks to invent. A hand-written corpus tests the cases its author already understood. Its feature wiring: `json` is separate from `serde`, because serialising the IR and shipping a JSON writer are different costs to ask of a dependent. And `wt-dsl` no longer requires `spans`. A conformance binary behind a feature is one a cross-implementation test can quietly be missing; byte ranges now appear when built with `spans` and are absent otherwise, degrading the message rather than the check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
worktable-parsereads a declaration on stdin and writes its canonical text to stdout.worktable-schemaswalks a tree and prints every declaration it finds as JSON, with the canonical text beside each.Both exist for the same reason: two implementations of a language drift unless something compares them, and the comparison has to be byte-exact, because they can agree on every meaning and still disagree on every character a person reads.
@pathscale/worktable-dslis the first consumer, and its whole test suite is built on these.worktable-parserunscheck, notSchema::parseThat is the point of it.
parseanswers "is this a declaration";checkanswers "would the macro accept it", which is the question an emitter has to get right.page_size: 4096besidepersist: trueparses perfectly and the macro refuses it, so a parse-only binary would report green for output that does not compile.It earned that twice while the TypeScript corpus was being written — that rule, and that a non-unique index cannot use
indexsetorcongee. Both were fixture mistakes a parse-only check would have accepted.worktable-schemasis behind a newjsonfeatureNot
serde. Serialising the IR and shipping a JSON writer are different costs to ask of a dependent, and this crate is compiled for the host as part ofworktable_codegenbefore anything else in one. Output is sorted, so a consumer diffing two runs sees only real change.It finds 121 declarations in this repository with 0 rejected, which is what the TypeScript emitter is now checked against.
Also
Removes
dsl/src/bin/wt-dsl.rs, a stray empty file that was never committed and madecargo test -p worktable_dslfail to build witherror[E0601]: `main` function not found.Tests pass, clippy clean with
--all-targets --features json.