SRE-984: Move Turborepo from generated package.json to Cargo workspaces - #9758
Conversation
…ge.json files Turborepo discovers the crates from the Cargo workspace behind `experimentalCargoWorkspaces`: every member is a package under its `Cargo.toml` name, its dependencies follow `Cargo.toml`, and `[workspace.metadata].name` names the package standing for the whole workspace. The 52 generated `package.json` files, `sync-turborepo` with its CLI, mise task, lint step and Cargo metadata sections, and the `@rust/` names go away; every reference uses the Cargo name. The four TypeScript packages that depended on `@apps/hash-graph` and the `hash-graph-integration` crate reach the graph through `hash-graph#start:test:healthcheck` instead, which is what they need from it. `turbo prune` writes the pruned `Cargo.toml`, `Cargo.lock` and Cargo configuration itself and follows dev-dependencies, so the Dockerfiles stop stubbing workspace members and `prune.py` drops its stubs and the rules that mirrored Cargo edges. The task documents cover the tasks a package implements — a `package.json` script or a `command` in a `turbo.json` — so the tasks turbo's Cargo toolchain brings for every crate stay out of them.
`turbo prune` follows the package graph, the task graph and Cargo's dev-dependencies itself, so the closure and the rules `prune.py` computed over them are gone. The Petrinaut docs the Brunch product tests read arrive through a `manifest` edge on their package; what remains to copy are the two Brunch directories and the oxlint configuration that no package owns. The rule that added the atlas crate for the frontend had no reader left in the tree.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 4 Skipped Deployments
|
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
…opes before validating them Under `experimentalCargoWorkspaces`, turbo 2.10.14-canary.3 validates a `dependencyOutputs.from` selector that names a crate task before it has loaded the crate's scope, so every package whose dependency closure holds such a block fails under a single `--filter`. The five blocks go until the fix lands; the `dependsOn` edges stay, so the crate tasks keep hashing into their consumers, and only the narrowing to the generated files is lost. Revert this commit once turbo resolves the selectors again.
nextest rejects a `package()` matcher that names no package of the workspace, and a pruned workspace holds only the crates a job needs, so the CI profile matched nothing for every other crate and failed to parse. The two test paths exist only in `harpc-net`.
Merging this PR will not alter performance
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
as_constant |
< 1 ns | < 1 ns | N/A | |
constant_equal |
< 1 ns | < 1 ns | N/A | |
constant_not_equal |
< 1 ns | < 1 ns | N/A | |
access |
< 1 ns | < 1 ns | N/A | |
runtime_equal |
< 1 ns | < 1 ns | N/A | |
runtime_not_equal |
< 1 ns | < 1 ns | N/A |
Comparing t/sre-984-move-turborepo-from-generated-packagejson-to-cargo (77a8e70) with t/sre-1061-move-the-crates-packagejson-scripts-into-turbojson-commands (2cd552e)
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## t/sre-1061-move-the-crates-packagejson-scripts-into-turbojson-commands #9758 +/- ##
=======================================================================================================
Coverage 66.28% 66.28%
=======================================================================================================
Files 1775 1775
Lines 191861 191861
Branches 7853 7853
=======================================================================================================
+ Hits 127171 127174 +3
+ Misses 63209 63206 -3
Partials 1481 1481 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
🌟 What is the purpose of this PR?
Turborepo reads the Cargo workspace natively behind
futureFlags.experimentalCargoWorkspaces. Every crate becomes a package under itsCargo.tomlname, its dependencies followCargo.toml, and[workspace.metadata].namenames the package that stands for the whole workspace. The generatedpackage.jsonmirrors andsync-turborepo, which existed only to describe the crates to turbo, go away.turbo prunealso learns the Cargo workspace: it writes the prunedCargo.toml,Cargo.lockand Cargo configuration itself and follows dev-dependencies. That retires the dummy crate manifests the Dockerfiles generated and most of the prune action's own scope computation.🔗 Related links
dependencyOutputsissue below🚫 Blocked by
🔍 What does this change?
experimentalCargoWorkspaces; the rootCargo.tomlnames the workspacehashpackage.jsonfiles andsync-turborepo: the repo-chores module and CLI subcommand, the mise task, the root script, the lint step, and the[package.metadata.sync.turborepo]sectionsturbo.jsoncommands, the changeset config, and the docshash-graph-integrationcrate that depended on@apps/hash-graphdepend onhash-graph#start:test:healthcheckinstead, which is what they need from itpackage.jsonscript or acommandin aturbo.json— so turbo's built-in Cargo tasks (build,check,lint,test,format) stay out of them; the generator no longer needspackages { tasks }turbo prunedirectly;prune.pyand its closure over both graphs are gone. It merges the root dot-files into the pruned tree and copies the two Brunch directories no package owns; the Petrinaut docs arrive through amanifestedge on@apps/brunch-agent#test:unitcargo metadatawhenever it loads the graph, so the prune stages carry the Rust toolchain, as does thepetrinaut-optbindings stagedependencyOutputsblocks are removed in their own commit while turbo's lazy native discovery rejectsdependencyOutputs.fromselectors that point at a crate outside the filter (reported in the discussion above); the commit is reverted once the fix shipsharpc-nettests matches by test name: nextest validates package matchers against the workspace, and a pruned tree without the crate rejects themhash-graphtests/graph/httpis named@tests/hash-graph-httpPre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
turbo.json's have been updated to reflect thisbuild,check,lint,testandformatfor every crate (runanddevfor binaries) with plain Cargo commands. Nothing runs them unfiltered in CI, and a crate's owncommandfor a name wins over the built-in one, so they stay; opting out would cost five entries in each of the 52 crate configurations.libs/@hashintel/brunch-agent/docsandlibs/@hashintel/brunch-agent/evaluations:@apps/brunch-agentreads them by path, but they sit next to the Brunch packages rather than inside one, so no edge can carry them into a pruned tree. Giving each directory apackage.jsonthe app depends on would turn them into ordinary edges; that is a Brunch change.hash-graph-benchesdoes not exist there and the base benches fail; that resolves itself once the base carries this change.package.jsonmirrors; turbo'sexperimentalPythonWorkspacesneeds a uv workspace, which is a separate step.🐾 Next steps
dependencyOutputsremoval once turbo loads crate scopes before validatingdependencyOutputs.fromexperimentalPythonWorkspacesonce they share a uv workspace🛡 What tests cover this?
hash-graph-postgres-store,@apps/hash-api,@apps/hash-frontendand@apps/brunch-agent:cargo metadata --lockedloads every pruned workspace, the background tasks match today's. Every crate task resolves the same command as before; the task documents of the JavaScript packages change only in the crate names they reference.❓ How to test this?
mise install --locked npm:turboturbo lslists the crates under their Cargo names andhashfor the workspaceturbo prune hash-graph --docker --out-dir /tmp/prune && cd /tmp/prune/full && cargo metadata --lockedsucceeds without a stub in sightturbo run lint:clippy --filter error-stackrunsjust clippyfrom the crate'sturbo.json