Skip to content

SRE-984: Move Turborepo from generated package.json to Cargo workspaces - #9758

Draft
TimDiekmann wants to merge 10 commits into
t/sre-1061-move-the-crates-packagejson-scripts-into-turbojson-commandsfrom
t/sre-984-move-turborepo-from-generated-packagejson-to-cargo
Draft

TimDiekmann wants to merge 10 commits into
t/sre-1061-move-the-crates-packagejson-scripts-into-turbojson-commandsfrom
t/sre-984-move-turborepo-from-generated-packagejson-to-cargo

Conversation

@TimDiekmann

@TimDiekmann TimDiekmann commented Sep 16, 2026

Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

Turborepo reads the Cargo workspace natively behind futureFlags.experimentalCargoWorkspaces. Every crate becomes a package under its Cargo.toml name, its dependencies follow Cargo.toml, and [workspace.metadata].name names the package that stands for the whole workspace. The generated package.json mirrors and sync-turborepo, which existed only to describe the crates to turbo, go away.

turbo prune also learns the Cargo workspace: it writes the pruned Cargo.toml, Cargo.lock and Cargo configuration itself and follows dev-dependencies. That retires the dummy crate manifests the Dockerfiles generated and most of the prune action's own scope computation.

🔗 Related links

🚫 Blocked by

🔍 What does this change?

  • Enables experimentalCargoWorkspaces; the root Cargo.toml names the workspace hash
  • Deletes the 52 crate package.json files and sync-turborepo: the repo-chores module and CLI subcommand, the mise task, the root script, the lint step, and the [package.metadata.sync.turborepo] sections
  • Refers to the crates by their Cargo names everywhere: the six TypeScript consumers' task edges, the workflows, the Dockerfile scopes, the crates' own turbo.json commands, the changeset config, and the docs
  • The four TypeScript packages and the hash-graph-integration crate that depended on @apps/hash-graph depend on hash-graph#start:test:healthcheck instead, which is what they need from it
  • The task documents cover the tasks a package implements — a package.json script or a command in a turbo.json — so turbo's built-in Cargo tasks (build, check, lint, test, format) stay out of them; the generator no longer needs packages { tasks }
  • The prune action calls turbo prune directly; prune.py and its closure over both graphs are gone. It merges the root dot-files into the pruned tree and copies the two Brunch directories no package owns; the Petrinaut docs arrive through a manifest edge on @apps/brunch-agent#test:unit
  • The Dockerfiles prune without stubbing workspace members. Turbo runs cargo metadata whenever it loads the graph, so the prune stages carry the Rust toolchain, as does the petrinaut-opt bindings stage
  • The five dependencyOutputs blocks are removed in their own commit while turbo's lazy native discovery rejects dependencyOutputs.from selectors that point at a crate outside the filter (reported in the discussion above); the commit is reverted once the fix ships
  • The nextest override for the retried harpc-net tests matches by test name: nextest validates package matchers against the workspace, and a pruned tree without the crate rejects them
  • The deploy catalog and the workflows name the graph image's package hash-graph
  • tests/graph/http is named @tests/hash-graph-http

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • require changes to docs which are made as part of this PR

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • affected the execution graph, and the turbo.json's have been updated to reflect this

⚠️ Known issues

  • Turbo registers build, check, lint, test and format for every crate (run and dev for binaries) with plain Cargo commands. Nothing runs them unfiltered in CI, and a crate's own command for a name wins over the built-in one, so they stay; opting out would cost five entries in each of the 52 crate configurations.
  • The prune action still copies libs/@hashintel/brunch-agent/docs and libs/@hashintel/brunch-agent/evaluations: @apps/brunch-agent reads them by path, but they sit next to the Brunch packages rather than inside one, so no edge can carry them into a pruned tree. Giving each directory a package.json the app depends on would turn them into ordinary edges; that is a Brunch change.
  • The Bench workflow prunes the base branch with the package names of the head. While the base is SRE-1061: Move the crates' package.json scripts into turbo.json commands #9755, hash-graph-benches does not exist there and the base benches fail; that resolves itself once the base carries this change.
  • The three Python packages keep their package.json mirrors; turbo's experimentalPythonWorkspaces needs a uv workspace, which is a separate step.

🐾 Next steps

  • Revert the dependencyOutputs removal once turbo loads crate scopes before validating dependencyOutputs.from
  • Move the Python packages onto experimentalPythonWorkspaces once they share a uv workspace

🛡 What tests cover this?

  • The prune action was replayed locally for hash-graph-postgres-store, @apps/hash-api, @apps/hash-frontend and @apps/brunch-agent: cargo metadata --locked loads every pruned workspace, the background tasks match today's. Every crate task resolves the same command as before; the task documents of the JavaScript packages change only in the crate names they reference.
  • CI on this branch: Lint, Test and Deploy (all eight images) pass on pruned trees.

❓ How to test this?

  1. Checkout the branch, mise install --locked npm:turbo
  2. turbo ls lists the crates under their Cargo names and hash for the workspace
  3. turbo prune hash-graph --docker --out-dir /tmp/prune && cd /tmp/prune/full && cargo metadata --locked succeeds without a stub in sight
  4. turbo run lint:clippy --filter error-stack runs just clippy from the crate's turbo.json

…ge.json files

Turborepo discovers the crates from the Cargo workspace behind `experimentalCargoWorkspaces`: every member is a package under its `Cargo.toml` name, its dependencies follow `Cargo.toml`, and `[workspace.metadata].name` names the package standing for the whole workspace. The 52 generated `package.json` files, `sync-turborepo` with its CLI, mise task, lint step and Cargo metadata sections, and the `@rust/` names go away; every reference uses the Cargo name.

The four TypeScript packages that depended on `@apps/hash-graph` and the `hash-graph-integration` crate reach the graph through `hash-graph#start:test:healthcheck` instead, which is what they need from it.

`turbo prune` writes the pruned `Cargo.toml`, `Cargo.lock` and Cargo configuration itself and follows dev-dependencies, so the Dockerfiles stop stubbing workspace members and `prune.py` drops its stubs and the rules that mirrored Cargo edges.

The task documents cover the tasks a package implements — a `package.json` script or a `command` in a `turbo.json` — so the tasks turbo's Cargo toolchain brings for every crate stay out of them.
`turbo prune` follows the package graph, the task graph and Cargo's dev-dependencies itself, so the closure and the rules `prune.py` computed over them are gone. The Petrinaut docs the Brunch product tests read arrive through a `manifest` edge on their package; what remains to copy are the two Brunch directories and the oxlint configuration that no package owns. The rule that added the atlas crate for the frontend had no reader left in the tree.
@TimDiekmann TimDiekmann self-assigned this Sep 16, 2026
@vercel

vercel Bot commented Sep 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

4 Skipped Deployments
Project Deployment Actions Updated
hash Ignored Ignored Preview Sep 16, 2026 2:50pm UTC
hashdotdesign-tokens Ignored Ignored Preview Sep 16, 2026 2:50pm UTC
petrinaut Skipped Skipped Sep 16, 2026 2:50pm UTC
petrinaut-docs Skipped Skipped Sep 16, 2026 2:50pm UTC

Request Review

@github-actions github-actions Bot added area/deps Relates to third-party dependencies (area) area/apps > hash* Affects HASH (a `hash-*` app) area/infra Relates to version control, CI, CD or IaC (area) area/apps > hash-api Affects the HASH API (app) area/libs > error-stack Affects the `error-stack` crate (library) area/libs Relates to first-party libraries/crates/packages (area) area/libs > antsi Affects the `antsi` crate (library) type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team area/tests New or updated tests area/tests > integration New or updated integration tests area/tests > playwright New or updated Playwright tests area/apps area/apps > hash-graph labels Sep 16, 2026
@TimDiekmann
TimDiekmann added this pull request to stack #9759 September 16, 2026 13:09
@github-advanced-security

Copy link
Copy Markdown
Contributor

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

Comment thread libs/@local/repo-chores/rust/src/task_dependencies/mod.rs Fixed
…opes before validating them

Under `experimentalCargoWorkspaces`, turbo 2.10.14-canary.3 validates a `dependencyOutputs.from` selector that names a crate task before it has loaded the crate's scope, so every package whose dependency closure holds such a block fails under a single `--filter`. The five blocks go until the fix lands; the `dependsOn` edges stay, so the crate tasks keep hashing into their consumers, and only the narrowing to the generated files is lost. Revert this commit once turbo resolves the selectors again.
@vercel
vercel Bot temporarily deployed to Preview – petrinaut-docs September 16, 2026 13:14 Inactive
@vercel
vercel Bot temporarily deployed to Preview – petrinaut September 16, 2026 13:14 Inactive
nextest rejects a `package()` matcher that names no package of the workspace, and a pruned workspace holds only the crates a job needs, so the CI profile matched nothing for every other crate and failed to parse. The two test paths exist only in `harpc-net`.
@codspeed

codspeed Bot commented Sep 16, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ 6 benchmarks measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. These results are not comparable, so they count as unchanged.

Preventing compiler optimizations

✅ 98 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚠️ as_constant < 1 ns < 1 ns N/A
⚠️ constant_equal < 1 ns < 1 ns N/A
⚠️ constant_not_equal < 1 ns < 1 ns N/A
⚠️ access < 1 ns < 1 ns N/A
⚠️ runtime_equal < 1 ns < 1 ns N/A
⚠️ runtime_not_equal < 1 ns < 1 ns N/A

Comparing t/sre-984-move-turborepo-from-generated-packagejson-to-cargo (77a8e70) with t/sre-1061-move-the-crates-packagejson-scripts-into-turbojson-commands (2cd552e)

Open in CodSpeed

@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.28%. Comparing base (2cd552e) to head (8fdc3b8).

Additional details and impacted files
@@                                           Coverage Diff                                           @@
##           t/sre-1061-move-the-crates-packagejson-scripts-into-turbojson-commands    #9758   +/-   ##
=======================================================================================================
  Coverage                                                                   66.28%   66.28%           
=======================================================================================================
  Files                                                                        1775     1775           
  Lines                                                                      191861   191861           
  Branches                                                                     7853     7853           
=======================================================================================================
+ Hits                                                                       127171   127174    +3     
+ Misses                                                                      63209    63206    -3     
  Partials                                                                     1481     1481           
Flag Coverage Δ
antsi 2.36% <ø> (?)
apps.hash-ai-worker-ts 1.99% <ø> (ø)
apps.hash-api 15.35% <ø> (ø)
blockprotocol.type-system 38.15% <ø> (ø)
error-stack 90.81% <ø> (?)
harpc-codec 84.70% <ø> (?)
harpc-net 96.24% <ø> (?)
harpc-tower 67.03% <ø> (?)
harpc-types 0.00% <ø> (?)
harpc-wire-protocol 92.23% <ø> (?)
hash-codec 72.76% <ø> (?)
hash-config 81.14% <ø> (?)
hash-graph 12.54% <ø> (?)
hash-graph-api 19.71% <ø> (?)
hash-graph-atlas 80.36% <ø> (?)
hash-graph-authentication 96.02% <ø> (?)
hash-graph-authorization 63.14% <ø> (?)
hash-graph-embeddings 91.88% <ø> (?)
hash-graph-postgres-store 32.15% <ø> (?)
hash-graph-store 48.41% <ø> (?)
hash-graph-temporal-versioning 50.18% <ø> (?)
hash-graph-types 0.00% <ø> (?)
hash-graph-validation 84.71% <ø> (?)
hash-middleware 90.92% <ø> (?)
hashql-ast 89.63% <ø> (?)
hashql-compiletest 28.39% <ø> (?)
hashql-core 78.92% <ø> (?)
hashql-diagnostics 72.51% <ø> (?)
hashql-eval 79.82% <ø> (?)
hashql-hir 89.09% <ø> (?)
hashql-mir 87.92% <ø> (?)
hashql-syntax-jexpr 94.04% <ø> (?)
local.claude-hooks 0.00% <ø> (ø)
local.harpc-client 51.49% <ø> (ø)
local.hash-backend-utils 3.27% <ø> (ø)
local.hash-graph-sdk 10.02% <ø> (ø)
local.hash-isomorphic-utils 12.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vercel
vercel Bot temporarily deployed to Preview – petrinaut September 16, 2026 14:48 Inactive
@vercel
vercel Bot temporarily deployed to Preview – petrinaut-docs September 16, 2026 14:48 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-api Affects the HASH API (app) area/apps > hash-graph area/apps area/deps Relates to third-party dependencies (area) area/infra Relates to version control, CI, CD or IaC (area) area/libs > antsi Affects the `antsi` crate (library) area/libs > error-stack Affects the `error-stack` crate (library) area/libs Relates to first-party libraries/crates/packages (area) area/tests > integration New or updated integration tests area/tests > playwright New or updated Playwright tests area/tests New or updated tests type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

2 participants