Description
Syncpack's file discovery walks the entire workspace tree, visiting every directory except node_modules and .git. It does not prune directories excluded by .gitignore, nor does it limit traversal to the directories implied by the workspace globs, even when explicit --source flags are passed.
In large monorepos this can dominate total runtime. In our workspace (~630 projects, pnpm workspaces, 120 package.json files), with a 1.4M-file jest cache directory at the workspace root (gitignored): syncpack lint took ~63 seconds. After removing that directory: ~4.5 seconds. In a minimal repro (3 packages), adding 40,000 gitignored junk files took syncpack lint from 0.05s to ~13.5s.
Additionally, on corporate machines like mine, endpoint protection (Microsoft Defender) intercepts the walker's per-file accesses, at ~0.3ms per not-yet-cached file. When there are more files on the walk, this adds up over time, making the walk even slower. Since build/test caches regenerate files constantly, the cost recurs on every run.
Reproduced on 14.3.0 and 15.3.2. The 15.2.0 fix for #334 corrected which files appear in results, but the walk itself still visits everything. Syncpack seems the right place to fix this because no CLI flag can currently avoid the traversal (--source filters results, not the walk).
Suggested Solution
No CLI or config surface change strictly needed, ideally discovery just gets faster.
- Prune traversal using
.gitignore (e.g. the ignore crate's WalkBuilder, which handles nested gitignores natively), the same way ripgrep does
- Only descend into directories that can match the workspace globs / source patterns (e.g.
packages: ["apps/*", "libs/*"] never requires entering ./.jestcache/** or ./dist/**).
If implicit gitignore behavior is undesirable, an opt-in config would also solve it:
{
"ignorePaths": [".jestcache/**", "dist/**", "coverage/**"]
}Expected outcome: syncpack lint output unchanged, but runtime proportional to the number of candidate package files rather than total files in the repo.
Optional comments
No response
Code of Conduct
Description
Syncpack's file discovery walks the entire workspace tree, visiting every directory except
node_modulesand.git. It does not prune directories excluded by.gitignore, nor does it limit traversal to the directories implied by the workspace globs, even when explicit--sourceflags are passed.In large monorepos this can dominate total runtime. In our workspace (~630 projects, pnpm workspaces, 120 package.json files), with a 1.4M-file jest cache directory at the workspace root (gitignored): syncpack lint took ~63 seconds. After removing that directory: ~4.5 seconds. In a minimal repro (3 packages), adding 40,000 gitignored junk files took syncpack lint from 0.05s to ~13.5s.
Additionally, on corporate machines like mine, endpoint protection (Microsoft Defender) intercepts the walker's per-file accesses, at ~0.3ms per not-yet-cached file. When there are more files on the walk, this adds up over time, making the walk even slower. Since build/test caches regenerate files constantly, the cost recurs on every run.
Reproduced on 14.3.0 and 15.3.2. The 15.2.0 fix for #334 corrected which files appear in results, but the walk itself still visits everything. Syncpack seems the right place to fix this because no CLI flag can currently avoid the traversal (
--sourcefilters results, not the walk).Suggested Solution
No CLI or config surface change strictly needed, ideally discovery just gets faster.
.gitignore(e.g. the ignore crate's WalkBuilder, which handles nested gitignores natively), the same way ripgrep doespackages: ["apps/*", "libs/*"]never requires entering./.jestcache/**or./dist/**).If implicit gitignore behavior is undesirable, an opt-in config would also solve it:
{ "ignorePaths": [".jestcache/**", "dist/**", "coverage/**"] }Expected outcome:
syncpack lintoutput unchanged, but runtime proportional to the number of candidate package files rather than total files in the repo.Optional comments
No response
Code of Conduct