Skip to content

test(vis): fuzz untrusted lockfile parsers - #32

Merged
prisis merged 1 commit into
mainfrom
test/fuzz-lockfile-parsers
Oct 17, 2026
Merged

prisis merged 1 commit into
mainfrom
test/fuzz-lockfile-parsers

Conversation

@prisis

@prisis prisis commented Oct 17, 2026

Copy link
Copy Markdown
Member

Summary

Adds a seeded mutation test for the untrusted-input lockfile parsers behind `lockedPackagesForEcosystem`. Each run takes a valid sample for a lockfile format, mutates it 300 times, and asserts the parser never throws and returns entries with non-empty string names and versions.

Covered formats: Cargo.lock / poetry.lock / uv.lock (TOML), Gemfile.lock, go.sum, mix.lock, pubspec.lock, composer.lock, Pipfile.lock, packages.lock.json, Package.resolved, pom.xml, and gradle.lockfile (Maven).

Design

  • Deterministic: a Lehmer generator with a fixed seed (`RNG_SEED`). Each case first checks that its sample parses to at least one package, so the loop can't pass vacuously.
  • Failures report the iteration index and the exact input, so a failure replays from the seed.
  • Mutations are mostly deletes and inserts, with rare truncation, so most inputs still reach the post-parse code instead of failing at parse time.

Limits

  • This is seeded mutation testing, not coverage-guided fuzzing (no cargo-fuzz or libFuzzer).
  • npm, pnpm, yarn and bun lockfiles are parsed by `@visulima/package` and are not covered here.
  • The Rust native crate is not covered.
  • `poetry.lock` has no sample of its own; it shares the TOML parser with Cargo and uv.
  • For `pom.xml`, `readPom` catches scanner exceptions itself, so a crash in the XML scanner is swallowed before this test can see it.

Test plan

  • `npx vitest run tests/security/multi-eco-lockfiles.fuzz.test.ts` (12/12)
  • eslint and prettier clean on the new file

🤖 Generated with Claude Code

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for following the naming conventions! 🙏

@github-actions github-actions Bot added vouch:trusted PR author is a collaborator, a bot or on the VOUCHED list. size:L Size of the change, tests and generated files excluded. labels Oct 17, 2026
@pkg-pr-new

pkg-pr-new Bot commented Oct 17, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@visulima/vis@32
npm i https://pkg.pr.new/@visulima/vis-mcp@32

commit: 93f1062

@prisis
prisis merged commit c71b539 into main Oct 17, 2026
30 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

🌟 Great job, @prisis! You've merged your 12th PR! 🎊

Your contributions to vis are making a real difference. Keep up the fantastic work! 💪
Remember, every PR counts and helps improve the project. What will you tackle next? 🤔

@prisis is here if you need any guidance. Onward and upward! 🚀

@codspeed

codspeed Bot commented Oct 17, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ 9 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

✅ 12 untouched benchmarks
⏩ 101 skipped benchmarks1


Comparing test/fuzz-lockfile-parsers (93f1062) with main (80d5a91)2

Open in CodSpeed

Footnotes

  1. 101 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on main (21ac5b8) during the generation of this report, so 80d5a91 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

package: vis size:L Size of the change, tests and generated files excluded. vouch:trusted PR author is a collaborator, a bot or on the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant