Skip to content

fix(search): stop silently returning zero results on large trees (v2.3.1) - #5

Merged
ytspar merged 1 commit into
mainfrom
fix/search-largetree-overflow
Jul 11, 2026
Merged

fix(search): stop silently returning zero results on large trees (v2.3.1)#5
ytspar merged 1 commit into
mainfrom
fix/search-largetree-overflow

Conversation

@ytspar

@ytspar ytspar commented Jul 11, 2026

Copy link
Copy Markdown
Owner

What

Fixes a silent-zero-results bug in search that made duplicate-literals scan nothing on a large tree, and ships it as v2.3.1.

The bug (found by using v2.3.0 on a real monorepo)

Pointing duplicate-literals at verticalint/tools' cli/ tree returned 0 findings / 0 files scanned. Root cause: ripgrep's --json output (one JSON object per submatch) overflows the 50MB execFileSync buffer on a big tree. The resulting ERR_CHILD_PROCESS_STDIO_MAXBUFFER error has no .status, so it fell straight through the err.status === 1 / === 2 checks to a final return []. A scan that matched thousands of files reported finding none — a hollow result that would silently no-op any drift gate built on top of it.

Fix

  • filesOnly discovery. search(..., { filesOnly: true }) uses rg -l / grep -rl (one path per line) instead of per-match JSON, so discovery stays bounded regardless of tree size. discoverFiles (the duplicate-literals discovery path) now uses it.
  • Never swallow an overflow. Both engines classify the overflow (isMaxBufferError) and throw an actionable error ("narrow the paths, or pass { filesOnly: true }") instead of returning []. Any other unexpected failure is thrown too — silent-zero is gone.
  • Raise the cap 50MB → 256MB as defense for full (non-discovery) match searches.

Verified

  • Before: duplicate-literals on cli/ → 0 files scanned. After: 1208 files scanned, 68 regex / 683 string findings surfaced.
  • New tests: filesOnly returns the same file SET as a full search (one row/file, no line/text), overflow-error classification, buffer-cap floor, parseFilesList shaping. 188/188 pass.

Why 2.3.1 now

The verticalint/tools DEV-5868 drift gate is built directly on this duplicate-literals scan; a hollow scan there means a gate that passes green while detecting nothing. Fixing at the tool layer ("strengthen the tool first") protects every consumer. Publishing is now tokenless via Trusted Publishing (OIDC), so merging this auto-publishes 2.3.1.

…3.1)

On a monorepo-scale tree, ripgrep's verbose --json output overflowed the
50MB execFileSync buffer. The resulting ERR_CHILD_PROCESS_STDIO_MAXBUFFER
error carries no .status, so it fell through the status checks to a
`return []` — a scan that matched thousands of files reported finding
NONE. This made `duplicate-literals` scan zero files on a large repo
(verified: 0 files on verticalint/tools' cli/ tree; 1208 after the fix)
— a hollow result that would silently no-op any drift gate built on it.

Fix:
- Add { filesOnly: true } to search: discovery uses `rg -l` / `grep -rl`
  (one path per line) instead of per-match JSON, so it stays bounded on
  any tree size. discoverFiles (the duplicate-literals discovery path)
  now uses it.
- Never swallow a buffer overflow: both engines classify it
  (isMaxBufferError) and THROW an actionable error instead of returning
  []; any other unexpected failure is thrown too, not silently zeroed.
- Raise the buffer cap 50MB -> 256MB as defense for full match searches.

Tests: filesOnly returns the same file SET as a full search (one row per
file, no line/text), overflow-error classification, buffer-cap floor,
parseFilesList shaping. 188/188 pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ytspar

ytspar commented Jul 11, 2026

Copy link
Copy Markdown
Owner Author

AI review (advisory) — cycle 1

Advisory only. GitHub blocks author self-approval, so this closes at handoff (your merge → auto-publishes v2.3.1 via OIDC).
Reviewed against: PR #5 HEAD (8324a1b)

Summary

Correct fix for a real hollow-scan bug (silent zero results on a large tree), found by running v2.3.0 on the actual tools monorepo. The fix is at the right layer and fails loud rather than silent — the important safety property for anything a drift gate sits on.

Verified

  • Root cause is exactly as described: ERR_CHILD_PROCESS_STDIO_MAXBUFFER has no .status, so it fell through to return []. Confirmed the isMaxBufferError classifier catches both the code and the message form, and that a real no-match (status === 1) is NOT misclassified as overflow (stays a legit empty result).
  • filesOnly discovery returns the same file SET as a full match search (proven by test), so wiring discoverFiles to it changes nothing about WHICH files are scanned — only the output volume.
  • End-to-end: duplicate-literals on cli/ went 0 → 1208 files scanned.
  • ci check green (jest + ripgrep); 188/188 including 5 new regression tests.

Findings

nit: MAX_STDOUT_BUFFER at 256MB is defense for full match searches; a truly pathological tree could still exceed it — but now it throws an actionable error instead of returning [], which is the correct behavior. No action.

No blockers.

Handoff

Merge → the main publish run sees 2.3.1 ≠ registry (2.3.0) and publishes tokenlessly. That unblocks the verticalint/tools DEV-5868 drift gate (design recorded on the issue; whole-tree scans are real now, so the gate can be built on a green-ratchet of duplicated-regex groups).

Verdict (8324a1b): APPROVE (handoff — merge is yours)

@ytspar
ytspar merged commit 835d6f5 into main Jul 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant