[diskann-garnet] Optimize reranking

[metajack]

This addresses several performance issues with reranking.

1. The existence check turned out to be expensive as it consults the FSM. This reads 1 extra key per Lsearch which is unnecessary as we skip missing vectors anyway.
2. The full vector reads were serial. Using multiread allows Garnet to parallelize these.
3. The reranked candidates buffer was allocated in the post processor; pooling this allocation removes it from the query path.

Since we don't have good recall based tests yet, I hand verified the recall was still ok with this change.

[Copilot]

Pull request overview

Optimizes the diskann-garnet reranking path to reduce per-query overhead by removing an expensive existence check, switching full-vector reads to Garnet multi-read, and pooling the reranked-candidates buffer. Also bumps the diskann-garnet package version.

Changes:

• Remove per-candidate FSM existence checks during rerank and rely on multi-read skipping missing keys.
• Use read_multi_lpiid to parallelize full-vector reads for reranking.
• Add an ObjectPool for rerank scratch buffers and bump package versions to 4.0.4.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

File	Description
diskann-garnet/src/provider.rs	Rerank path rewritten to use multi-read and pooled rerank buffers; adds rerank pool plumbing.
diskann-garnet/diskann-garnet.nuspec	Bumps NuGet package version to 4.0.4.
diskann-garnet/Cargo.toml	Bumps crate version to 4.0.4.
Cargo.lock	Updates lockfile for the version bump.

Suppressed comments (2)

diskann-garnet/src/provider.rs:185

• Undef::new(RERANK_BUFFER_LENGTH) for a Vec<Neighbor<u32>> creates pooled vectors with len=1024 (filled with defaults) rather than reserving capacity. That means pool construction eagerly touches/initializes 1024 elements per pooled buffer, which is likely the opposite of the intended query-path optimization. Prefer pooling empty vecs and reserving capacity on first use (and reusing that capacity thereafter).

        let rerank_pool = ObjectPool::new(
            Undef::new(RERANK_BUFFER_LENGTH),
            parallelism,
            Some(parallelism),
        );

diskann-garnet/src/provider.rs:1306

• Using get_ref(Undef::new(RERANK_BUFFER_LENGTH)) forces the pooled rerank vec to be resized to len=1024 (default-initializing 1024 entries) and then immediately cleared. This adds a per-query O(1024) initialization cost and undermines the performance goal. Instead, request a zero-length vec from the pool and reserve the desired capacity (which will be retained across pool returns).

        let mut reranked = provider
            .rerank_pool
            .get_ref(Undef::new(RERANK_BUFFER_LENGTH));
        reranked.clear();

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (2)

diskann-garnet/src/provider.rs:1323

• rerank_pool.get_ref(Undef::new(RERANK_BUFFER_LENGTH)) forces the pooled Vec<Neighbor<_>> to be resized to length 1024 (default-filling elements) on every checkout, and then it’s immediately cleared. That adds avoidable per-query work and partially defeats the goal of moving rerank overhead off the query path. Consider checking out the pooled vec at length 0 and only reserving capacity as needed.

        let mut reranked = provider
            .rerank_pool
            .get_ref(Undef::new(RERANK_BUFFER_LENGTH));
        reranked.clear();

diskann-garnet/src/provider.rs:66

• The comment for RERANK_BUFFER_LENGTH says it is a starting capacity, but Undef::new(len) is a length-based initializer (it resizes and default-fills). Since this constant is used to size/allocate pooled buffers, it would be clearer to describe it as the target reserved capacity (or initial length if that remains the intent).

/// Starting capacity of the pre-allocated rerank buffers.
const RERANK_BUFFER_LENGTH: usize = 1024;

[codecov-commenter]

Codecov Report

❌ Patch coverage is 96.42857% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 91.47%. Comparing base (60c4ac0) to head (96415b3).

Files with missing lines	Patch %	Lines
diskann-garnet/src/provider.rs	96.42%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1308      +/-   ##
==========================================
- Coverage   91.47%   91.47%   -0.01%     
==========================================
  Files         516      516              
  Lines       98266    98276      +10     
==========================================
+ Hits        89888    89895       +7     
- Misses       8378     8381       +3     

Flag	Coverage Δ
miri	91.47% <96.42%> (-0.01%)	⬇️
unittests	91.14% <96.42%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
diskann-garnet/src/provider.rs	81.70% <96.42%> (+0.21%)	⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.