feat(search): default to potion-code-16M-v2, invalidate stale-model caches#77
Conversation
…aches
Port semble#219: bump the default Model2Vec model to
`minishlab/potion-code-16M-v2` and drop chunks with a 0.0 fused score
before ranking.
- dense.rs: DEFAULT_MODEL_NAME -> potion-code-16M-v2 (+ doc/comments).
- index.rs: try_reuse now compares manifest.model_id against the
resolved model name, so a cache built with a different model (e.g. the
old v1 default) is treated as stale and rebuilt — its persisted
vectors are incompatible with v2-embedded queries. New test
try_reuse_rejects_stale_model.
- search.rs: filter combined scores == 0.0 before rerank, matching
upstream `combined_scores = {... if score}`; strengthen
search_alpha_one_is_semantic to assert the BM25-only chunk is dropped.
- README.md / README.ko.md / references/semble.md: model name sync.
Refs #73
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 79 |
| Duplication | 6 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the default Model2Vec model to potion-code-16M-v2 across documentation, code, and tests. It implements cache invalidation upon model changes by checking the model ID in the index manifest, and filters out search results with a fused score of exactly 0.0 before ranking. The reviewer flagged that index.rs, search.rs, and dense.rs exceed the organization's 500 LOC limit, recommending that these files be split by responsibility.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
No issues found across 6 files
Architecture diagram
sequenceDiagram
participant CLI as CLI / Search Request
participant LoadIndex as load_or_build_index()
participant TryReuse as try_reuse()
participant Manifest as Manifest File (cache)
participant Model as Dense Model (potion-code-16M-v2)
participant BM25 as BM25 Index
participant Search as search() / fused scoring
participant Rerank as rerank / filter zero scores
Note over CLI,Rerank: CURRENT-STATE FLOW (after PR)
CLI->>LoadIndex: search(query, source, options)
LoadIndex->>TryReuse: try_reuse(cache_dir, source_hash, expected_model)
alt Cache exists & manifest.model_id == expected_model
TryReuse->>Manifest: read manifest.json
Manifest-->>TryReuse: { model_id, chunk_size, content_hash }
Note over TryReuse: Same model → vectors compatible
TryReuse-->>LoadIndex: cached CspIndex (load from disk)
LoadIndex-->>CLI: loaded index
else Cache missing or model_id mismatch (e.g., old v1 cache)
TryReuse-->>LoadIndex: None (stale/missing)
LoadIndex->>Model: load_model(expected_model) // NEW: v2 from HF
Model-->>LoadIndex: static embeddings model
LoadIndex->>LoadIndex: compute embeddings for all chunks
LoadIndex->>BM25: build BM25 index
LoadIndex->>LoadIndex: save new manifest with model_id = expected_model
LoadIndex-->>CLI: new index
end
CLI->>Search: search(query, index, alpha)
Search->>Search: compute semantic scores (model)
Search->>Search: compute BM25 scores
Search->>Search: fused score = alpha*sem + (1-alpha)*bm25
Search->>Search: CHANGED: retain only scores != 0.0
Note over Search: Drops chunks with exactly 0.0 (e.g., alpha=1.0, BM25-only)
Search->>Rerank: rank with optional rerank
Rerank-->>Search: ranked results
Search-->>CLI: search results
Greptile SummaryThis PR upgrades the default embedding model to
Confidence Score: 5/5Safe to merge. All changed paths are covered by new or updated tests, the cache validation logic is conservative, and the zero-score filter only activates at the exact alpha boundaries where it is intentional. The model bump and cache-invalidation logic are tightly scoped and well-tested: try_reuse validates model_id, model_kind, and chunk_size before reuse, and the fallback is a clean rebuild with no data loss path. The zero-score retain is a mathematically exact comparison safe against floating-point accumulation errors because RRF denominators are always >= 61. Backend hardening addresses real edge cases with targeted tests. The public API surface is unchanged. No files require special attention. All touched modules have corresponding tests and the changes are well-bounded. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[load_or_build_index] --> B[resolve_cache_dir]
B --> C[compute_content_hash_from_paths]
C --> D{try_reuse}
D --> E{manifest.json exists?}
E -- No --> R[rebuild]
E -- Yes --> F{chunk_size matches?}
F -- No --> R
F -- Yes --> G{model_id matches?}
G -- No --> R
G -- Yes --> H[load_model to get kind]
H --> I{model_kind matches?}
I -- No --> R
I -- Yes --> J{content hash matches?}
J -- No --> R
J -- Yes --> K[CspIndex::load_from_disk]
K --> L[Cache reused]
R --> M[CspIndex::from_path or from_git]
M --> N[index.save with model_id + model_kind]
N --> O[Fresh index]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[load_or_build_index] --> B[resolve_cache_dir]
B --> C[compute_content_hash_from_paths]
C --> D{try_reuse}
D --> E{manifest.json exists?}
E -- No --> R[rebuild]
E -- Yes --> F{chunk_size matches?}
F -- No --> R
F -- Yes --> G{model_id matches?}
G -- No --> R
G -- Yes --> H[load_model to get kind]
H --> I{model_kind matches?}
I -- No --> R
I -- Yes --> J{content hash matches?}
J -- No --> R
J -- Yes --> K[CspIndex::load_from_disk]
K --> L[Cache reused]
R --> M[CspIndex::from_path or from_git]
M --> N[index.save with model_id + model_kind]
N --> O[Fresh index]
Reviews (4): Last reviewed commit: "test: strengthen cache and reranking cov..." | Re-trigger Greptile |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request refactors the indexing and search modules of the csp crate. Key changes include updating the default Model2Vec model to minishlab/potion-code-16M-v2, extracting the cache orchestration logic into a dedicated cache_orchestrator module, splitting dense backend and search tests into separate files, and filtering out chunks with a fused score of 0.0 during search. The reviewer feedback suggests two improvements: safely handling missing git commands in tests to prevent panics in environments without git, and optimizing the vector loading loop in SelectableBasicBackend::load using chunks_exact(4) to eliminate bounds checks and improve performance.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request updates the default Model2Vec model to minishlab/potion-code-16M-v2 for parity with semble#219, refactors the indexing module by extracting cache orchestration and dense vector backend logic into separate files, and drops chunks with a fused score of exactly 0.0 before ranking. A medium-severity issue was identified in the dense backend where calculating the expected vector file size could result in an integer overflow, and using checked multiplication is recommended to improve robustness.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request refactors the indexing and search modules of the csp crate by extracting cache orchestration into cache_orchestrator.rs, moving SelectableBasicBackend to dense/backend.rs, and organizing tests into separate files. It also updates the default Model2Vec model to minishlab/potion-code-16M-v2 and filters out chunks with a fused score of 0.0 before ranking. Feedback is provided regarding a high-severity Denial of Service (DoS) vulnerability in dense/backend.rs where a zero-dimension vector with a non-zero row count can trigger an Out Of Memory (OOM) panic during allocation.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request updates the default Model2Vec model to minishlab/potion-code-16M-v2 (aligning with semble#219), modularizes the codebase by extracting dense backend and search tests into dedicated files, and introduces the cache_orchestrator module to handle index build-or-reuse logic. Additionally, search logic is updated to drop chunks with a fused score of exactly 0.0 before ranking. Feedback was provided regarding a potential dimension distortion issue when loading an empty index in SelectableBasicBackend::load, as well as a memory bloat risk in collect_source_files when reading all file contents into memory simultaneously.
|
/gemini review |
There was a problem hiding this comment.
Code Review
이번 Pull Request는 csp 인덱싱의 글로벌 온디스크 캐시 오케스트레이션을 포팅하고, 기본 모델을 minishlab/potion-code-16M-v2로 업데이트하며, 대규모 리포지토리에서 메모리 점유를 줄이기 위해 파일 단위로 스트리밍하여 콘텐츠 해시를 계산하도록 개선했습니다. 또한 벡터 백엔드 및 검색 테스트 코드를 별도 파일로 분리하여 모듈화했습니다. 리뷰 피드백으로는 Windows와 Unix/Linux 환경 간의 경로 구분자(백슬래시와 슬래시) 차이로 인해 플랫폼 간 콘텐츠 해시가 일관되지 않게 생성되어 캐시가 무효화될 수 있는 문제가 지적되었으며, 경로 구분자를 슬래시로 통일할 것을 권장했습니다.
|
/gemini review |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 89 |
| Duplication | 10 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
There was a problem hiding this comment.
Code Review
이번 풀 리퀘스트는 기본 Model2Vec 임베딩 모델을 minishlab/potion-code-16M-v2로 업그레이드하고, 대용량 리포지토리에서 메모리 효율성을 높이기 위해 스트리밍 방식의 파일 해싱(compute_content_hash_from_paths)을 도입하며, 온디스크 인덱스 캐시 빌드 및 재사용 오케스트레이터(cache_orchestrator.rs)를 추가하는 변경사항을 담고 있습니다. 또한 검색 점수가 0.0인 청크를 필터링하는 로직이 추가되었습니다. 리뷰 피드백에서는 오프라인 상태에서 실제 모델 로드가 실패하여 스텁 모델로 폴백되었을 때, 실제 모델로 빌드된 기존 캐시를 그대로 재사용함으로써 검색 결과가 왜곡되는 문제를 지적했습니다. 이를 해결하기 위해 try_reuse 함수에서 스텁 모델과 기대 모델 간의 불일치를 검증하는 필터링 로직을 추가할 것을 제안합니다.
|
/gemini review |
There was a problem hiding this comment.
Code Review
이번 풀 리퀘스트는 기본 임베딩 모델을 minishlab/potion-code-16M-v2로 업데이트하고, 대규모 저장소에서 메모리 효율성을 높이기 위해 파일 콘텐츠 해시를 스트리밍 방식으로 계산하는 기능을 도입했습니다. 또한 캐시 오케스트레이션 로직을 cache_orchestrator 모듈로 분리하고, SelectableBasicBackend 및 관련 테스트 코드를 구조화된 서브모듈로 정리했습니다. 검색 로직에서는 융합 점수가 0.0인 청크를 필터링하는 변경사항이 포함되었습니다. 이번 변경사항에 대해 검토를 진행하였으며, 별도의 피드백은 없습니다.
There was a problem hiding this comment.
All reported issues were addressed across 12 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
|
1 participant
Closes #73. Ports upstream semble#219 (
949559b).Changes
dense.rs—DEFAULT_MODEL_NAMEminishlab/potion-code-16M→minishlab/potion-code-16M-v2(+ doc/comment sync).index.rs—try_reusenow comparesmanifest.model_idagainst the resolved model name. A cache built with a different model (e.g. the old v1 default) is treated as stale and rebuilt — its persisted vectors are incompatible with queries embedded by the new model. Mirrors the intent of upstream's_metadata_matches. New testtry_reuse_rejects_stale_model.search.rs— drop chunks whose fused score is exactly0.0before ranking, matching upstreamcombined_scores = {chunk: score for ... if score}. Strengthenedsearch_alpha_one_is_semanticto assert the BM25-only chunk is dropped atalpha=1.0.README.md/README.ko.mdprose +references/semble.mddefault-model row synced (bilingual).Verification
cargo fmt --all✅ ·cargo clippy --all-targets --all-features -- -D warnings✅ (clean)cargo test --workspace✅ — 258 lib + 20 CLI tests pass (incl. newtry_reuse_rejects_stale_model)csp search "add numbers" <dir>returns real-embedding results with the new default model (no stub fallback → v2 resolves + loads from HF).Notes
~/.csp/index/) built with v1 auto-invalidate on the next query/index thanks to themodel_idgate — no manualcsp clear indexneeded.0.0-score filter is a near-no-op under normal adaptive alpha (RRF never yields 0), but applies atalpha ∈ {0.0, 1.0}where one leg is fully weighted out — kept for behavioral parity.Summary by cubic
Sets
minishlab/potion-code-16M-v2as the default embedding model, drops 0.0 fused‑score chunks, and auto‑rebuilds caches when the model, model kind, or chunk size changes. Adds streamed, cross‑platform content hashing and splits cache orchestration and the cosine backend; aligns with semble#219 and satisfies #73.Bug Fixes
Refactors
compute_content_hash_from_pathsto avoid loading all files when validating caches.Written for commit 1f5a6c3. Summary will update on new commits.