Skip to content

feat(sessions): BM25-ranked session search with search-index feature Refs #758 - #831

Merged
AlexMikhalev merged 8 commits into
mainfrom
task/758-session-search-index
Apr 22, 2026
Merged

AlexMikhalev merged 8 commits into
mainfrom
task/758-session-search-index

Conversation

@AlexMikhalev

Copy link
Copy Markdown
Contributor

Summary

Replaces brute-force substring matching in session search with BM25 (Okapi) scoring for relevance-ranked results, reusing the existing BM25 infrastructure from terraphim_types::score.

Changes

Step 1: Extract score module to terraphim_types

  • Moved BM25/BM25F/BM25Plus/TF-IDF/Jaccard/QueryRatio scorers from terraphim_service::score to terraphim_types::score
  • All 19 score tests pass in new location

Step 2: Re-export from terraphim_service

  • terraphim_service::score is now a thin re-export shim: pub use terraphim_types::score::*
  • Removed ~3400 lines of duplicated code, backwards compatible

Step 3: search-index feature for terraphim_sessions

  • New search-index feature flag depending on terraphim_types
  • session_to_document() adapter converts Session → Document (title + body from messages + metadata)
  • search_sessions() performs BM25-ranked search with relative score threshold filtering

Step 4: Replace brute-force search

  • SessionService::search() now uses BM25 when search-index feature is enabled
  • Falls back to substring matching when feature is disabled (feature-gated with #[cfg])

Step 5: /sessions index CLI command

  • Added SessionsSubcommand::Index { verbose } with parser
  • Shows session count, message count, scorer type, and optional source breakdown

Testing

  • 61 tests pass with search-index feature
  • 52 tests pass without (fallback path)
  • All pre-commit checks pass (fmt, clippy, build, tests, UBS)

Refs #758 (Gitea)

@AlexMikhalev
AlexMikhalev merged commit 6871da0 into main Apr 22, 2026
22 checks passed
@AlexMikhalev
AlexMikhalev deleted the task/758-session-search-index branch April 22, 2026 13:34
AlexMikhalev added a commit that referenced this pull request Apr 25, 2026
…Refs #758 (#831)

* feat(types): extract BM25 score module to terraphim_types Refs #758

* refactor(service): replace score module with re-export from terraphim_types Refs #758

* feat(sessions): add search-index feature with BM25 session search adapter Refs #758

* feat(sessions): replace brute-force search with BM25-ranked results Refs #758

* feat(agent): add /sessions index CLI command for BM25 search index stats Refs #758

* fix(types): add Default derives for all scorer structs (clippy) Refs #758

* feat(sessions): hybrid search with KG concepts first, BM25 fallback Refs #758

* fix(sessions): UTF-8 boundary panic in build_body truncation, log BM25 errors Refs #758
AlexMikhalev added a commit that referenced this pull request May 1, 2026
…Refs #758 (#831)

* feat(types): extract BM25 score module to terraphim_types Refs #758

* refactor(service): replace score module with re-export from terraphim_types Refs #758

* feat(sessions): add search-index feature with BM25 session search adapter Refs #758

* feat(sessions): replace brute-force search with BM25-ranked results Refs #758

* feat(agent): add /sessions index CLI command for BM25 search index stats Refs #758

* fix(types): add Default derives for all scorer structs (clippy) Refs #758

* feat(sessions): hybrid search with KG concepts first, BM25 fallback Refs #758

* fix(sessions): UTF-8 boundary panic in build_body truncation, log BM25 errors Refs #758
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant