Skip to content

Add oMLX embedding provider, cloud file support, and interactive demo - #3

Merged
edgardmello merged 22 commits into
mainfrom
claude/project-thread-13488h
Oct 23, 2026
Merged

edgardmello merged 22 commits into
mainfrom
claude/project-thread-13488h

Conversation

@edgardmello

Copy link
Copy Markdown
Owner

This PR introduces several major features and improvements to Lap:

Summary

Adds support for optional oMLX embedding provider for image search, implements cloud-only file handling, introduces relocatable thumbnail cache storage, and creates an interactive browser-based demo of the Lap frontend.

Key Changes

oMLX Embedding Provider Integration

  • New t_omlx.rs module providing OpenAI-compatible /v1/embeddings endpoint support for local embedding computation
  • Configurable server URL (defaults to localhost:8000) with health checks and timeout handling
  • Score calibration system for different embedding models with automatic tuning based on library content
  • Concurrent re-indexing with configurable batch sizes and worker threads
  • Search score thresholds and coherent cluster detection for improved result relevance

Cloud File Support

  • New CloudScanAction enum and cloud_scan_action() function to handle online-only files without downloading
  • Added cloud_only and metadata_pending fields to AFile struct for tracking file state
  • Dedup scan now tracks skipped online-only files in DedupScanStatus
  • Proper handling of file metadata updates for cloud-only files during library scans

Relocatable Thumbnail Cache

  • New t_cache_storage.rs module for managing thumbnail cache location independently from system defaults
  • Migration support with cancellation capability for moving cache between locations
  • Atomic operations with read/write locking for cache consistency
  • Configuration stored in app config with thumbnail_cache_dir field

Interactive Browser Demo

  • New docs/demo-native/ directory with browser-based Lap frontend demo
  • Deterministic mock IPC adapter (tauri.mjs) providing fixture-based responses
  • Demo-specific build script and test infrastructure
  • Actual Vue components running in browser without native dependencies
  • Sample photo fixtures with realistic metadata

UI/UX Improvements

  • Language selection now uses changeLanguage() callback for proper async handling
  • Preview navigation buttons now properly disabled when at boundaries
  • Album list supports drag-and-drop folder addition with visual feedback
  • Added cloud icon for indicating online-only files
  • Improved preview button styling with icon transitions
  • Settings panel for embedding provider configuration

Testing & Documentation

  • New test files for thumbnail scheduler, album recovery, external drag/drop, and image search provider
  • Browser-based E2E tests using Playwright against demo components
  • Updated documentation with interactive demo component
  • Comprehensive test coverage for new cloud file and cache features

Configuration & Dependencies

  • Added reqwest with JSON feature for HTTP requests to embedding server
  • Updated Node.js requirement to 22 in CI/CD
  • New database migration (version 19) for cloud file support
  • Extended app config with thumbnail cache directory option

Notable Implementation Details

  • Embedding search parallelization threshold (PARALLEL_SCORE_MIN_ROWS) set to 4096 rows for optimal performance
  • Score calibration uses mean pairwise cosine similarity (0.75 threshold) for coherent cluster detection
  • Cloud file scanning avoids unnecessary downloads while maintaining metadata accuracy
  • Cache migration uses atomic operations to prevent data loss during relocation
  • Demo uses localStorage for persistence and Vue's reactivity for state management

https://claude.ai/code/session_01W6JQYbxMg96TrsUQD1cGsm

OPGERUIMD97 and others added 19 commits October 15, 2026 15:21
- Fix the misspelled 'Ophelderd' (brightened) -> 'Helderder'
- Use 'Gekoppelde bestanden' consistently for paired files
- Use the verb-last pattern of other options ('Systeem volgen', 'App-taal volgen')
- Complete the offline 'requires original' message
- 'Willekeurig draaien' for random rotation in collage


Claude-Session: https://claude.ai/code/session_01XFzC7XqToaa3ABP8G83GgS

Co-authored-by: OPGERUIMD97 <OPGERUIMD97@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…lyx10#394 part 2) (julyx10#432)

* feat: add cloud-only read guards and OneDrive release helper (julyx10#394)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: store cloud-only and pending-metadata state per file (julyx10#394)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: index cloud-only files from their directory entry without downloading (julyx10#394)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: skip cloud-only originals in thumbnail, AI, face and duplicate tasks (julyx10#394)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: show a cloud badge and keep online-only originals unread in the grid (julyx10#394)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: add 'Download, index, then free up space' for cloud-only selections (julyx10#394)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep cloud-only files unread in Live Photo pairing, previews and duplicate hashes (julyx10#394)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cloud): re-sync a folder when returning to it from another view

Free up space leaves the folder mtime unchanged, so the cloud flags are only
refreshed by the folder sync, which was skipped while the debounce key still
named the folder.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cloud): keep a vanished file's cloud flag off at rename

update_path ran for any unreadable renamed file and always set cloud_only = 1;
read the flag from disk instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cloud): harden online-only handling with fail-safe reads and async release

- refresh cloud state per opened folder without loading full catalog rows
- guard face/embedding/thumbnail/person reads with content_readable
- report OneDrive release as requested (async), surface partial errors
- fail-safe isCloudOnlyFile, invalidate stale hover/warmup attribute checks
- drop output-only live_photo_video_cloud_only in favor of a live check
- backfill cloud-only UI strings across all locales

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: julyx10 <tiangle@gmail.com>
The Welcome screen carried its own language selector, so a new install showed
English until the user happened to notice it.

- resolve the system locale once before components read messages, mapping
  zh-Hant (TW/HK/MO) to zh-TW and falling back to en for unsupported or
  malformed values
- record languageInitialized, and let setLanguage(fromUser) claim user intent
  before the locale bundle loads, so the startup probe and the settings sync
  echo can never overwrite a choice the user just made
- migrate existing installs to "already chosen" on config load
- drop the Welcome language selector; the same 15 options stay in
  Settings > General
Entering scan streaming from a previously sorted view or another album kept
the old content request and thumbnail queue alive, so pages of the previous
query could still hydrate the cleared placeholders and enqueue their
thumbnails.

Bump currentContentRequestId and invalidate the thumbnail scheduler, matching
what updateContent already does.
Let image-search embeddings be computed by a local oMLX server
(OpenAI-compatible /v1/embeddings) instead of the bundled ONNX CLIP
models. The provider is opt-in from Settings > Search, runs entirely on
the user's machine, and falls back gracefully: the built-in engine stays
the default and the ONNX models simply stay unloaded while oMLX is
active.

- New t_omlx module: connection test, query/image embeddings, batched
  parallel background re-index with progress events and cancellation
- Version embeddings by model (afiles.embed_model, migration v20) so
  vectors from different models never mix; searches, coverage, and
  incremental indexing filter by the active model
- Library guards around batch writes: requests run without the guard and
  results are stored only if the library is still the active one
- Per-library score calibration and strictness levels for oMLX searches
- EmbeddingGemma query task prompt; optional Matryoshka dimensions
- Settings UI with server URL / API key / model / dimensions, connection
  test, coverage, and re-index progress; locale strings for all languages

Squashed and rebased onto main (previously 20 commits plus a merge).
- Show embedding coverage and the re-index controls for the built-in
  provider too, so its vectors can be rebuilt after switching back from
  oMLX. Scans now treat a file as embedded only when its vector comes
  from the active model (has_embedding), so regular scans also re-embed
  files whose vectors were produced by the other provider.
- Compute oMLX query vectors outside the search command's library guard.
  The guard is taken only for the database reads/writes around the HTTP
  request, so a slow server no longer blocks library switching, and a
  switch in the meantime fails the search with "Library changed" instead
  of reading or writing the wrong library. The built-in path no longer
  loads the ONNX engine for oMLX searches.
- Await provider restoration at startup before calling finish_startup,
  so the bundled ONNX models are not loaded when oMLX is active.
- Never read an online-only original to embed it with oMLX.
- Tests: re-index controls outside the oMLX-only block, startup order,
  has_embedding per model (including quoted model names), and upgrading
  a v18/v19 database through the cloud-state and embed_model migrations.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6JQYbxMg96TrsUQD1cGsm
Resolve conflicts with the reviewed oMLX provider, keeping the fork-only
changes: word-start filename/comment search and English translation of
oMLX search text (now used by the search command's query embedding).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6JQYbxMg96TrsUQD1cGsm

Copy link
Copy Markdown
Owner Author

build-macos (macos-15, aarch64-apple-darwin, aarch64) failed, but the cause is outside this PR's code. The Rust release build finished, and then bundling failed at code signing: security import → failed to import keychain certificate. Signing needs the Apple certificate secrets, and they aren't set (or are invalid) in this fork, so a re-run would fail the same way. To fix it, add the signing secrets to the fork or skip signing for fork builds. No change in this PR addresses it.


Generated by Claude Code

edgardmello and others added 3 commits October 22, 2026 17:00
The generate_embedding command held the library guard while the oMLX
request was in flight, the same issue the review flagged for search.
It now shares the search path's per-file helper: read the input under
the guard, embed without it, and store only if the library is unchanged.
AFile::generate_embedding is built-in only again, close to main.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6JQYbxMg96TrsUQD1cGsm
The fork has no importable Apple certificate, so tauri-action failed at
`security import` after a successful build. Check the certificate first
(the same import into a throwaway keychain) and, when it is missing or
invalid, build an unsigned DMG and skip notarization instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6JQYbxMg96TrsUQD1cGsm
@edgardmello
edgardmello merged commit b284a96 into main Oct 23, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants