Skip to content

Bug: namespace store and tree sign the same embedder with different provider labels (managed vs cloud), so every vector written under the other alias is skipped by recall #139

Description

@YellowSnnowmann

Version or commit

tinymemory v1.14.1 (52836fd), running as the prebuilt module in openhuman main f5c5ba892. tinycortex vendored at cb1c163.

Rust toolchain

rustc 1.96.1 (aarch64-apple-darwin)

Reproduction

  1. Run openhuman with the managed cloud embedder ([memory] embedding_provider = "managed", the desktop default) on a store that already holds vectors signed with the cloud alias. Such rows come from the host's hard-coded managed embedder, which names itself cloud regardless of the configured provider (Memory sync ignores the configured memory embedding provider and 401s against the cloud embedder on local installs openhuman#5730), or from a config that carried embedding_provider = "cloud".
  2. Sync a connector or write any document, then inspect memory.db:
SELECT model_signature, COUNT(*), SUM(embedding IS NULL) FROM vector_chunks GROUP BY 1;
  1. Compare with the tree sidecar in memory_tree/chunks.db:
SELECT model_signature, COUNT(*) FROM mem_tree_chunk_embeddings GROUP BY 1;

Observed on a live desktop store (2026-09-04):

store signature rows
vector_chunks (namespace store), written after the slug changed to managed (2026-09-01 17:13) provider=managed;model=embedding-v1;dims=1024 8,365
vector_chunks, written before that (label cloud) provider=cloud;model=embedding-v1;dims=1024 12,672
vector_chunks, embed failed, never repaired NULL 30,358
mem_tree_chunk_embeddings (tree), written the same day provider=cloud;model=embedding-v1;dims=1024 1,078

The flip is sharp and matches a settings change, not a release. At 17:10:20 the host logged [embeddings::rpc] update_settings applied provider="managed" model="embedding-v1" dimensions=1024 sig_changed=true requeued=0 (the user switched the Embeddings tab from Ollama to Managed), the app restarted at 17:12:24, the last provider=cloud row is from 17:11:55 and the first provider=managed row landed at 17:13:46. Same endpoint, same model, same dimensions, new label. Two tree sidecar rows written at 17:10:41, between the switch and the restart, are signed provider=cloud;model=nomic-embed-text:latest;dims=768 while holding 1024-wide vectors, which shows the tree signature is assembled from config fields rather than from the embedder that actually ran.

(An earlier draft of this issue blamed the module-driver cut-over. The driver was bound at every boot that day, so that was wrong.)

Expected behavior

One embedder produces one signature everywhere it writes, and a label-only change never hides vectors. Rows whose model and dims match the active embedder stay retrievable (or are re-signed once), and the tree and namespace store agree about what a stored vector is.

Actual behavior

Two code paths spell the provider component differently for the same physical embedder:

  • Tree: crates/tinymemory-core/src/engine/config.rs:60 sets EmbeddingConfig.provider = effective_embedder_slug(config), which is "cloud" for EmbedderChoice::Cloud (tree/score/embed/factory.rs:330); build_cloud_embedder also hard-codes ProviderEmbedder::new(provider, "cloud") (factory.rs:377). tree_active_signature (tinycortex src/memory/chunks/embeddings.rs) formats that → provider=cloud;….
  • Namespace store: crates/tinymemory-core/src/store/factories.rs:264 takes memory.embedding_provider from the host config ("managed") and calls require_embedding_host().create_embedding_provider_with_credentials("managed", …); the module's BusEmbeddingProvider keeps that literal as name() (crates/tinymemory-module/src/embedding.rs:46-49), so the default trait signature() (tinymemory-api/src/host/embeddings.rs:82) yields provider=managed;…. The older rows carry cloud because the host's default_embedding_provider_with_config() hard-codes OpenHumanCloudEmbedding (inner tinyinference CloudEmbeddingModel::name() is "cloud") regardless of the configured provider (Memory sync ignores the configured memory embedding provider and 401s against the cloud embedder on local installs openhuman#5730), and because openhuman's config migrations write both spellings (unify_ai_provider_settings falls back to "cloud", migrate_legacy_embedding_provider writes "managed"). Both aliases exist in the wild, and the tree resolves either of them to EmbedderChoice::Cloud whenever a cloud session exists (factory.rs:184).

The namespace reader (store/namespace_store/query.rs:630-637) does if sig != active_signature { continue } and then if embedding.len() != query_embedding.len() { continue }, so every pre-cutover row is skipped silently. Nothing re-signs or re-embeds vector_chunks (the only writers are the per-document DELETE … WHERE document_id on rewrite and DELETE … WHERE namespace on forget; the reembed_backfill job covers only the tree sidecars). On the store above the global namespace, the one per-turn recall reads, has 0 rows at the active signature (124 null + 24 cloud).

tinycortex#132 unified the format ({model}@{dims} vs provider=…;model=…;dims=…); the label still comes from two sources.

Additional context

Suggested fix, in order:

  1. One label source. Derive the namespace store's provider component from the same resolution the tree uses (effective_embedder_slug), or map the managed/cloud aliases to one canonical slug before format_embedding_signature. "managed" | "cloud" | "openhuman" are already treated as aliases in tree/score/embed/openai_compat.rs:97.
  2. Re-sign, don't re-embed. A one-time migration: rows whose model and dim match the active signature and whose provider is an alias of it get model_signature rewritten. No provider calls, no budget.
  3. Repair the null rows (the openhuman#5300 item 1 sweep): a durable pass over vector_chunks WHERE embedding IS NULL, re-embedding in provider-sized batches and writing only width-matching vectors. Separate PR; listed here because the same store shows 30k of them.
  4. Doctor/health should report a signature split between the two stores; today nothing surfaces it.

Related: tinyhumansai/openhuman#5300 (parent, failure 3), tinyhumansai/openhuman#6025, tinyhumansai/tinycortex#132, #138.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: p1Next. Wrong behaviour a user will hit, or a security weakness behind a condition.rustPull requests that update rust code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions