[hu-ja-ja]

Summary

Add an opt-in embeddingIdleTimeoutMs: after that many milliseconds without an
embed, the local ONNX pipeline is disposed and the next memory operation re-warms it
from the local Hugging Face cache. OpenCode v2 runs plugins inside a long-lived
serve --service process, so today the weights stay resident for days even when no
memory operation has run for hours — measured at +561 MB RSS with the default
model. Default is 0 (disabled): residency, first-call latency and vectors are
unchanged unless the user opts in.

Test plan

• Exercised the change locally (or N/A for docs/meta-only) — ran as the actual
  plugin in an OpenCode v2 service for four days with the option enabled;
  49 instrumented warmups / 54 releases, +561 MB model cost, ~500 MB handed back
  on release, 0 dispose errors (numbers below)
• Noted any follow-up or known limitations — see Risks below: measurements are
  Windows-only and no issue is filed for tracking

Checklist

• Branch is based on main (do not open PRs from commits already on main) —
  rebased onto 0d96489 (v2.37.0), ahead 1 / behind 0
• bun test passes (or N/A) — 977 pass / 7 skip / 0 fail across 134 files
• bun run typecheck passes (or N/A)
• bun run check passes (format + lint + typecheck; or N/A)
• Docs / README updated when behavior or contribution process changes —
  README.md (embedding paragraph + option table) and CONFIG_TEMPLATE;
  translated READMEs untouched

Behaviour

• The timer is armed after every embed(), so it measures idle time rather than
  uptime, and it is unref'd so it can never hold the process open. Remote embeddings
  never arm it: there is no local model to release.
• In-flight embeds defer disposal: a dispose() that arrives mid-flight runs as soon as
  the running call drains, instead of pulling the pipeline from under it.
• A generation counter covers the other direction: if pipeline() finishes loading
  after a dispose() ran, the new pipeline is disposed immediately instead of being
  installed, so a release cannot be undone by a slow warmup.
• LocalMemoryClient.close() releases the pipeline with the last Turso lease
  (releasePluginTurso() === true). EmbeddingService lives on globalThis, so an
  unconditional dispose would drop the model for a sibling instance still serving.
• Teardown logging uses safeLog(): a lazily created log directory can throw on
  ENOSPC/EACCES, which must not fail the idle timer or close().

Measurement

Method: the plugin logs process.memoryUsage().rss of the OpenCode service process at
two points — immediately before pipeline() builds the model, and at the end of every
release — so the model cost is a within-process paired delta that unrelated allocations
cannot inflate.

Windows 11, Bun 1.4.3, Xenova/nomic-embed-text-v1 (fp32), four days of ordinary use
(2026-10-27 → 2026-10-30), embeddingIdleTimeoutMs: 600000:

State	RSS, median [p25–p75]	n
Service before model load	356 MB [341–388]	49
Service after model load	920 MB [904–946]	49
Model cost (paired delta)	+561 MB [557–564]	49
After idle release	442 MB [355–491]	54
Before next model load	356 MB [341–388]	49

Peak observed RSS: 1001 MB. Re-warm takes 1.16 s median, 1.57 s p90, 3.1 s worst (first
use of the day). RSS drops by ~500 MB at release and returns to baseline by the next build.

Across the 86.8 h window the model was resident 50% of the time — the feature working,
and also why the default stays off: with a 10-minute timeout, 29 of 34 idle releases
were followed by another embed within 30 minutes, so short values mostly churn. No
dispose errors or races were logged across those four days.

Verification

• bun test tests/embedding-idle-dispose.test.ts tests/config.test.ts tests/plugin-turso-refcount.test.ts — 40 pass, 0 fail.
• Mutation checks, so the new tests are not vacuous: skipping the idle-timer arming
  fails 1 of 3 idle-release tests; disposing on every close() instead of the last
  lease fails 1 of 2 refcount tests.
• Full suite: 977 pass / 7 skip / 0 fail. One memory-portability test timed out once
  under full-suite load and passes standalone on both runs; unrelated to this change.

Risks / open questions

• Measurements are Windows 11 only, and RSS returning to the OS is engine
  behaviour rather than a documented contract (it did here: ≈500 MB back immediately,
  more by the next load). macOS/Linux residency and re-warm time are unmeasured.

AI assistance

Developed with an AI coding agent (OpenCode, DeepSeek V4.1 Flash): implementation,
tests, the measurements above and this description. The numbers come from real runs on
the contributor's machine and the result was reviewed before submission.

[Copilot]

🟡 Changes recommended

Embedding lifecycle races can retain or recreate pipelines after disposal, while idle release can stall profile learning until an unrelated operation reloads the model.

3 open issues

• Warm idle-released models before merging profile observations · New
• Capture pipeline generation before awaiting model load · New
• Deferred disposal is lost when the last client closes · New

What changed in this PR

Adds opt-in idle release of the local ONNX embedding model to reduce memory usage in long-running OpenCode hosts. The default remains disabled.

Changes:

• Adds embeddingIdleTimeoutMs configuration and documentation.
• Introduces pipeline disposal, idle scheduling, and final-client teardown.
• Adds tests for timeout behavior, in-flight embeddings, and shared-client ownership.

File	Description
tests/​plugin-turso-refcount.test.ts	Checks model release only after the final client lease closes.
tests/​fixtures/​memory-injection-eligibility.mjs	Adds disposal support to the embedding mock.
tests/​embedding-idle-dispose.test.ts	Tests idle release, disabled timeout, and in-flight protection.
tests/​config.test.ts	Tests timeout defaults and validation.
src/​services/​embedding.ts	Adds idle timers, pipeline disposal, and warmup generation tracking.
src/​services/​client.ts	Releases embeddings when the final Turso lease closes.
src/​config.ts	Defines, validates, and documents the timeout option.
README.md	Documents idle release and subsequent model warmup.

_{Review effort: Balanced}

[hu-ja-ja]

This is the existing cold-buffer path (buffered while embeddings are unavailable, drained on the next warm merge, cap 50); warming on demand here would reload the model at session idle, which is when profile learning runs, so it is out of scope for this PR.