Skip to content

feat(recall): add retained history and shared intent briefs with use telemetry - #55

Merged
entelecheia merged 7 commits into
mainfrom
feat/40-recall-quality
Oct 24, 2026
Merged

entelecheia merged 7 commits into
mainfrom
feat/40-recall-quality

Conversation

@entelecheia

@entelecheia entelecheia commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Refs #40

Public recall now omits expired and currently superseded notes. The person can inspect verified retained history through explicitly authorized APIs and CLI flags, while current forget/prune removals hide the text at every date. CLI and MCP share one guarded, fail-first intent brief, and nullable use observations distinguish rendered output from explicit delivery/citation evidence.

F10 / AC5:

  • Public search/show/graph/timeline and briefs use live rows. Internal Store.get and default Store.graph keep their mutation/revision/raw-person semantics. A forgotten/expired/pruned replacement revives the older note, matching the existing reducer and eval.
  • Exported recallRows/recallShow/recallGraph/Store.recallState require person:true before any historical query; rawMemoryCommand also checks before opening its store. MCP exposes no historical arguments.
  • History reuses verified retained legacy/generation/shard inputs and the same reducer. Current forget/prune and known promotion ancestry mask every as-of. Forgotten rows can return only text-free id/time stubs; physically pruned text is never reconstructed. Generation skipped rows and successful-capture incomplete tails propagate diagnostics.
  • Dates mean UTC midnight for YYYY-MM-DD, or canonical millisecond/Z time. Future finite valid-at values remain current-visible for clock skew; as-of alone applies the time cutoff.
  • The fixed promotion-time-unknown notice is limited to actually emitted promoted rows, preserving separate source-incompleteness diagnostics. Actual promotion time is not invented.
  • Advisory stale probes resolve against the bound repo root from subdirectories. Missing paths at an unconfirmed root stay unknown; denies/folded names/private callbacks precede metadata probes, symlinks are not followed, and no code structure is indexed. Current CLI JSON remains byte-compatible raw View data; human/brief output carries hints.

F11 / AC6:

  • buildIntentBrief, planGuardNodes and matchingPlanFails are host-neutral exports. Fails precede decisions/contracts; scope is explicit paths/symbols plus one public co-reference step. Invalid named scope cannot broaden into lexical fallback; fallback excludes stopwords. The plan matcher excludes effective superseded and explicit expired fail rows.
  • One immutable policy per call, one screen per eligible row and one bounded source snapshot. Config, scope and metadata/SQLite revision guards fence awaits and refuse changes, instead of re-screening/re-reading bodies. Every adapter supplies policyVersion (constant only for immutable policy); mutable sources supply assertCurrent. PII gates run before queries and across awaits. Source snapshots are call-scoped.
  • Complete UTF-8 brief body:2048 default,8192 maximum, including untrusted-data header/JSON lines, with visible clipping/fetch pointers. Only emitted items are tracked. CLI/MCP return bounded body/status, not a second copy of full item bodies.
  • Public-export compatibility test accepts a structural hub task/adapter. The hub adoption/pin bump and hub-side test are agent-hub backlog card436, blocked on this PR and a published release. No hub source is changed here.

F12:

  • Existing machine-local nullable CandidateStore counters; missing/invalid/overflow metrics remain null, explicit zero remains zero. Known empty records is zero with unknown metrics; unavailable record count is unknown.
  • Rendered is recorded only after a successful output-write callback, never preparation/handler return/error. A shared writer rejects close-only failure. Delivery and citation require explicit host callbacks and are not inferred from tool output or note text. Replays are idempotent per emission; reused RPC ids do not collapse separate emissions. Telemetry errors fail open and never change curated JSONL.

Review changes:

  • Default CLI graph leaks and ungated exported history repaired, with pre-query person:false/undefined controls, live graph versus person-history tests and current removal masks.
  • Late-policy search/graph/timeline leaks and close-only hangs were reproduced red, then fenced with snapshots/versions and confirmed writes. Generation byte budget covers refresh/rebuild/retries before data read/hash/allocation.
  • Approved main-test changes: mcp-read-window timeline[old,current] -> [current], old show null, person history retains old; forgotten/revival controls retained. candidate-tools/mcp.test/mcp-cli and budget counts10->11 include full name lists with amem_brief; structured-array/authority assertions unchanged. Golden installed-old-version JSON parity test is unchanged and passes.
  • The same5001-row corpus includes3fail+3decision notes on named paths. R3 asserts6real rendered ids, fail-first order and byte cap, plus briefReads1; existing10search/4timeline/2work/corpus sizes and all R2/R3 limits/retry policies stay unchanged.

Measured latency (Bun1.4.2, native macOS,3000 synthetic notes,3 samples): reviewed23f2de6 search median1235.19ms and brief3089.72ms; repaired source15dc044 search343.40ms and brief400.10ms. Base main search355.04ms. This is native diagnostic evidence, not a cross-machine guarantee. Reproduce with scripts/measure-recall-latency.ts 3; exact samples/source refs in docs/verification/2026-10-25-amem422-recall-latency.json.

Documented ceilings: retained history50000records/32MiB row-data verification,64member show chain; unknown promotion event time; opened-retained-source diagnostics (unrouted invalid global pre-split rows are not fabricated per shard); advisory filesystem metadata and concurrent ancestor-swap ceiling; call-scoped snapshot consistency; automatic rendered observations at most256 curated ids per output. No new dependency, schema/hash/op/path or authored-row rewrite.

Validation at full headbeee63bd71f54a2351af3f5e0c8d10e80cb9d051 on fresh mainf654574c53306841d93ec57053cf9c2849bb8ebb:

  • Frozen Bun1.4.2 install; TypeScript clean; boundary:OK80 source files.
  • Full suite1486 pass,2 skip,0 fail (1488 tests,147 files,253.70s); npm pack dry-run88 files.
  • R3: idle48.36MB/peak174.38MB;6real rendered brief rows/952UTF8bytes,briefReads1,existing10search/4timeline/2work reads/corpus sizes unchanged,EOFchildreaped. Strict100/200MB and30s watchdog unchanged.
  • R2 allgroups pass: maxp9550.01ms/peak79.03MB,zerostalls/retainedprocesses;150ms/100MB andcallcounts unchanged.
  • Actual installed-tarball smoke passes unchanged private HOME,seeded MCP evidence andfresh durable native capture. No real CLI/HOME/account used.
  • Complete admitted log /tmp/amem422-r1-final.log ends exit_code0/outcome completed. Native focused core35/0,intent19/0,CLI/MCP15/0,late-policy3/0,privacy-matrix6/0 andoldJSONparity1/0 passed during development; fullgate contains them all.
  • Reproducible before/after native latency samples are committed with source refs; temporary clean baseline worktrees removed. Policy/source mutation,close/error,getter,unknown/clock-skew andprivilege guards remain covered.
  • Fresh-main own34-file/clean/conflict-marker guards,OCR rules andfull-diff review pass; no435or425unmergeddiff. Exact-head Ubuntu/macOS CI remains required; no local Linux result is claimed. Slot explicitly released.

All3Important/all5nits andgeneration diagnostic botP2 addressed. Defaultfuture-clock-skew policy follows the reviewer's explicit ruling, with finite validation and historical-only cutoff.

No publish, merge, real-home mutation, #425 or unrelated backlog implementation.

Copilot AI balanced review requested due to automatic review settings October 24, 2026 17:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T17:25:40.602682Z 23f2de6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 23f2de6ade

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/memory/store.ts
Comment thread src/memory/intent-brief.ts Outdated
@entelecheia
entelecheia force-pushed the feat/40-recall-quality branch from 23f2de6 to beee63b Compare October 24, 2026 18:05
@entelecheia
entelecheia merged commit a99e6c0 into main Oct 24, 2026
2 checks passed
@entelecheia
entelecheia deleted the feat/40-recall-quality branch October 24, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants