diff --git a/README.md b/README.md index 1b056f21..8bccb8ff 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ license CI node - tests + tests coverage Awesome

@@ -272,7 +272,7 @@ dsh-mneme-serve # 默认 ~/.dsh/memory + 127.0.0.1:8790 ```bash cd dsh-mneme && npm install -npm test # 2660 个测试 +npm test # 2680 个测试 npm run stress # 三轴线压测 npm run sync # src → lib 同步 ``` @@ -554,7 +554,7 @@ Community projects that wire mneme into more platforms — not part of this repo ```bash cd dsh-mneme && npm install -npm test # 2660 tests +npm test # 2680 tests npm run stress # three-axis stress test npm run sync # src → lib sync ``` diff --git a/dsh-mneme/CHANGELOG.md b/dsh-mneme/CHANGELOG.md index e1fa8cb6..6f1e6992 100644 --- a/dsh-mneme/CHANGELOG.md +++ b/dsh-mneme/CHANGELOG.md @@ -2,6 +2,20 @@ ## [Unreleased] +- **图谱的锚定层补上另外两路起点,被动确认有了能查的地方(#24 收尾)**:issue #24 的四块设计里,块 1 的「精确名 / Tag / 向量三路起点」当初只接了精确名一路——`anchoring.js` 本身早就支持多路(`anchorSeeds` 累加、`HOP_QUOTA` 分层都在),缺的是 `service.js` 的调用点只喂了 `normalizePath(..., { path: "exact" })` 一路,而通路身份在合并那一步就丢了(`normalizePath` 的返回值里没有 `path`,`anchorSeeds` 里 `p.path ?? "?"` 把每一路都记成 `"?"`)。块 4 的「例外管理」同样只有自动确认、没有可见面——面板里 `passiveConfirm` 一次都不出现。 + + **Tag 路走全等匹配,不走子串**:`store.findEntitiesByTags` 把查询按标点和空白切成词、滤掉单字(中文单字作 tag 没有区分度),再要求 `tags` 数组里**某个 tag 与词完全相等**。子串匹配会让 `ai` 命中 `mail`、让 `Java` 命中 `JavaScript`,而这条路的作用是「用户提到了某个主题词」,模糊命中的代价是把整类无关记忆拉进图轴。排序按命中记忆数降序、名称升序——同分时要有确定性顺序,否则同一查询两次跑出来的种子次序不同。 + + **锚点反查要看关系两端,不能只看 attr 行**:`findEntitiesByTags` 与 `entityIdsForMemories` 首版都只读 `entity_attrs`,而 `dream/sleep.js` 的 `phaseRelations` 只写 `entity_relations.memory_id`、不建 attr 行——仅由关系关联的记忆,对关系两端一个种子都生不出来(实测:只建关系不建 attr 时两个反查都返回空,而走 `saveAttr` 的对照组正常)。两个函数改为 `UNION` 三源(attrs / from / to),与 `getLinkedMemoryIds` 早就有的 attr|relation 双 tier 口径一致。同一条语句有三处 IN 列表共用记忆 id 集合,命中一多就撞 SQLite 变量上限,而调用方 `try/catch` 兜着——越限的表现是**整条 Tag 锚点静默消失**,故按 500 一批分块,块间 `via` 直接相加(各块记忆集合互不相交)。 + + **向量路复用同一份 query 向量**:`searchMemories` 里已经有 `qv`,把它以 `{ queryVector }` 传进 `entityRecall`,而不是在图轴内部再嵌一次——二次嵌入既多一次模型调用,也会因为本地模型的随机性让「检索用的向量」和「图轴用的向量」不是同一个,而这种偏差只在图轴结果上体现,排查时看不到。近邻记忆到实体仍是 MAX 语义(一条记忆挂多个实体时取最高分),最低分项**必须滤掉**:min-max 会把这路里最弱的那条压成 0,而 `anchorSeeds` 照收 0 分种子,下游给种子挂联记忆用的是**固定满权 tier**(attr 1.0 / relation 0.9),不滤就等于「0 分的种子按满权拉记忆」。 + + **早退条件也得改**:原来 `if (!entities.length && !tagEntities.length) return []` 会把一个只有向量路能工作的查询直接掐死(比如查询里不含任何已知实体名,但语义上和若干记忆很近)——那恰恰是向量路存在的理由。 + + **被动确认的复核面**:新增 `GET /semantic/graph/boosted-relations`(`policy: "open"`,同 `/semantic/graph/*` 前缀,`SECURITY.md` 两张表不必新增行),返回「合法抬升量 > 0 的边」——`gain` 必须由后端算,因为要和**来源基线**比对(`RELATION_SOURCE_DEFAULTS`:manual 1.0 / confirmed 1.0 / tag 0.3 / llm 0.4),前端手里只有当前 `weight`。判据用 `gain > 0` 而不是 `>= 0`:`weight` 是 REAL,`1.0 - 1.0` 在浮点里可能是 `1e-17`,用 `>=` 会把每一条从未被抬过的边都列进来。**排序也必须在算完 `gain` 之后做**:SQL 按 `weight DESC` 排、循环里再按条数截断,会让 llm 边(0.95,gain 0.55)压掉 tag 边(0.90,gain 0.60),被 `limit` 砍掉的恰恰是抬得更多的那条——故收全后按 `gain` 降序再 `slice`。ego 路由的 `edges` 同时补上 `weight` 与 `source`——只看 `weight` 会把「tag 边被抬到 0.5」误读成「比 manual 边弱」,而 `0.3 → 0.5` 恰恰是被用出来的证据。面板里这一面**默认收起、展开才取数**:它是审计视角而不是日常视角;闸关时只说「今后不会再自动抬权;下面列的是此前已经抬过的边」,**不隐藏列表**(关闸之后正是复核既有提升的时候),且空表提示与闸关状态解耦——否则「闸关 + 一条都没有」会只剩一句提示、没有任何行,读起来像这一屏坏了;取数失败单独说,不冒充闸关。 + + 验收:`test/graph-multipath.test.js`(10 条)与 `test/graph-boosted-relations.test.js`(9 条)新增,面板侧锁在 `test/client.test.js` 的 #24 用例;每条新用例都做过变异自证(把被它锁住的那行改坏,看它是否真的变红),其中两处首版用例是**空转的**、被变异自证当场抓住后重写:0 分种子用例原本把两条记忆挂在同一个实体上,滤不滤 0 分都到不了;排序用例把宽实体命名成字母序靠前的名字,按名字排也能过。全量 2680 tests / 2679 pass / 0 fail / 1 skipped。 + ## [0.10.0] - 2026-11-02 ## 🆕 新增 diff --git a/dsh-mneme/README.md b/dsh-mneme/README.md index 626f8f09..7a57c70b 100644 --- a/dsh-mneme/README.md +++ b/dsh-mneme/README.md @@ -5,7 +5,7 @@ [![npm version](https://img.shields.io/npm/v/@modusensus/dsh-mneme?color=blue&label=npm)](https://www.npmjs.com/package/@modusensus/dsh-mneme) [![license](https://img.shields.io/badge/license-MIT-green)](LICENSE) [![Awesome](https://awesome-dsh-plugin.com/badge.svg)](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin) -[![tests](https://img.shields.io/badge/tests-2660%20passed-success)](https://github.com/slow-stack/mneme) +[![tests](https://img.shields.io/badge/tests-2680%20passed-success)](https://github.com/slow-stack/mneme) [![CI](https://img.shields.io/github/actions/workflow/status/slow-stack/mneme/ci.yml)](https://github.com/slow-stack/mneme/actions) [![node](https://img.shields.io/badge/node-22%2B-blue)](https://nodejs.org) [![npm downloads](https://img.shields.io/npm/d18m/@modusensus/dsh-mneme.svg?color=blue&label=downloads)](https://www.npmjs.com/package/@modusensus/dsh-mneme) @@ -721,7 +721,7 @@ src/ ├── api.js # HTTP 路由(Web 面板数据通道,含 /conflicts 冲突队列) └── index.js # 插件接线 lib/ # src 的同步分发产物(npm run sync,prepack 钩子在发布时自动跑;一致性由 scripts/check-sync.js 在 CI 与 release.yml 校验——它只查不同步;唯一手写例外 lib/client.js——Web 面板 bundle,sync 不覆盖) -test/ # 2660 个 node:test 测试(审计与三轴线压测不变量;src↔lib 一致性由 scripts/check-sync.js 发布闸门校验) +test/ # 2680 个 node:test 测试(审计与三轴线压测不变量;src↔lib 一致性由 scripts/check-sync.js 发布闸门校验) scripts/ # e2e-dsh.js 端到端演示 · stress-dsh.js 三轴线压测 · sync-lib.js 同步 · check-sync.js 发布闸门 · benchmark-recall.js / benchmark-embed.js / benchmark-rerank.js 基准 · sync-test-badge.mjs 测试徽章 · build-runtime-manifest.mjs 运行时清单 ``` @@ -730,7 +730,7 @@ scripts/ # e2e-dsh.js 端到端演示 · stress-dsh.js 三轴线压 ```bash cd dsh-mneme npm install # 安装 peer 依赖(以 devDependencies 形式,用于本地测试) -npm test # 运行 2660 个测试 +npm test # 运行 2680 个测试 npm run stress # 三轴线压测:长会话检索 / 冲突仲裁 / 多 Agent 并发(离线 mock LLM) npm run sync # 把 src/ 同步到 lib/(发布时由 prepack 钩子自动执行) ``` diff --git a/dsh-mneme/lib/api.js b/dsh-mneme/lib/api.js index ae15bb95..418ff570 100644 --- a/dsh-mneme/lib/api.js +++ b/dsh-mneme/lib/api.js @@ -1438,7 +1438,57 @@ export function createApi(ctx, service, settings, commands, embedder, semantic = to: e.to_entity, relation_type: e.relation_type, memory_id: e.memory_id ?? null, - created_at: e.created_at + created_at: e.created_at, + // issue #24 块4:被动确认的复核要靠这两个字段——weight 是「用出来 + // 的分」,source 决定它的起点(manual 1.0 / tag 0.3 / llm 0.4)。 + // 少了它们,「这条边被抬过」在面板上根本看不出来。取不到时给 + // 初值形状而不是 undefined,前端不必到处判空。 + weight: e.weight ?? 1.0, + source: e.source ?? "manual" + })) + }); + } catch { + sendJson(res, 500, { error: "internal" }); + } + } + }); + + // --- graph boosted relations: the review side of passive confirmation --- + // issue #24 块4(被动确认)的复核出口。正常触达即 bump 边权重是安静的设计, + // 但「抬过哪些边」必须能看见,否则「例外管理」只剩自动确认没有例外。 + // 判据是 gain = 现权 − 该 source 初值 > 0(见 store.listBoostedRelations), + // 不另设记账表:抬升是唯一会改变检索形状的动作,它自己就是证据。 + // 只读、挂在 /semantic/graph/ 前缀下(SECURITY.md 的开放读清单已含该通配)。 + register({ + kind: "exact", + path: "/api/dsh-mneme/semantic/graph/boosted-relations", + policy: "open", + handler(req, res) { + try { + const url = new URL(req.url, "http://localhost"); + const limit = Math.max(1, Math.min(200, Number(url.searchParams.get("limit") ?? 100) || 100)); + const rows = service.listBoostedRelations?.({ limit }) ?? []; + sendJson(res, 200, { + count: rows.length, + // 提示「这张表为什么可能为空」:被确认抬升的边只在图真的被用过之后 + // 才出现,两个相关闸默认都关。面板据此说明而不是让人以为坏了。 + // 必须两个闸同时开才算「会继续产生新提升」——touchRecalled 里 + // passive = graphPassiveConfirm && graphWeightEnabled,只看前者会让面板 + // 说「会自动提升」而实际上一条都不会抬(评审指出)。 + passiveConfirmEnabled: config?.graphPassiveConfirm === true && config?.graphWeightEnabled === true, + items: rows.map((r) => ({ + id: r.id, + from: r.from_entity, + to: r.to_entity, + from_name: r.from_name ?? null, + to_name: r.to_name ?? null, + relation_type: r.relation_type, + memory_id: r.memory_id ?? null, + created_at: r.created_at, + weight: r.weight, + base_weight: r.base_weight, + gain: r.gain, + source: r.source })) }); } catch { diff --git a/dsh-mneme/lib/client.js b/dsh-mneme/lib/client.js index c0cddef4..4ae38763 100644 --- a/dsh-mneme/lib/client.js +++ b/dsh-mneme/lib/client.js @@ -452,6 +452,23 @@ window.__ModuleLoader__.load({ "memory.graph.viewInGraph": "在实体视图中打开", "memory.graph.loading": "加载中…", "memory.graph.distance": "距中心 {n} 跳", + // issue #24 块4:被动确认的复核面。边侧卡显示权重/来源,复核区列「抬过的边」。 + "memory.graph.weight": "权重 {w}", + "memory.graph.source": "来源 {s}", + "memory.graph.boosted.title": "被用出来的边", + "memory.graph.boosted.hint": "正常触达会自动抬高边的权重,这里列出被抬过的那些——抬升是唯一会改变检索形状的动作。", + "memory.graph.boosted.empty": "当前筛选下没有边(只列权重已高于其来源基线的边)", + "memory.graph.boosted.loadFailed": "这一屏没取到数据(接口不可用),不是「没有边」——稍后重试。", + "memory.graph.boosted.gateOff": "被动确认当前是关的,今后不会再自动抬权;下面列的是此前已经抬过的边。", + "memory.graph.boosted.show": "查看被用出来的边", + "memory.graph.boosted.hide": "收起", + "memory.graph.boosted.gain": "+{g}", + // 边的来源(#24 块2 的 RELATION_SOURCE_DEFAULTS):决定权重的起点, + // 所以复核时必须和权重一起显示,否则「0.5 比 1.0 弱」这种误读无可避免。 + "memory.graph.src.manual": "人工建立", + "memory.graph.src.confirmed": "人工确认", + "memory.graph.src.tag": "标签共现", + "memory.graph.src.llm": "模型抽取", "memory.settings.vectorTitle": "向量搜索", "memory.settings.vectorHint": "搜索不再只认字面:接入任意 OpenAI 兼容的 embedding 接口后,意思相近的记忆也能被召回。", "memory.settings.vectorEnabled": "启用向量搜索", @@ -1242,6 +1259,23 @@ window.__ModuleLoader__.load({ "memory.graph.viewInGraph": "Open in entity explorer", "memory.graph.loading": "Loading…", "memory.graph.distance": "{n} hop(s) from root", + // issue #24 块4:被动确认的复核面。 + "memory.graph.weight": "weight {w}", + "memory.graph.source": "via {s}", + "memory.graph.boosted.title": "Edges put to use", + "memory.graph.boosted.hint": "A normal touch raises an edge's weight automatically. These are the ones that rose — raising is the only action that changes what gets recalled.", + "memory.graph.boosted.empty": "No edges match this filter (only edges already above their source baseline are listed)", + "memory.graph.boosted.loadFailed": "Couldn't load this panel (endpoint unavailable) — that is not the same as \"no edges\". Try again later.", + "memory.graph.boosted.gateOff": "Passive confirmation is off, so nothing more will be raised; the edges below were raised earlier.", + "memory.graph.boosted.show": "Show edges put to use", + "memory.graph.boosted.hide": "Hide", + "memory.graph.boosted.gain": "+{g}", + // 边的来源(#24 块2 的 RELATION_SOURCE_DEFAULTS):决定权重的起点, + // 所以复核时必须和权重一起显示,否则「0.5 比 1.0 弱」这种误读无可避免。 + "memory.graph.src.manual": "manual", + "memory.graph.src.confirmed": "confirmed", + "memory.graph.src.tag": "tag overlap", + "memory.graph.src.llm": "model-extracted", "memory.settings.vectorTitle": "Vector Search", "memory.settings.vectorHint": "Match by meaning, not just wording — connect any OpenAI-compatible embeddings API.", "memory.settings.vectorEnabled": "Enable vector search", @@ -2089,6 +2123,13 @@ window.__ModuleLoader__.load({ ".mneme-relrow:hover{background:var(--dsw-alias-interactive-bg-hover)}", ".mneme-reltype{flex:none;font-size:11px;line-height:16px;color:var(--mneme-accent);border:1px solid color-mix(in srgb,var(--dsw-alias-state-business-primary) 35%,transparent);border-radius:6px;padding:0 6px}", ".mneme-relname{min-width:0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}", + // issue #24 块4 复核面:抬过的边。用虚线边色(--mneme-graphic-dim 同层) + // 而不是新增颜色——它表达的是「同一条边,只是被用过」,不是新类别。 + ".mneme-boosted{margin-top:6px;padding-top:6px;border-top:1px dashed var(--dsw-alias-border-l2)}", + ".mneme-boostedbody{margin-top:6px}", + ".mneme-boostedrow{cursor:default;font-size:12px;line-height:18px;padding:5px 10px}", + ".mneme-boostedrow:hover{background:none}", + ".mneme-boostedgain{font-size:11px;color:var(--mneme-accent)}", // --- settings: quiet stacked sections, one concern per section --- ".mneme-set{flex:1;min-height:0;overflow-y:auto;padding:8px 24px 48px;box-sizing:border-box}", ".mneme-set-inner{max-width:720px;margin:0 auto}", @@ -2801,6 +2842,10 @@ window.__ModuleLoader__.load({ const [selected, setSelected] = useState(null); // { kind: "node"|"edge", node?|edge? } const [attrs, setAttrs] = useState([]); const [related, setRelated] = useState([]); + // issue #24 块4:被动确认的复核面。默认收起——它是一个「想查的时候 + // 才查」的面,不该占实体页第一屏;展开时才去取,省一次请求。 + const [boostedOpen, setBoostedOpen] = useState(false); + const [boosted, setBoosted] = useState(null); // { count, passiveConfirmEnabled, items } const svgRef = useRef(null); const posRef = useRef([]); // live simulation positions, not React state const dragRef = useRef(null); // { id, moved } @@ -2866,6 +2911,21 @@ window.__ModuleLoader__.load({ return () => { cancelled = true; }; }, [entityName]); + // issue #24 块4 复核面:只在展开时取,收起时不发请求。 + const loadBoosted = useCallback(async () => { + try { + const res = await apiFetch("/api/dsh-mneme/semantic/graph/boosted-relations?limit=100"); + setBoosted(res.ok + ? await res.json() + : { count: 0, items: [], passiveConfirmEnabled: false, error: true }); + } catch { + setBoosted({ count: 0, items: [], passiveConfirmEnabled: false, error: true }); + } + }, []); + useEffect(() => { + if (boostedOpen) loadBoosted(); + }, [boostedOpen, loadBoosted]); + // Simulation: run in rAF against posRef, tick React only every few frames. // Re-seeds when data changes; dragging writes straight into posRef. useEffect(() => { @@ -3096,6 +3156,17 @@ window.__ModuleLoader__.load({ ), h("div", { className: "mneme-gs-meta" }, `${t("memory.graph.relation")} · ${formatRelativeTime(selected.edge.created_at, t)}`), + // issue #24 块4:复核要看得见「这条边被抬过」。weight 是当前分, + // source 决定它的起点(manual 1.0 / confirmed 1.0 / tag 0.3 / llm 0.4)—— + // 只看 weight 会把「tag 边被抬到 0.5」误读成「比 manual 边弱」。 + (selected.edge.weight !== undefined || selected.edge.source !== undefined) && h("div", { className: "mneme-gs-meta" }, + selected.edge.weight !== undefined + ? t("memory.graph.weight").replace("{w}", Number(selected.edge.weight).toFixed(2)) + : null, + selected.edge.weight !== undefined && selected.edge.source ? " · " : null, + selected.edge.source + ? t("memory.graph.source").replace("{s}", t(`memory.graph.src.${selected.edge.source}`) || selected.edge.source) + : null), selected.edge.memory_id && h("button", { className: "mneme-footbtn", onClick: () => onJumpMemory && onJumpMemory({ id: selected.edge.memory_id }) @@ -3200,7 +3271,53 @@ window.__ModuleLoader__.load({ h("span", { className: "mneme-reltype" }, e.relation_type), h("span", { className: "mneme-relname", style: { flex: 1 } }, otherName) ); - }) + }), + // issue #24 块4「例外管理」的可见面。被动确认是唯一会悄悄 + // 改变检索形状的动作(正常触达就抬权),所以它必须有个地方 + // 能查——否则「自动确认」和「没人看过的自动确认」没有区别。 + // 默认收起:这是审计视角,不是日常视角。 + h("div", { className: "mneme-boosted" }, + h("button", { + className: boostedOpen ? "mneme-chip mneme-active" : "mneme-chip", + "aria-expanded": boostedOpen ? "true" : "false", + onClick: () => setBoostedOpen(!boostedOpen), + title: t("memory.graph.boosted.hint") + }, boostedOpen ? t("memory.graph.boosted.hide") : t("memory.graph.boosted.show")), + boostedOpen && h("div", { className: "mneme-boostedbody" }, + (() => { + if (boosted === null) return h("div", { className: "mneme-hint" }, t("memory.graph.loading")); + // 加载失败 ≠ 闸关:接口挂了却说「被动确认是关的」,会把人 + // 引去翻设置,而真正的问题是这一屏没取到数据。 + if (boosted.error === true) return h("div", { className: "mneme-hint" }, t("memory.graph.boosted.loadFailed")); + const items = boosted.items || []; + const off = boosted.passiveConfirmEnabled !== true; + // 闸关 ≠ 没有数据:已经抬过的边仍然在表里,关掉闸只表示 + // 「今后不再自动抬」。所以先出闸关提示,再照常列历史—— + // 否则人永远没法复核既有的提升。 + // 空表提示与闸门状态无关:闸关且一条都没有时,光说「下面列的是 + // 此前已经抬过的边」却不给任何行,人会以为这一屏坏了。 + return h(react.Fragment, null, + off && h("div", { className: "mneme-hint" }, t("memory.graph.boosted.gateOff")), + items.length === 0 + ? h("div", { className: "mneme-hint" }, t("memory.graph.boosted.empty")) + : h(react.Fragment, null, + h("div", { className: "mneme-hint" }, t("memory.graph.boosted.hint")), + items.map((b) => h("div", { + key: b.id, + className: "mneme-relrow mneme-boostedrow", + title: `${t("memory.graph.source").replace("{s}", t(`memory.graph.src.${b.source}`) || b.source)} · ${Number(b.weight).toFixed(2)}` + }, + h("span", { className: "mneme-relname", style: { textAlign: "right", flex: 1 } }, b.from_name ?? b.from), + h("span", { className: "mneme-reltype" }, b.relation_type), + h("span", { className: "mneme-relname", style: { flex: 1 } }, `${b.to_name ?? b.to} `, + h("span", { className: "mneme-boostedgain" }, + t("memory.graph.boosted.gain").replace("{g}", Number(b.gain).toFixed(2)))) + )) + ) + ); + })() + ) + ) ) ), // ② 属性 — current snapshot (valid_until IS NULL);空层整层隐藏 diff --git a/dsh-mneme/lib/graph/anchoring.js b/dsh-mneme/lib/graph/anchoring.js index ba722f8e..62da7130 100644 --- a/dsh-mneme/lib/graph/anchoring.js +++ b/dsh-mneme/lib/graph/anchoring.js @@ -46,7 +46,12 @@ export function minMax(scores) { /** * 把一条检测通路的结果归一化成 0..1 的打分表。 * path 是离散通路名(exact/tag/...)→ 查 DISCRETE_LEVELS;是连续通路 - * (continuous=true)→ min-max。返回 { byId: Map, order: [id...] }。 + * (continuous=true)→ min-max。返回 { path, byId: Map, order }。 + * + * issue #24 块1 补充:返回值带上 `path`。anchorSeeds 用它记「这个种子被哪几路 + * 激活」(`seeds[i].paths`),复核时据此判断一条边是被精确名、Tag 桥接还是向量 + * 拉进来的。此前只有 { byId, order },`p.path ?? "?"` 把每一路都记成 "?", + * 通路身份在合并那一步就丢了。连续通路调用方传 `path: "vector"`(cos 相似度)。 */ export function normalizePath(hits, { path = "exact", continuous = false } = {}) { const ids = Array.isArray(hits) ? hits : []; @@ -62,7 +67,7 @@ export function normalizePath(hits, { path = "exact", continuous = false } = {}) const level = DISCRETE_LEVELS[path] ?? DISCRETE_LEVELS.substring; for (const id of ids) byId.set(id, level); } - return { byId, order: [...byId.keys()] }; + return { path, byId, order: [...byId.keys()] }; } /** diff --git a/dsh-mneme/lib/service.js b/dsh-mneme/lib/service.js index 8af55bfa..162ca625 100644 --- a/dsh-mneme/lib/service.js +++ b/dsh-mneme/lib/service.js @@ -560,16 +560,36 @@ export function createService({ store, mirror, config, onWrite, logger, document * 用出来的」延伸成检索信号:查询命中的不只有直接挂联,还有跳跃可达的活跃子图。 * 邻居权重 = HOP_QUOTA[hop] × linked tier(attr 1.0 / relation 0.9),仍低于 * 直达种子的满权,绝不喧宾夺主。 + * + * #24 块1 补充(三路起点接线):前半段只喂了 exact 一路,Tag 桥接与向量两路 + * 能力在 anchoring.js 里却从未被调用。现补齐—— + * · exact:查询里出现实体名(布尔 1.0); + * · tag :查询词命中某条记忆的 tag,该记忆挂联的实体即种子(计数级别 0.4); + * · vector:查询向量近邻的记忆挂联的实体(cos 走 min-max,连续通路)。 + * 三路在累加前各按自己的尺度归一化(离散级别 vs min-max),混加不会让高频 + * Tag 淹没强向量命中。两路都是弱信号入口:任一路为空就少一路,全空则 + * anchorSeeds 返回空、整个图轴让位(与加这两路之前的行为一致)。 + * 三者共用 graphAnchoringEnabled 一个闸:默认关时仍是逐字节的 #219 单跳轴。 */ - function entityRecall(q, limit) { + function entityRecall(q, limit, { queryVector = null } = {}) { if (config?.entityRecallEnabled !== true) return []; try { const entities = store.findEntitiesMentionedIn(q); - if (!entities.length) return []; + // Tag 桥接与向量两路只服务锚定层:关锚定时不查(少两次库读,也守住 + // 「关档行为逐字节不变」的承诺——见下方 if 分支)。 + const anchoring = config?.graphAnchoringEnabled === true; + const tagEntities = anchoring && typeof store.findEntitiesByTags === "function" + ? (() => { try { return store.findEntitiesByTags(q); } catch { return []; } })() + : []; + // 早退只在**三路全无来源**时才成立。向量路的存在感恰恰在查询里没有实体名 + // 的时候最大(「数据库迁移」说不出任何实体),若仍以「名字没命中」早退, + // 第二、三路就永远等不到自己被调用——接上了也等于没接。 + const canVector = anchoring && Array.isArray(queryVector) && queryVector.length > 0 && !!vectorIndex; + if (!entities.length && !tagEntities.length && !canVector) return []; // 关闭锚定级联=逐字节复用 #219 单跳轴:种子裁剪/重排/截断全是 #24 新增的 // 行为,默认关时一条都不许漏进旧路径(graphSeedCap 也只在开启后生效—— // 无论 cap 调多小,关档时命中实体仍全量挂联)。 - if (config?.graphAnchoringEnabled !== true) { + if (!anchoring) { const linked = store.getLinkedMemoryIds(entities.map((e) => e.id)); if (!linked.size) return []; const hits = []; @@ -581,9 +601,43 @@ export function createService({ store, mirror, config, onWrite, logger, document } return hits; } - // 锚定种子(精确名布尔通路——命名实体逻辑上是离散匹配,不参与 min-max)。 - const exact = normalizePath(entities.map((e) => e.id), { path: "exact" }); - const seeds = anchorSeeds({ paths: [exact], cap: config?.graphSeedCap ?? 12 }); + // 三路激活信号,各自先归一化再累加(尺度不同,混加会让高频通路淹没强相关)。 + const paths = []; + // 第一路:精确名(布尔通路——命名实体逻辑上是离散匹配,不参与 min-max)。 + if (entities.length) { + paths.push(normalizePath(entities.map((e) => e.id), { path: "exact" })); + } + // 第二路:Tag 桥接(tag 共现,离散级别 0.4——比精确名弱、比子串强)。 + if (tagEntities.length) { + paths.push(normalizePath(tagEntities.map((e) => e.id), { path: "tag" })); + } + // 第三路:向量(连续通路,cos 相似度走 min-max)。记忆 → 挂联实体反查, + // 同一实体被多条近邻记忆挂联时取最高分(一条更近的记忆就够证明相关)。 + if (Array.isArray(queryVector) && queryVector.length && vectorIndex) { + try { + const near = vectorIndex.search(queryVector, { limit: limit * 2, threshold: 0 }); + const bestOf = new Map(); + for (const m of near) { + const score = m?.score ?? 0; + if (!(score > 0)) continue; + for (const eid of store.entityIdsForMemories([m.id])) { + if (!(bestOf.get(eid) >= score)) bestOf.set(eid, score); + } + } + if (bestOf.size) { + paths.push(normalizePath( + [...bestOf.entries()].map(([id, score]) => ({ id, score })), + { path: "vector", continuous: true } + )); + } + } catch { /* 向量路失败只是少一路,不拖垮整条图轴 */ } + } + if (!paths.length) return []; + // 激活分为 0 的种子不算种子:min-max 会把**连续通路里最弱的那条**压成 0, + // 可下游给种子挂联记忆的是固定满权 tier(1.0/0.9),不看种子自己的分数。 + // 留着 0 分种子,等于让「这一路里最不像的那条」以最强的方式把记忆拉进池。 + // 零分项在 anchorSeeds 里排最后,所以先取 cap 再滤掉不会浪费名额。 + const seeds = anchorSeeds({ paths, cap: config?.graphSeedCap ?? 12 }).filter((s) => s.score > 0); if (!seeds.length) return []; // 0-hop 直达:挂联记忆按 tier 定分(attr 1.0 / relation 0.9)——与 #219 // 单跳轴同分。MAX 语义:同一条记忆被多实体挂联时保留最高分。 @@ -939,6 +993,9 @@ export function createService({ store, mirror, config, onWrite, logger, document const keyword = rawKeyword.map((m) => ({ ...m, score: scoreKeyword(m, q), source: "keyword" })); const wantVector = mode === "vector" || mode === "hybrid" || (mode === "auto" && !!embedder); let vector = []; + // #24 块1:查询向量在这里算出来,图轴的向量起点要用同一份——再嵌一次既 + // 多一次模型调用,也可能拿到不同的向量(本地模型带随机性时)。 + let vectorQuery = null; if (wantVector && embedder) { try { // Legacy embedders expose embed(query); local ones expose embedSingle. @@ -947,6 +1004,7 @@ export function createService({ store, mirror, config, onWrite, logger, document : embedder.embed.bind(embedder); const qv = await embedSingle(q); if (qv?.length) { + vectorQuery = qv; // Adaptive threshold (v0.5.0 1.2): the fetch runs at the loosest // branch floor so the head-gap rule can still re-admit the tail; // the final cutoff is computed against the fetched score @@ -980,7 +1038,7 @@ export function createService({ store, mirror, config, onWrite, logger, document // mentioned entities pull their linked memories into the pool. Keyword // mode stays text-only per the mode contract, so the axis is skipped // there even when enabled. - const entity = mode === "keyword" ? [] : entityRecall(q, lim); + const entity = mode === "keyword" ? [] : entityRecall(q, lim, { queryVector: vectorQuery }); // Entity axis shares BM25's backfill weight (we = wb) until #217 recall // data justifies a distinct entity weight. @@ -1819,7 +1877,7 @@ export function createService({ store, mirror, config, onWrite, logger, document // 为空时兜底检索,避免注入路径双倍候选成本。 if (config?.entityRecallEnabled === true && q) { try { - const graphHits = entityRecall(q, maxItems * 2); + const graphHits = entityRecall(q, maxItems * 2, { queryVector }); if (graphHits.length) { const graphIds = new Set(graphHits.map((m) => m.id)); candidates = [ @@ -2872,6 +2930,9 @@ export function createService({ store, mirror, config, onWrite, logger, document saveRelation: (r) => store.saveRelation(r), listEntities: (o) => store.listEntities(o), getRelations: (id) => store.getRelations(id), + // #24 块4 被动确认的复核出口:列出权重被 bump 抬过的边(gain = 现权 − + // 该 source 的初值)。纯读,不动权重——「看哪些边被用出来过」是人的动作。 + listBoostedRelations: (o) => store.listBoostedRelations(o), // v0.7.0 实体热投影:实体热 = 关联记忆 heat 聚合(取 max)。无关联记忆 // 或 heatEnabled=false 时返回 null;前端据此决定图谱节点大小/明暗。 entityHeat: (entityId) => { diff --git a/dsh-mneme/lib/store.js b/dsh-mneme/lib/store.js index cc16f19c..82b3d261 100644 --- a/dsh-mneme/lib/store.js +++ b/dsh-mneme/lib/store.js @@ -1647,6 +1647,11 @@ export function createStore(path) { // 解析成本可以忽略、又永远够不到那格上限,块数只随包条数线性增长。 const DELETE_ID_CHUNK = 500; + // Tag 锚点的 IN 列表分块。比 DELETE_ID_CHUNK 多一层顾虑:那条语句只有一处 IN, + // 这条有**三处**(attrs / relations.from / relations.to)共用同一个记忆 id 集合, + // 所以单块占用的参数是 3×chunk。取 500 后最坏 1500 个参数,离 32766 仍有二十倍余量。 + const TAG_ID_CHUNK = 500; + /** * 整包退出(车道 C)。删空一个不存在的包是幂等的 0,不是错误——对方点第二次「收起 * 这个包」与「收起一个从来没导入过的包」都该是这个答案;早退也不 bump generation: @@ -3712,6 +3717,152 @@ export function createStore(path) { return { migrated, invalidated }; } + /** + * Tag 桥接(issue #24 · 块1 三路起点里的第二路):查询词命中某些记忆的 tag + * 时,那些记忆挂联的实体本身就是「这个查询正在谈的东西」——即使查询里没有 + * 出现它们的名字。Tag 是 LLM 打的标签,比精确名松,但比向量具体,正好填在 + * 二者之间。 + * + * 匹配口径与 findEntitiesMentionedIn 对齐(JS 侧 toLowerCase,不受 SQLite + * 只折 ASCII 的限制):把查询按空白与常见中英标点切词,任一词与某个 tag + * 全等即算命中(不做子串——tag 通常是短词,子串匹配会让「ai」命中「mail」)。 + * 只认未归档未遗忘的行:一条已归档记忆的 tag 不该再拉出种子。 + * 返回 {id, name},按命中记忆数降序、名称升序(稳定)。 + */ + function findEntitiesByTags(text, { limit = 5 } = {}) { + const words = String(text ?? "") + .toLowerCase() + .split(/[\s,,、;;::!!??.。()()[\]【】"'“”‘’/|]+/u) + .map((w) => w.trim()) + .filter((w) => w.length >= 2); + if (!words.length) return []; + const rows = db.prepare( + `SELECT id, tags FROM memories WHERE archived = 0 AND forgotten = 0` + ).all(); + const memHits = new Set(); + for (const row of rows) { + const tags = parseTags(row.tags).map((t) => String(t).toLowerCase()); + if (!tags.length) continue; + if (words.some((w) => tags.includes(w))) memHits.add(row.id); + } + if (!memHits.size) return []; + // 计数必须在 SQL 里做:entityIdsForMemories 返回的是 DISTINCT entity_id, + // 拿它反推「命中几条记忆」永远是 1,排序就静默失效了(评审抓到的正是这个)。 + // + // 两个来源都要算:只读 entity_attrs 会漏掉「记忆只通过关系挂到实体上」的情况 + // ——phaseRelations 只写 entity_relations.memory_id、不建 attr 行,于是仅由关系 + // 关联的记忆对关系两端一个种子都生不出来。这与 getLinkedMemoryIds 的口径一致 + // (那边 attr 与 relation 两端同样并列)。UNION 去重的是 (实体, 记忆) 对, + // 所以同一条记忆的多条关联记录不会把 via 抬高。 + // + // 分块理由同 DELETE_ID_CHUNK:一条语句一行一个 `?`,撞上 SQLITE_MAX_VARIABLE_NUMBER + // (32766)会整笔抛错,而调用方 try/catch 兜底 → 静默丢掉整条 Tag 锚点。 + // 这里每块绑 3×TAG_ID_CHUNK 个参数(三处 IN 列表),500 仍有二十倍余量。 + const rowsByEntity = new Map(); + const memoryIds = [...memHits]; + for (let offset = 0; offset < memoryIds.length; offset += TAG_ID_CHUNK) { + const batch = memoryIds.slice(offset, offset + TAG_ID_CHUNK); + const marks = batch.map(() => "?").join(","); + const batchRows = db.prepare( + `SELECT hits.id AS id, e.name AS name, COUNT(DISTINCT hits.memory_id) AS via + FROM ( + SELECT entity_id AS id, memory_id FROM entity_attrs + WHERE memory_id IN (${marks}) AND entity_id IS NOT NULL + UNION + SELECT from_entity AS id, memory_id FROM entity_relations + WHERE memory_id IN (${marks}) AND from_entity IS NOT NULL + UNION + SELECT to_entity AS id, memory_id FROM entity_relations + WHERE memory_id IN (${marks}) AND to_entity IS NOT NULL + ) hits JOIN entities e ON e.id = hits.id + GROUP BY hits.id` + ).all(...batch, ...batch, ...batch); + for (const row of batchRows) { + const prev = rowsByEntity.get(row.id); + // 各块的记忆集合互不相交,所以 via 直接相加就是全集的 DISTINCT 计数。 + rowsByEntity.set(row.id, { id: row.id, name: row.name, via: (prev?.via ?? 0) + Number(row.via) }); + } + } + const rows2 = [...rowsByEntity.values()]; + if (!rows2.length) return []; + return rows2 + .map((r) => ({ id: r.id, name: r.name, via: r.via })) + .sort((a, b) => b.via - a.via || (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)) + .slice(0, limit) + .map(({ id, name }) => ({ id, name })); + } + + /** + * 按记忆批量反查实体 id(Tag 桥接用)。与 entitiesForMemory 的区别是「批量 + + * 带 id」:那边是面板详情页的单条展示(只要 name/type),这边要的是能直接 + * 喂给 normalizePath 的 id 列表。同一 (记忆, 实体) 只出一次。 + * + * 与 findEntitiesByTags 同口径:attr 行与关系两端并列。只读 entity_attrs 时, + * 一条只通过关系挂到实体上的记忆在向量路里也生不出种子(phaseRelations 不建 attr)。 + */ + function entityIdsForMemories(memoryIds) { + const ids = Array.isArray(memoryIds) ? memoryIds.filter(Boolean) : []; + if (!ids.length) return []; + const placeholders = ids.map(() => "?").join(","); + const rows = db.prepare( + `SELECT entity_id FROM entity_attrs + WHERE memory_id IN (${placeholders}) AND entity_id IS NOT NULL + UNION + SELECT from_entity AS entity_id FROM entity_relations + WHERE memory_id IN (${placeholders}) AND from_entity IS NOT NULL + UNION + SELECT to_entity AS entity_id FROM entity_relations + WHERE memory_id IN (${placeholders}) AND to_entity IS NOT NULL` + ).all(...ids, ...ids, ...ids); + return rows.map((r) => r.entity_id); + } + + /** + * 被动确认的复核出口(issue #24 · 块4):列出「权重已经被抬过」的边。 + * + * 设计口径:被动确认本身是安静的——正常触达就 bump,不打扰人。要复核的恰恰 + * 是「抬过的那些」,因为抬升是唯一会改变检索形状的动作。判据不看有没有被 + * bump 过(那要额外记账),看权重是否高于该 source 的初值:RELATION_SOURCE_ + * DEFAULTS 是建边时的起点,高出去的部分只能来自 bumpRelationWeight。 + * + * 显式传过 weight 的边(manual 自带权重)会被算成「抬过」,这在语义上没错 + * ——它确实不是初值——但它是人工意图,所以把 source 一起返回,面板自己决定 + * 怎么标。整数比较用 gain > 0 而不是 >= 0,防浮点累加的 1e-17 噪声。 + * 返回按 gain 降序、created_at 降序,附两侧实体名(面板不必再查一遍)。 + */ + function listBoostedRelations({ limit = 200, minGain = 0 } = {}) { + const { limit: lim } = sanitizePage(limit, 1, 200); + const rows = db.prepare( + `SELECT r.*, ef.name AS from_name, et.name AS to_name + FROM entity_relations r + LEFT JOIN entities ef ON ef.id = r.from_entity + LEFT JOIN entities et ON et.id = r.to_entity + ORDER BY r.weight DESC, r.created_at DESC` + ).all(); + const out = []; + for (const row of rows) { + const src = row.source ?? "manual"; + const base = Object.hasOwn(RELATION_SOURCE_DEFAULTS, src) + ? RELATION_SOURCE_DEFAULTS[src] + : 1.0; + const gain = (row.weight ?? 1.0) - base; + if (!(gain > minGain)) continue; + out.push({ + ...toRelation(row), + from_name: row.from_name ?? null, + to_name: row.to_name ?? null, + base_weight: base, + gain + }); + } + // 排序必须在算完 gain 之后做,不能交给 SQL 的 weight DESC:不同 source 的 + // base 不同,weight 大不等于 gain 大(llm 边 0.95 / gain 0.55 会压过 + // tag 边 0.90 / gain 0.60)。早先在循环里按 lim 提前 break 时,被截掉的 + // 恰恰是 gain 更大的边——面板按 limit=100 取数,截断处正是最该看见的那几条。 + out.sort((a, b) => b.gain - a.gain || String(b.created_at).localeCompare(String(a.created_at))); + return out.slice(0, lim); + } + /** Relations where the entity appears on either side (from or to). */ function getRelations(entityId) { return db.prepare( @@ -4202,6 +4353,10 @@ export function createStore(path) { entitiesForMemory, findMemoriesByAttr, findEntitiesMentionedIn, + // #24 块1 第二路(Tag 桥接)与块4 复核出口。 + findEntitiesByTags, + entityIdsForMemories, + listBoostedRelations, getLinkedMemoryIds, saveRelation, bumpRelationWeight, diff --git a/dsh-mneme/src/api.js b/dsh-mneme/src/api.js index ae15bb95..418ff570 100644 --- a/dsh-mneme/src/api.js +++ b/dsh-mneme/src/api.js @@ -1438,7 +1438,57 @@ export function createApi(ctx, service, settings, commands, embedder, semantic = to: e.to_entity, relation_type: e.relation_type, memory_id: e.memory_id ?? null, - created_at: e.created_at + created_at: e.created_at, + // issue #24 块4:被动确认的复核要靠这两个字段——weight 是「用出来 + // 的分」,source 决定它的起点(manual 1.0 / tag 0.3 / llm 0.4)。 + // 少了它们,「这条边被抬过」在面板上根本看不出来。取不到时给 + // 初值形状而不是 undefined,前端不必到处判空。 + weight: e.weight ?? 1.0, + source: e.source ?? "manual" + })) + }); + } catch { + sendJson(res, 500, { error: "internal" }); + } + } + }); + + // --- graph boosted relations: the review side of passive confirmation --- + // issue #24 块4(被动确认)的复核出口。正常触达即 bump 边权重是安静的设计, + // 但「抬过哪些边」必须能看见,否则「例外管理」只剩自动确认没有例外。 + // 判据是 gain = 现权 − 该 source 初值 > 0(见 store.listBoostedRelations), + // 不另设记账表:抬升是唯一会改变检索形状的动作,它自己就是证据。 + // 只读、挂在 /semantic/graph/ 前缀下(SECURITY.md 的开放读清单已含该通配)。 + register({ + kind: "exact", + path: "/api/dsh-mneme/semantic/graph/boosted-relations", + policy: "open", + handler(req, res) { + try { + const url = new URL(req.url, "http://localhost"); + const limit = Math.max(1, Math.min(200, Number(url.searchParams.get("limit") ?? 100) || 100)); + const rows = service.listBoostedRelations?.({ limit }) ?? []; + sendJson(res, 200, { + count: rows.length, + // 提示「这张表为什么可能为空」:被确认抬升的边只在图真的被用过之后 + // 才出现,两个相关闸默认都关。面板据此说明而不是让人以为坏了。 + // 必须两个闸同时开才算「会继续产生新提升」——touchRecalled 里 + // passive = graphPassiveConfirm && graphWeightEnabled,只看前者会让面板 + // 说「会自动提升」而实际上一条都不会抬(评审指出)。 + passiveConfirmEnabled: config?.graphPassiveConfirm === true && config?.graphWeightEnabled === true, + items: rows.map((r) => ({ + id: r.id, + from: r.from_entity, + to: r.to_entity, + from_name: r.from_name ?? null, + to_name: r.to_name ?? null, + relation_type: r.relation_type, + memory_id: r.memory_id ?? null, + created_at: r.created_at, + weight: r.weight, + base_weight: r.base_weight, + gain: r.gain, + source: r.source })) }); } catch { diff --git a/dsh-mneme/src/graph/anchoring.js b/dsh-mneme/src/graph/anchoring.js index ba722f8e..62da7130 100644 --- a/dsh-mneme/src/graph/anchoring.js +++ b/dsh-mneme/src/graph/anchoring.js @@ -46,7 +46,12 @@ export function minMax(scores) { /** * 把一条检测通路的结果归一化成 0..1 的打分表。 * path 是离散通路名(exact/tag/...)→ 查 DISCRETE_LEVELS;是连续通路 - * (continuous=true)→ min-max。返回 { byId: Map, order: [id...] }。 + * (continuous=true)→ min-max。返回 { path, byId: Map, order }。 + * + * issue #24 块1 补充:返回值带上 `path`。anchorSeeds 用它记「这个种子被哪几路 + * 激活」(`seeds[i].paths`),复核时据此判断一条边是被精确名、Tag 桥接还是向量 + * 拉进来的。此前只有 { byId, order },`p.path ?? "?"` 把每一路都记成 "?", + * 通路身份在合并那一步就丢了。连续通路调用方传 `path: "vector"`(cos 相似度)。 */ export function normalizePath(hits, { path = "exact", continuous = false } = {}) { const ids = Array.isArray(hits) ? hits : []; @@ -62,7 +67,7 @@ export function normalizePath(hits, { path = "exact", continuous = false } = {}) const level = DISCRETE_LEVELS[path] ?? DISCRETE_LEVELS.substring; for (const id of ids) byId.set(id, level); } - return { byId, order: [...byId.keys()] }; + return { path, byId, order: [...byId.keys()] }; } /** diff --git a/dsh-mneme/src/service.js b/dsh-mneme/src/service.js index 8af55bfa..162ca625 100644 --- a/dsh-mneme/src/service.js +++ b/dsh-mneme/src/service.js @@ -560,16 +560,36 @@ export function createService({ store, mirror, config, onWrite, logger, document * 用出来的」延伸成检索信号:查询命中的不只有直接挂联,还有跳跃可达的活跃子图。 * 邻居权重 = HOP_QUOTA[hop] × linked tier(attr 1.0 / relation 0.9),仍低于 * 直达种子的满权,绝不喧宾夺主。 + * + * #24 块1 补充(三路起点接线):前半段只喂了 exact 一路,Tag 桥接与向量两路 + * 能力在 anchoring.js 里却从未被调用。现补齐—— + * · exact:查询里出现实体名(布尔 1.0); + * · tag :查询词命中某条记忆的 tag,该记忆挂联的实体即种子(计数级别 0.4); + * · vector:查询向量近邻的记忆挂联的实体(cos 走 min-max,连续通路)。 + * 三路在累加前各按自己的尺度归一化(离散级别 vs min-max),混加不会让高频 + * Tag 淹没强向量命中。两路都是弱信号入口:任一路为空就少一路,全空则 + * anchorSeeds 返回空、整个图轴让位(与加这两路之前的行为一致)。 + * 三者共用 graphAnchoringEnabled 一个闸:默认关时仍是逐字节的 #219 单跳轴。 */ - function entityRecall(q, limit) { + function entityRecall(q, limit, { queryVector = null } = {}) { if (config?.entityRecallEnabled !== true) return []; try { const entities = store.findEntitiesMentionedIn(q); - if (!entities.length) return []; + // Tag 桥接与向量两路只服务锚定层:关锚定时不查(少两次库读,也守住 + // 「关档行为逐字节不变」的承诺——见下方 if 分支)。 + const anchoring = config?.graphAnchoringEnabled === true; + const tagEntities = anchoring && typeof store.findEntitiesByTags === "function" + ? (() => { try { return store.findEntitiesByTags(q); } catch { return []; } })() + : []; + // 早退只在**三路全无来源**时才成立。向量路的存在感恰恰在查询里没有实体名 + // 的时候最大(「数据库迁移」说不出任何实体),若仍以「名字没命中」早退, + // 第二、三路就永远等不到自己被调用——接上了也等于没接。 + const canVector = anchoring && Array.isArray(queryVector) && queryVector.length > 0 && !!vectorIndex; + if (!entities.length && !tagEntities.length && !canVector) return []; // 关闭锚定级联=逐字节复用 #219 单跳轴:种子裁剪/重排/截断全是 #24 新增的 // 行为,默认关时一条都不许漏进旧路径(graphSeedCap 也只在开启后生效—— // 无论 cap 调多小,关档时命中实体仍全量挂联)。 - if (config?.graphAnchoringEnabled !== true) { + if (!anchoring) { const linked = store.getLinkedMemoryIds(entities.map((e) => e.id)); if (!linked.size) return []; const hits = []; @@ -581,9 +601,43 @@ export function createService({ store, mirror, config, onWrite, logger, document } return hits; } - // 锚定种子(精确名布尔通路——命名实体逻辑上是离散匹配,不参与 min-max)。 - const exact = normalizePath(entities.map((e) => e.id), { path: "exact" }); - const seeds = anchorSeeds({ paths: [exact], cap: config?.graphSeedCap ?? 12 }); + // 三路激活信号,各自先归一化再累加(尺度不同,混加会让高频通路淹没强相关)。 + const paths = []; + // 第一路:精确名(布尔通路——命名实体逻辑上是离散匹配,不参与 min-max)。 + if (entities.length) { + paths.push(normalizePath(entities.map((e) => e.id), { path: "exact" })); + } + // 第二路:Tag 桥接(tag 共现,离散级别 0.4——比精确名弱、比子串强)。 + if (tagEntities.length) { + paths.push(normalizePath(tagEntities.map((e) => e.id), { path: "tag" })); + } + // 第三路:向量(连续通路,cos 相似度走 min-max)。记忆 → 挂联实体反查, + // 同一实体被多条近邻记忆挂联时取最高分(一条更近的记忆就够证明相关)。 + if (Array.isArray(queryVector) && queryVector.length && vectorIndex) { + try { + const near = vectorIndex.search(queryVector, { limit: limit * 2, threshold: 0 }); + const bestOf = new Map(); + for (const m of near) { + const score = m?.score ?? 0; + if (!(score > 0)) continue; + for (const eid of store.entityIdsForMemories([m.id])) { + if (!(bestOf.get(eid) >= score)) bestOf.set(eid, score); + } + } + if (bestOf.size) { + paths.push(normalizePath( + [...bestOf.entries()].map(([id, score]) => ({ id, score })), + { path: "vector", continuous: true } + )); + } + } catch { /* 向量路失败只是少一路,不拖垮整条图轴 */ } + } + if (!paths.length) return []; + // 激活分为 0 的种子不算种子:min-max 会把**连续通路里最弱的那条**压成 0, + // 可下游给种子挂联记忆的是固定满权 tier(1.0/0.9),不看种子自己的分数。 + // 留着 0 分种子,等于让「这一路里最不像的那条」以最强的方式把记忆拉进池。 + // 零分项在 anchorSeeds 里排最后,所以先取 cap 再滤掉不会浪费名额。 + const seeds = anchorSeeds({ paths, cap: config?.graphSeedCap ?? 12 }).filter((s) => s.score > 0); if (!seeds.length) return []; // 0-hop 直达:挂联记忆按 tier 定分(attr 1.0 / relation 0.9)——与 #219 // 单跳轴同分。MAX 语义:同一条记忆被多实体挂联时保留最高分。 @@ -939,6 +993,9 @@ export function createService({ store, mirror, config, onWrite, logger, document const keyword = rawKeyword.map((m) => ({ ...m, score: scoreKeyword(m, q), source: "keyword" })); const wantVector = mode === "vector" || mode === "hybrid" || (mode === "auto" && !!embedder); let vector = []; + // #24 块1:查询向量在这里算出来,图轴的向量起点要用同一份——再嵌一次既 + // 多一次模型调用,也可能拿到不同的向量(本地模型带随机性时)。 + let vectorQuery = null; if (wantVector && embedder) { try { // Legacy embedders expose embed(query); local ones expose embedSingle. @@ -947,6 +1004,7 @@ export function createService({ store, mirror, config, onWrite, logger, document : embedder.embed.bind(embedder); const qv = await embedSingle(q); if (qv?.length) { + vectorQuery = qv; // Adaptive threshold (v0.5.0 1.2): the fetch runs at the loosest // branch floor so the head-gap rule can still re-admit the tail; // the final cutoff is computed against the fetched score @@ -980,7 +1038,7 @@ export function createService({ store, mirror, config, onWrite, logger, document // mentioned entities pull their linked memories into the pool. Keyword // mode stays text-only per the mode contract, so the axis is skipped // there even when enabled. - const entity = mode === "keyword" ? [] : entityRecall(q, lim); + const entity = mode === "keyword" ? [] : entityRecall(q, lim, { queryVector: vectorQuery }); // Entity axis shares BM25's backfill weight (we = wb) until #217 recall // data justifies a distinct entity weight. @@ -1819,7 +1877,7 @@ export function createService({ store, mirror, config, onWrite, logger, document // 为空时兜底检索,避免注入路径双倍候选成本。 if (config?.entityRecallEnabled === true && q) { try { - const graphHits = entityRecall(q, maxItems * 2); + const graphHits = entityRecall(q, maxItems * 2, { queryVector }); if (graphHits.length) { const graphIds = new Set(graphHits.map((m) => m.id)); candidates = [ @@ -2872,6 +2930,9 @@ export function createService({ store, mirror, config, onWrite, logger, document saveRelation: (r) => store.saveRelation(r), listEntities: (o) => store.listEntities(o), getRelations: (id) => store.getRelations(id), + // #24 块4 被动确认的复核出口:列出权重被 bump 抬过的边(gain = 现权 − + // 该 source 的初值)。纯读,不动权重——「看哪些边被用出来过」是人的动作。 + listBoostedRelations: (o) => store.listBoostedRelations(o), // v0.7.0 实体热投影:实体热 = 关联记忆 heat 聚合(取 max)。无关联记忆 // 或 heatEnabled=false 时返回 null;前端据此决定图谱节点大小/明暗。 entityHeat: (entityId) => { diff --git a/dsh-mneme/src/store.js b/dsh-mneme/src/store.js index cc16f19c..82b3d261 100644 --- a/dsh-mneme/src/store.js +++ b/dsh-mneme/src/store.js @@ -1647,6 +1647,11 @@ export function createStore(path) { // 解析成本可以忽略、又永远够不到那格上限,块数只随包条数线性增长。 const DELETE_ID_CHUNK = 500; + // Tag 锚点的 IN 列表分块。比 DELETE_ID_CHUNK 多一层顾虑:那条语句只有一处 IN, + // 这条有**三处**(attrs / relations.from / relations.to)共用同一个记忆 id 集合, + // 所以单块占用的参数是 3×chunk。取 500 后最坏 1500 个参数,离 32766 仍有二十倍余量。 + const TAG_ID_CHUNK = 500; + /** * 整包退出(车道 C)。删空一个不存在的包是幂等的 0,不是错误——对方点第二次「收起 * 这个包」与「收起一个从来没导入过的包」都该是这个答案;早退也不 bump generation: @@ -3712,6 +3717,152 @@ export function createStore(path) { return { migrated, invalidated }; } + /** + * Tag 桥接(issue #24 · 块1 三路起点里的第二路):查询词命中某些记忆的 tag + * 时,那些记忆挂联的实体本身就是「这个查询正在谈的东西」——即使查询里没有 + * 出现它们的名字。Tag 是 LLM 打的标签,比精确名松,但比向量具体,正好填在 + * 二者之间。 + * + * 匹配口径与 findEntitiesMentionedIn 对齐(JS 侧 toLowerCase,不受 SQLite + * 只折 ASCII 的限制):把查询按空白与常见中英标点切词,任一词与某个 tag + * 全等即算命中(不做子串——tag 通常是短词,子串匹配会让「ai」命中「mail」)。 + * 只认未归档未遗忘的行:一条已归档记忆的 tag 不该再拉出种子。 + * 返回 {id, name},按命中记忆数降序、名称升序(稳定)。 + */ + function findEntitiesByTags(text, { limit = 5 } = {}) { + const words = String(text ?? "") + .toLowerCase() + .split(/[\s,,、;;::!!??.。()()[\]【】"'“”‘’/|]+/u) + .map((w) => w.trim()) + .filter((w) => w.length >= 2); + if (!words.length) return []; + const rows = db.prepare( + `SELECT id, tags FROM memories WHERE archived = 0 AND forgotten = 0` + ).all(); + const memHits = new Set(); + for (const row of rows) { + const tags = parseTags(row.tags).map((t) => String(t).toLowerCase()); + if (!tags.length) continue; + if (words.some((w) => tags.includes(w))) memHits.add(row.id); + } + if (!memHits.size) return []; + // 计数必须在 SQL 里做:entityIdsForMemories 返回的是 DISTINCT entity_id, + // 拿它反推「命中几条记忆」永远是 1,排序就静默失效了(评审抓到的正是这个)。 + // + // 两个来源都要算:只读 entity_attrs 会漏掉「记忆只通过关系挂到实体上」的情况 + // ——phaseRelations 只写 entity_relations.memory_id、不建 attr 行,于是仅由关系 + // 关联的记忆对关系两端一个种子都生不出来。这与 getLinkedMemoryIds 的口径一致 + // (那边 attr 与 relation 两端同样并列)。UNION 去重的是 (实体, 记忆) 对, + // 所以同一条记忆的多条关联记录不会把 via 抬高。 + // + // 分块理由同 DELETE_ID_CHUNK:一条语句一行一个 `?`,撞上 SQLITE_MAX_VARIABLE_NUMBER + // (32766)会整笔抛错,而调用方 try/catch 兜底 → 静默丢掉整条 Tag 锚点。 + // 这里每块绑 3×TAG_ID_CHUNK 个参数(三处 IN 列表),500 仍有二十倍余量。 + const rowsByEntity = new Map(); + const memoryIds = [...memHits]; + for (let offset = 0; offset < memoryIds.length; offset += TAG_ID_CHUNK) { + const batch = memoryIds.slice(offset, offset + TAG_ID_CHUNK); + const marks = batch.map(() => "?").join(","); + const batchRows = db.prepare( + `SELECT hits.id AS id, e.name AS name, COUNT(DISTINCT hits.memory_id) AS via + FROM ( + SELECT entity_id AS id, memory_id FROM entity_attrs + WHERE memory_id IN (${marks}) AND entity_id IS NOT NULL + UNION + SELECT from_entity AS id, memory_id FROM entity_relations + WHERE memory_id IN (${marks}) AND from_entity IS NOT NULL + UNION + SELECT to_entity AS id, memory_id FROM entity_relations + WHERE memory_id IN (${marks}) AND to_entity IS NOT NULL + ) hits JOIN entities e ON e.id = hits.id + GROUP BY hits.id` + ).all(...batch, ...batch, ...batch); + for (const row of batchRows) { + const prev = rowsByEntity.get(row.id); + // 各块的记忆集合互不相交,所以 via 直接相加就是全集的 DISTINCT 计数。 + rowsByEntity.set(row.id, { id: row.id, name: row.name, via: (prev?.via ?? 0) + Number(row.via) }); + } + } + const rows2 = [...rowsByEntity.values()]; + if (!rows2.length) return []; + return rows2 + .map((r) => ({ id: r.id, name: r.name, via: r.via })) + .sort((a, b) => b.via - a.via || (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)) + .slice(0, limit) + .map(({ id, name }) => ({ id, name })); + } + + /** + * 按记忆批量反查实体 id(Tag 桥接用)。与 entitiesForMemory 的区别是「批量 + + * 带 id」:那边是面板详情页的单条展示(只要 name/type),这边要的是能直接 + * 喂给 normalizePath 的 id 列表。同一 (记忆, 实体) 只出一次。 + * + * 与 findEntitiesByTags 同口径:attr 行与关系两端并列。只读 entity_attrs 时, + * 一条只通过关系挂到实体上的记忆在向量路里也生不出种子(phaseRelations 不建 attr)。 + */ + function entityIdsForMemories(memoryIds) { + const ids = Array.isArray(memoryIds) ? memoryIds.filter(Boolean) : []; + if (!ids.length) return []; + const placeholders = ids.map(() => "?").join(","); + const rows = db.prepare( + `SELECT entity_id FROM entity_attrs + WHERE memory_id IN (${placeholders}) AND entity_id IS NOT NULL + UNION + SELECT from_entity AS entity_id FROM entity_relations + WHERE memory_id IN (${placeholders}) AND from_entity IS NOT NULL + UNION + SELECT to_entity AS entity_id FROM entity_relations + WHERE memory_id IN (${placeholders}) AND to_entity IS NOT NULL` + ).all(...ids, ...ids, ...ids); + return rows.map((r) => r.entity_id); + } + + /** + * 被动确认的复核出口(issue #24 · 块4):列出「权重已经被抬过」的边。 + * + * 设计口径:被动确认本身是安静的——正常触达就 bump,不打扰人。要复核的恰恰 + * 是「抬过的那些」,因为抬升是唯一会改变检索形状的动作。判据不看有没有被 + * bump 过(那要额外记账),看权重是否高于该 source 的初值:RELATION_SOURCE_ + * DEFAULTS 是建边时的起点,高出去的部分只能来自 bumpRelationWeight。 + * + * 显式传过 weight 的边(manual 自带权重)会被算成「抬过」,这在语义上没错 + * ——它确实不是初值——但它是人工意图,所以把 source 一起返回,面板自己决定 + * 怎么标。整数比较用 gain > 0 而不是 >= 0,防浮点累加的 1e-17 噪声。 + * 返回按 gain 降序、created_at 降序,附两侧实体名(面板不必再查一遍)。 + */ + function listBoostedRelations({ limit = 200, minGain = 0 } = {}) { + const { limit: lim } = sanitizePage(limit, 1, 200); + const rows = db.prepare( + `SELECT r.*, ef.name AS from_name, et.name AS to_name + FROM entity_relations r + LEFT JOIN entities ef ON ef.id = r.from_entity + LEFT JOIN entities et ON et.id = r.to_entity + ORDER BY r.weight DESC, r.created_at DESC` + ).all(); + const out = []; + for (const row of rows) { + const src = row.source ?? "manual"; + const base = Object.hasOwn(RELATION_SOURCE_DEFAULTS, src) + ? RELATION_SOURCE_DEFAULTS[src] + : 1.0; + const gain = (row.weight ?? 1.0) - base; + if (!(gain > minGain)) continue; + out.push({ + ...toRelation(row), + from_name: row.from_name ?? null, + to_name: row.to_name ?? null, + base_weight: base, + gain + }); + } + // 排序必须在算完 gain 之后做,不能交给 SQL 的 weight DESC:不同 source 的 + // base 不同,weight 大不等于 gain 大(llm 边 0.95 / gain 0.55 会压过 + // tag 边 0.90 / gain 0.60)。早先在循环里按 lim 提前 break 时,被截掉的 + // 恰恰是 gain 更大的边——面板按 limit=100 取数,截断处正是最该看见的那几条。 + out.sort((a, b) => b.gain - a.gain || String(b.created_at).localeCompare(String(a.created_at))); + return out.slice(0, lim); + } + /** Relations where the entity appears on either side (from or to). */ function getRelations(entityId) { return db.prepare( @@ -4202,6 +4353,10 @@ export function createStore(path) { entitiesForMemory, findMemoriesByAttr, findEntitiesMentionedIn, + // #24 块1 第二路(Tag 桥接)与块4 复核出口。 + findEntitiesByTags, + entityIdsForMemories, + listBoostedRelations, getLinkedMemoryIds, saveRelation, bumpRelationWeight, diff --git a/dsh-mneme/test/client.test.js b/dsh-mneme/test/client.test.js index b4b15aa4..5ce2fdb1 100644 --- a/dsh-mneme/test/client.test.js +++ b/dsh-mneme/test/client.test.js @@ -2611,3 +2611,95 @@ test("memory drawer surfaces distillation provenance, both layers (issue #512)", assert.ok(clientSource.includes("/api/dsh-mneme/memory/provenance?id=") && clientSource.includes("setProvenance(null)"), "a failed read must clear the block instead of pretending there is no attribution"); }); + +// Issue #24 块4:被动确认让「正常触达」自动抬高边的权重——这是唯一会悄悄改变 +// 检索形状的写动作,所以它必须有一个能查的地方,否则「自动确认」与「没人看过的 +// 自动确认」在界面上没有区别。这里锁四件事: +// ① 复核区读的是那条专用端点,而不是把 ego 的边自己过滤一遍(抬高量 gain 只有 +// 后端知道——它要和来源基线比对,前端手里只有当前 weight); +// ② 边侧卡必须同时显示 weight 和 source:只看 weight 会把「tag 边被抬到 0.5」 +// 误读成「比 manual 边弱」,而 0.3→0.5 恰恰是被用出来的证据; +// ③ 闸关着时要说「闸关着」,而不是显示「还没有边被抬过」——后者让人以为系统 +// 在工作但没数据,前者才说明是开关没开; +// ③c 空表文案也不能断言「从未抬过权」:端点只列「权重高于其来源基线」的边, +// 一条人工边以 0.5 建立、被抬到 0.6 仍然低于基线 1.0,它确实被抬过却不在表里。 +// 文案只能描述「当前筛选结果为空」(评审第四轮指出)。 +// ④ 收起时不发请求(默认收起是刻意的:这是审计视角,不是日常视角)。 +test("entity graph surfaces the passive-confirmation review surface (issue #24)", () => { + for (const key of [ + "memory.graph.boosted.title", + "memory.graph.boosted.hint", + "memory.graph.boosted.empty", + "memory.graph.boosted.loadFailed", + "memory.graph.boosted.gateOff", + "memory.graph.boosted.show", + "memory.graph.boosted.hide", + "memory.graph.boosted.gain", + "memory.graph.weight", + "memory.graph.source", + "memory.graph.src.manual", + "memory.graph.src.confirmed", + "memory.graph.src.tag", + "memory.graph.src.llm" + ]) { + const defined = clientSource.split(`"${key}":`).length - 1; + assert.equal(defined, 2, `i18n key ${key} must be defined exactly once in zh and once in en (got ${defined})`); + } + + // ① 专用端点 + assert.ok(clientSource.includes("/api/dsh-mneme/semantic/graph/boosted-relations?limit="), + "the review surface must read the boosted-relations endpoint"); + + // ② 边侧卡:weight 与 source 一起出现,且 source 走字典而不是直接上屏原词 + assert.ok(clientSource.includes("memory.graph.weight\").replace(\"{w}\""), + "the edge side card must render the weight"); + assert.ok(clientSource.includes("memory.graph.src.${selected.edge.source}"), + "the edge side card must localize the source, not print the raw enum"); + + // ③ 闸关与空表是两句不同的话。 + // 断言必须落在「闸关分支之后的那一小段」而不是全文件 contains:gateOff 这个键 + // 在字典定义里本来就有一次,全文件搜一遍,把渲染处换成 empty 也照样命中—— + // 本条用例的首版变异自证就是这么翻车的。 + const gateIdx = clientSource.indexOf("boosted.passiveConfirmEnabled !== true"); + assert.ok(gateIdx > 0, "the off-gate branch must be driven by the server's passiveConfirmEnabled, not guessed"); + const gateBranch = clientSource.slice(gateIdx, gateIdx + 700); + assert.ok(gateBranch.includes("memory.graph.boosted.gateOff"), + "a closed gate must say so instead of looking like an empty table"); + // 闸关时既有的提升仍然要列出来(评审指出:关掉闸只表示今后不再自动抬, + // 隐藏整张表会让人永远没法复核已经发生过的提升)。 + assert.ok(/off\s*&&\s*h\([^)]*gateOff/.test(gateBranch), + "the off-gate notice must be rendered alongside the list, not as a replacement for it"); + assert.ok(gateBranch.includes("items.length === 0"), + "the empty-table copy must be decided by item count, independently of the gate"); + // 空表提示本身也不能被闸门状态挡住:闸关且一条都没有时,光说「下面列的是此前 + // 已经抬过的边」却不给任何行,读起来像这一屏坏了(评审第二轮指出)。 + assert.ok(!/items\.length === 0\s*\?\s*\(!\s*off/.test(gateBranch), + "the empty-table copy must render even when the gate is closed"); + assert.ok(!/off\s*\?[^:]*gateOff[^:]*:\s*\(boosted\.items/.test(gateBranch), + "the gate must not short-circuit the list branch"); + + // ③c 空表文案只描述「当前筛选为空」,不能断言「从未有边被抬过」:端点按 + // gain > 0 筛,而 gain 是相对来源基线算的——manual 边 0.5→0.6 涨了,却仍低于 + // 基线 1.0,不在表里。说「还没抬过」就是事实错误。 + for (const lang of ["zh", "en"]) { + const re = new RegExp(`"memory\\.graph\\.boosted\\.empty": "([^"]*)"`, "g"); + const all = [...clientSource.matchAll(re)].map((m) => m[1]); + assert.equal(all.length, 2, `empty copy must exist in both languages (${lang})`); + for (const copy of all) { + assert.ok(!/还没|从未|nothing raised|no edge has been raised/i.test(copy), + `empty copy must not claim nothing was ever raised: ${copy}`); + assert.ok(/筛选|filter|筛选下|只列|only edges/i.test(copy), + `empty copy must describe the current filter instead: ${copy}`); + } + } + + // ③b 加载失败不能被说成「闸关」或「没有数据」——那会把接口故障误导成配置问题 + const failIdx = clientSource.indexOf("boosted.error === true"); + assert.ok(failIdx > 0, "the panel must distinguish a failed fetch from a closed gate"); + assert.ok(clientSource.slice(failIdx, failIdx + 160).includes("memory.graph.boosted.loadFailed"), + "a failed fetch must render its own copy"); + + // ④ 收起时不取数 + assert.ok(clientSource.includes("if (boostedOpen) loadBoosted();"), + "the review surface must not fetch while collapsed"); +}); diff --git a/dsh-mneme/test/graph-boosted-relations.test.js b/dsh-mneme/test/graph-boosted-relations.test.js new file mode 100644 index 00000000..2a0f9ad5 --- /dev/null +++ b/dsh-mneme/test/graph-boosted-relations.test.js @@ -0,0 +1,189 @@ +// issue #24 块4(被动确认)的复核出口: +// GET /api/dsh-mneme/semantic/graph/boosted-relations +// 复核的对象是「权重被抬过的边」——抬升是唯一会改变检索形状的动作,所以它自己 +// 就是证据,不必再开一张记账表。这里同时钉住 ego 边的 weight/source 形状: +// 少了那两个字段,面板就分不清「这条边被用过」和「这条边从没被用过」。 +import test from "node:test"; +import assert from "node:assert/strict"; +import { EventEmitter } from "node:events"; +import { createStore } from "../src/store.js"; +import { createService } from "../src/service.js"; +import { createApi } from "../src/api.js"; +import { createSettings } from "../src/settings.js"; + +class FakeRes extends EventEmitter { + constructor() { super(); this.statusCode = 200; this.body = ""; } + writeHead(code, headers) { this.statusCode = code; this.headers = headers; return this; } + end(text) { this.body = text ?? ""; this.emit("end"); return this; } +} + +function req(path) { + const r = new EventEmitter(); + r.url = path; + r.method = "GET"; + r.headers = {}; + return r; +} + +function setup({ config = {}, apiToken = "" } = {}) { + const store = createStore(":memory:"); + const service = createService({ store, mirror: null, config }); + const settings = createSettings(store.db); + const commands = { add: () => {}, remove: () => {}, list: () => [] }; + const routes = []; + const ctx = { webServer: { register(route) { routes.push(route); return () => {}; } } }; + createApi(ctx, service, settings, commands, null, undefined, apiToken, config); + return { store, service, routes }; +} + +const BOOSTED = "/api/dsh-mneme/semantic/graph/boosted-relations"; + +test("boosted-relations route is registered and open without apiToken", async () => { + const { routes } = setup({ apiToken: "secret-token" }); + const route = routes.find((r) => r.path === BOOSTED); + assert.ok(route, "route must be registered"); + const res = new FakeRes(); + await route.handler(req(BOOSTED), res); + assert.equal(res.statusCode, 200, "read-only route must not demand a token"); +}); + +test("boosted-relations lists only edges whose weight rose above its source baseline", async () => { + const { store, service, routes } = setup(); + const a = service.createEntity({ name: "A", type: "concept" }); + const b = service.createEntity({ name: "B", type: "concept" }); + const c = service.createEntity({ name: "C", type: "concept" }); + // tag 边初值 0.3,抬两次 → 0.5,gain 0.2,必进表。 + const boosted = service.saveRelation({ + from_entity: a.id, to_entity: b.id, relation_type: "uses", source: "tag" + }); + store.bumpRelationWeight(boosted.id, 0.1); + store.bumpRelationWeight(boosted.id, 0.1); + // manual 边留在初值 1.0:gain 为 0,不得进表。 + service.saveRelation({ from_entity: b.id, to_entity: c.id, relation_type: "uses", source: "manual" }); + + const res = new FakeRes(); + await routes.find((r) => r.path === BOOSTED).handler(req(BOOSTED), res); + assert.equal(res.statusCode, 200); + const data = JSON.parse(res.body); + assert.equal(data.count, 1, "only the raised edge qualifies"); + const item = data.items[0]; + assert.equal(item.id, boosted.id); + assert.equal(item.from, a.id); + assert.equal(item.to, b.id); + assert.equal(item.from_name, "A", "both endpoint names ride along so the panel need not re-query"); + assert.equal(item.to_name, "B"); + assert.equal(item.source, "tag"); + assert.ok(Math.abs(item.base_weight - 0.3) < 1e-9, "baseline comes from RELATION_SOURCE_DEFAULTS"); + assert.ok(Math.abs(item.weight - 0.5) < 1e-9, `weight rose to 0.5, got ${item.weight}`); + assert.ok(Math.abs(item.gain - 0.2) < 1e-9, `gain = weight − base, got ${item.gain}`); +}); + +test("boosted-relations reports an empty table honestly, with the gate that explains why", async () => { + const { service, routes } = setup(); + const a = service.createEntity({ name: "A", type: "concept" }); + const b = service.createEntity({ name: "B", type: "concept" }); + service.saveRelation({ from_entity: a.id, to_entity: b.id, relation_type: "uses", source: "llm" }); + + const res = new FakeRes(); + await routes.find((r) => r.path === BOOSTED).handler(req(BOOSTED), res); + const data = JSON.parse(res.body); + assert.equal(data.count, 0, "an unraised edge is not a finding"); + assert.equal(data.passiveConfirmEnabled, false, "the gate is off by default — the panel can say so"); +}); + +test("boosted-relations surfaces passiveConfirmEnabled=true when the gate is on", async () => { + const { routes } = setup({ config: { graphPassiveConfirm: true, graphWeightEnabled: true } }); + const res = new FakeRes(); + await routes.find((r) => r.path === BOOSTED).handler(req(BOOSTED), res); + assert.equal(JSON.parse(res.body).passiveConfirmEnabled, true); +}); + +// 只有 graphPassiveConfirm 开着不算「会继续产生新提升」:touchRecalled 里 +// passive = graphPassiveConfirm && graphWeightEnabled,只看前者面板会说 +// 「会自动提升」而实际一条都不会抬(评审指出)。 +test("boosted-relations reports the gate as off when only graphPassiveConfirm is set", async () => { + const { routes } = setup({ config: { graphPassiveConfirm: true, graphWeightEnabled: false } }); + const res = new FakeRes(); + await routes.find((r) => r.path === BOOSTED).handler(req(BOOSTED), res); + assert.equal(JSON.parse(res.body).passiveConfirmEnabled, false, + "passive confirmation only bites when weight evolution is also on"); +}); + +// 排序口径:按 gain 降序,不是按 weight 降序。不同 source 的 base 不同, +// 所以 weight 大的边 gain 可能更小。首版把排序交给 SQL 的 weight DESC 并在 +// 循环里按 limit 提前 break,截断处丢掉的正是 gain 更大的边(评审指出)。 +test("boosted-relations ranks by gain, not by raw weight", async () => { + const { store, routes } = setup(); + const a = store.createEntity({ name: "A", type: "technology" }); + const b = store.createEntity({ name: "B", type: "technology" }); + const c = store.createEntity({ name: "C", type: "technology" }); + // llm 边:base 0.4,weight 0.95 → gain 0.55 + const llmEdge = store.saveRelation({ from_entity: a.id, to_entity: b.id, relation_type: "uses", source: "llm" }); + store.bumpRelationWeight(llmEdge.id, 0.55); + // tag 边:base 0.3,weight 0.90 → gain 0.60(weight 更小、gain 更大) + const tagEdge = store.saveRelation({ from_entity: a.id, to_entity: c.id, relation_type: "uses", source: "tag" }); + store.bumpRelationWeight(tagEdge.id, 0.60); + + const res = new FakeRes(); + await routes.find((r) => r.path === BOOSTED).handler(req(BOOSTED), res); + const items = JSON.parse(res.body).items; + assert.ok(items.length >= 2); + assert.ok(items[0].gain >= items[1].gain, "the list must be ordered by gain"); + assert.equal(items[0].to_name, "C", + "the tag edge (weight 0.90, gain 0.60) must outrank the llm edge (weight 0.95, gain 0.55)"); +}); + +// 截断也必须发生在按 gain 排完之后:limit=1 时留下的得是 gain 最大那条。 +test("boosted-relations truncates after ranking by gain", async () => { + const { store, routes } = setup(); + const a = store.createEntity({ name: "A", type: "technology" }); + const b = store.createEntity({ name: "B", type: "technology" }); + const c = store.createEntity({ name: "C", type: "technology" }); + const llmEdge = store.saveRelation({ from_entity: a.id, to_entity: b.id, relation_type: "uses", source: "llm" }); + store.bumpRelationWeight(llmEdge.id, 0.55); + const tagEdge = store.saveRelation({ from_entity: a.id, to_entity: c.id, relation_type: "uses", source: "tag" }); + store.bumpRelationWeight(tagEdge.id, 0.60); + + const res = new FakeRes(); + await routes.find((r) => r.path === BOOSTED).handler(req(BOOSTED + "?limit=1"), res); + const items = JSON.parse(res.body).items; + assert.equal(items.length, 1); + assert.equal(items[0].to_name, "C", + "truncation must keep the largest gain, not the largest weight"); +}); + +test("boosted-relations honours limit and never exceeds the hard cap", async () => { + const { store, service, routes } = setup(); + for (let i = 0; i < 6; i++) { + const a = service.createEntity({ name: `L${i}a`, type: "concept" }); + const b = service.createEntity({ name: `L${i}b`, type: "concept" }); + const r = service.saveRelation({ from_entity: a.id, to_entity: b.id, relation_type: "x", source: "tag" }); + store.bumpRelationWeight(r.id, 0.1); + } + const route = routes.find((r) => r.path === BOOSTED); + let res = new FakeRes(); + await route.handler(req(`${BOOSTED}?limit=2`), res); + assert.equal(JSON.parse(res.body).count, 2, "limit caps the returned rows"); + res = new FakeRes(); + await route.handler(req(`${BOOSTED}?limit=9999`), res); + assert.equal(JSON.parse(res.body).count, 6, "an absurd limit clamps to the cap, it does not error"); +}); + +// ego 的边要带 weight/source:面板靠它区分「这条边被用过」与「从没被用过」。 +test("ego edges carry weight and source", async () => { + const { service, routes } = setup(); + const a = service.createEntity({ name: "A", type: "concept" }); + const b = service.createEntity({ name: "B", type: "concept" }); + const rel = service.saveRelation({ from_entity: a.id, to_entity: b.id, relation_type: "uses", source: "llm" }); + + const res = new FakeRes(); + await routes + .find((r) => r.path === "/api/dsh-mneme/semantic/graph/ego") + .handler(req("/api/dsh-mneme/semantic/graph/ego?entity=A&depth=1"), res); + assert.equal(res.statusCode, 200); + const data = JSON.parse(res.body); + const edge = data.edges.find((e) => e.id === rel.id); + assert.ok(edge, "the relation must appear among ego edges"); + assert.ok(Math.abs(edge.weight - 0.4) < 1e-9, `llm edge starts at 0.4, got ${edge.weight}`); + assert.equal(edge.source, "llm"); +}); diff --git a/dsh-mneme/test/graph-multipath.test.js b/dsh-mneme/test/graph-multipath.test.js new file mode 100644 index 00000000..986670ac --- /dev/null +++ b/dsh-mneme/test/graph-multipath.test.js @@ -0,0 +1,269 @@ +// issue #24 块1 三路起点的回归(tag 桥接 / 向量 / 零分种子)。 +// 这三条都在钉「调用点真的接上了」:能力早就写在 graph/anchoring.js 里, +// 之前的病是 service.js 只喂了一路 exact,另两路永远等不到自己被调用。 +import test from "node:test"; +import assert from "node:assert/strict"; +import { createStore } from "../src/store.js"; +import { createService } from "../src/service.js"; +import { createVectorIndex } from "../src/vector-index.js"; +import { HOP_QUOTA } from "../src/graph/anchoring.js"; + +function mockEmbedder(vec = [1, 0, 0, 0]) { + return { + embedSingle: async () => vec, + embed: async (texts) => texts.map(() => vec), + schedule: () => {}, + modelHash: "mock#abc", + dimension: 4 + }; +} + +function makeService(config = {}, { withEmbedder = false } = {}) { + const store = createStore(":memory:"); + const service = createService({ store, mirror: null, config }); + if (withEmbedder) { + const vectorIndex = createVectorIndex({ store }); + service.setEmbedder(mockEmbedder()); + service.setVectorIndex(vectorIndex); + return { store, service, vectorIndex }; + } + return { store, service, vectorIndex: null }; +} + +function putMemory(store, service, { type = "project", title, content, tags = [], importance = 3 }) { + return service.saveWithDedupe({ type, title, content, tags, importance }).memory; +} + +function linkAttr(store, entityName, memory, { key = "k", value = "v", type = "technology" } = {}) { + let entity = store.findEntityByName(entityName); + if (!entity) entity = store.createEntity({ name: entityName, type }); + store.saveAttr({ entity_id: entity.id, attr_key: key, attr_value: value, memory_id: memory.id }); + return entity; +} + +function linkEntities(store, fromName, toName, { relType = "uses", type = "technology" } = {}) { + const a = store.findEntityByName(fromName) ?? store.createEntity({ name: fromName, type }); + const b = store.findEntityByName(toName) ?? store.createEntity({ name: toName, type }); + store.saveRelation({ from_entity: a.id, to_entity: b.id, relation_type: relType }); + return { a, b }; +} + +// ---- Tag 桥接 ------------------------------------------------------------- +// 查询「数据库迁移」既不是任何实体名,也不含实体名的子串。tag 命中一条记忆后, +// 该记忆挂联的实体就是「这次在谈的东西」,沿它级联出去的邻居记忆才有机会进池。 +// 判别点:farMem 只能经 PgTuner → PgCache 这条边到达;tag 路不接线时它必然缺席。 +test("tag path bridges a tag hit into an entity seed (query names no entity)", async () => { + const { store, service } = makeService({ entityRecallEnabled: true, graphAnchoringEnabled: true }); + // nearMem 带 tag,标题正文都不含查询词——但 store.search 的 tags LIKE 也会命中它, + // 所以它的存在不是本条用例的判别点,判别点是 farMem。 + const nearMem = putMemory(store, service, { + title: "alpha", content: "beta gamma", tags: ["数据库迁移"] + }); + const farMem = putMemory(store, service, { title: "delta", content: "epsilon zeta" }); + linkAttr(store, "PgTuner", nearMem); + linkAttr(store, "PgCache", farMem); + linkEntities(store, "PgTuner", "PgCache"); + + const rows = await service.searchMemories("数据库迁移", { mode: "hybrid" }); + const ids = rows.map((r) => r.id); + assert.ok(ids.includes(farMem.id), "1-hop memory reached only via the tag-bridged seed"); + const far = rows.find((r) => r.id === farMem.id); + assert.ok( + Math.abs(far.score - 1.0 * HOP_QUOTA[1] * 0.3) < 1e-9, + `hop-quota weighting preserved through the tag path, got ${far.score}` + ); +}); + +// 关锚定时 tag 路一次都不许查:否则「默认关=旧单跳轴逐字节不变」这条承诺就破了。 +test("tag path is not consulted when anchoring is off", async () => { + const { store, service } = makeService({ entityRecallEnabled: true }); + const nearMem = putMemory(store, service, { + title: "alpha", content: "beta gamma", tags: ["数据库迁移"] + }); + const farMem = putMemory(store, service, { title: "delta", content: "epsilon zeta" }); + linkAttr(store, "PgTuner", nearMem); + linkAttr(store, "PgCache", farMem); + linkEntities(store, "PgTuner", "PgCache"); + + const rows = await service.searchMemories("数据库迁移", { mode: "hybrid" }); + assert.ok(!rows.some((r) => r.id === farMem.id), "no cascade when anchoring is off"); +}); + +// ---- 向量起点 ------------------------------------------------------------- +// 查询里没有实体名、也没有 tag 命中,唯一能给出种子的是向量。早退条件若仍以 +// 「名字没命中」收场,第三路接上了也永远不被调用——这条用例正是钉那个早退。 +test("vector path seeds from near memories when the query names no entity", async () => { + const { store, service, vectorIndex } = makeService( + { entityRecallEnabled: true, graphAnchoringEnabled: true }, + { withEmbedder: true } + ); + const nearMem = putMemory(store, service, { title: "alpha", content: "beta gamma" }); + const farMem = putMemory(store, service, { title: "delta", content: "epsilon zeta" }); + vectorIndex.saveEmbedding(nearMem.id, [1, 0, 0, 0]); // farMem 刻意不嵌:它只能经图轴到达 + linkAttr(store, "VecSeed", nearMem); + linkAttr(store, "VecNeighbor", farMem); + linkEntities(store, "VecSeed", "VecNeighbor"); + + const rows = await service.searchMemories("omega", { mode: "hybrid" }); + const ids = rows.map((r) => r.id); + assert.ok(ids.includes(farMem.id), "1-hop memory reached through the vector seed"); + const far = rows.find((r) => r.id === farMem.id); + assert.ok( + Math.abs(far.score - HOP_QUOTA[1] * 0.3) < 1e-9, + `vector seed keeps hop-quota weighting, got ${far.score}` + ); +}); + +// ---- 向量起点也要看关系端点 ----------------------------------------------- +// 与 Tag 路同一条病:一条近邻记忆若只通过关系挂到实体上(phaseRelations 不建 +// attr 行),只读 entity_attrs 的反查会漏掉它,向量路就凭空少了种子。 +test("vector path seeds from relation-only memories, not just attr rows", async () => { + const { store, service, vectorIndex } = makeService( + { entityRecallEnabled: true, graphAnchoringEnabled: true }, + { withEmbedder: true } + ); + const nearMem = putMemory(store, service, { title: "alpha", content: "beta gamma" }); + const farMem = putMemory(store, service, { title: "delta", content: "epsilon zeta" }); + vectorIndex.saveEmbedding(nearMem.id, [1, 0, 0, 0]); + // nearMem 不写 attr,只通过一条关系挂到 RelSeed / RelNeighbor 上。 + const seed = store.createEntity({ name: "RelSeed", type: "module" }); + const neighbor = store.createEntity({ name: "RelNeighbor", type: "module" }); + store.saveRelation({ + from_entity: seed.id, to_entity: neighbor.id, relation_type: "uses", + memory_id: nearMem.id, metadata: { source: "sleep_relation_completion" } + }); + linkAttr(store, "RelNeighbor", farMem, { key: "k2" }); + + const rows = await service.searchMemories("omega", { mode: "hybrid" }); + assert.ok(rows.some((r) => r.id === farMem.id), + "a relation-only near memory must still produce vector seeds"); +}); + +// 对照:没有 embedder 就没有向量路,三路全空时必须老实返回空,而不是凭空造种子。 +test("vector path absent leaves the axis empty (no phantom seeds)", async () => { const { store, service } = makeService({ entityRecallEnabled: true, graphAnchoringEnabled: true }); + const nearMem = putMemory(store, service, { title: "alpha", content: "beta gamma" }); + const farMem = putMemory(store, service, { title: "delta", content: "epsilon zeta" }); + linkAttr(store, "VecSeed", nearMem); + linkAttr(store, "VecNeighbor", farMem); + linkEntities(store, "VecSeed", "VecNeighbor"); + + const rows = await service.searchMemories("omega", { mode: "hybrid" }); + assert.ok(!rows.some((r) => r.id === farMem.id), "no seed sources → no cascade"); +}); + +// ---- 零分种子 ------------------------------------------------------------- +// min-max 会把连续通路里**最弱的那条**压成 0。0 分种子仍会被 anchorSeeds 收下, +// 而下游给种子挂联记忆的是**固定满权 tier**(不看种子自己的分数)——于是「这一路 +// 里最不像的那条」会以最强的姿态把记忆拉进池。 +// +// 两条近邻记忆都必须拿到**正分**:若让第二条与查询正交(cos=0),它在进向量 +// 路时就被 score > 0 挡掉了,min-max 根本轮不到把它压成 0——那样这条用例会 +// 以「从没发生过」的方式通过,看着绿、其实什么都没验。 +// +// zeroMem 必须**挂在 0 分种子自己身上**(VoZero 同时挂 nearB 与 zeroMem)。 +// 首版把它挂在另一个孤立实体上,于是无论滤不滤 0 分种子它都到不了——变异自证 +// 当场抓到那条用例是空转的。 +test("a zero-score seed does not inject its linked memory at full tier", async () => { + const { store, service, vectorIndex } = makeService( + { entityRecallEnabled: true, graphAnchoringEnabled: true }, + { withEmbedder: true } + ); + const nearA = putMemory(store, service, { title: "alpha", content: "beta" }); + const nearB = putMemory(store, service, { title: "gamma", content: "delta" }); + const zeroMem = putMemory(store, service, { title: "iota", content: "kappa" }); + vectorIndex.saveEmbedding(nearA.id, [1, 0, 0, 0]); // cos = 1.0 + vectorIndex.saveEmbedding(nearB.id, [0.7, 0.7, 0, 0]); // cos ≈ 0.707,正分但更低 → min-max 压成 0 + linkAttr(store, "VoSeed", nearA); + linkAttr(store, "VoZero", nearB); + linkAttr(store, "VoZero", zeroMem, { key: "k2" }); // 同一实体挂第二条记忆,只能经这颗 0 分种子到达 + // VoSeed 与 VoZero 之间不建边:zeroMem 拿不到任何别的到达路径。 + + const rows = await service.searchMemories("omega", { mode: "hybrid" }); + assert.ok(rows.some((r) => r.id === nearB.id), "the weak memory itself still comes back via vector"); + assert.ok( + !rows.some((r) => r.id === zeroMem.id), + "a 0-score seed must not pull its linked memory in at full tier" + ); +}); + +// ---- Tag 桥接的排序 ------------------------------------------------------- +// 「一个实体命中了几条记忆」是这条路的强弱信号,必须在 SQL 里数。 +// 首版拿 entityIdsForMemories 的 DISTINCT 结果反推计数,每个实体恒为 1, +// 排序静默失效、只剩名字序(评审指出)。判别点:limit=1 时返回的是宽实体。 +// 命名上必须让「宽实体」在字母序里排后面——首版把宽实体叫 AWide,于是字母序 +// 也选它,用例空转通过(变异自证抓到的)。 +test("findEntitiesByTags ranks entities by how many tagged memories they cover", async () => { + const { store, service } = makeService(); + const m1 = putMemory(store, service, { title: "m1", content: "c1", tags: ["数据库迁移"] }); + const m2 = putMemory(store, service, { title: "m2", content: "c2", tags: ["数据库迁移"] }); + // ZWide 挂两条命中记忆;ANarrow 只挂一条。名字序让 ANarrow 排前面, + // 所以只要计数生效,返回的必然是 ZWide。 + linkAttr(store, "ZWide", m1); + linkAttr(store, "ZWide", m2, { key: "k2" }); + linkAttr(store, "ANarrow", m1); + + const hits = store.findEntitiesByTags("数据库迁移", { limit: 1 }); + assert.equal(hits.length, 1); + assert.equal(hits[0].name, "ZWide", + "the entity covering more tagged memories must win over the alphabetical order"); +}); + +// ---- Tag 锚点的两个来源 --------------------------------------------------- +// phaseRelations 只写 entity_relations.memory_id,不建 attr 行。所以「只通过关系 +// 挂到实体上」的记忆,若锚点只读 entity_attrs,关系两端一个种子都生不出来。 +// 判别点:这两个实体都没有任何 attr 行,只能从关系侧被看到。 +test("findEntitiesByTags sees relation endpoints, not only attr rows", () => { + const { store, service } = makeService(); + const mem = putMemory(store, service, { title: "pg 调优", content: "一次调优", tags: ["postgres"] }); + const a = store.createEntity({ name: "PgTuner", type: "module" }); + const b = store.createEntity({ name: "PgCache", type: "module" }); + store.saveRelation({ + from_entity: a.id, to_entity: b.id, relation_type: "uses", + memory_id: mem.id, metadata: { source: "sleep_relation_completion" } + }); + + const names = store.findEntitiesByTags("postgres", { limit: 5 }).map((h) => h.name).sort(); + assert.deepEqual(names, ["PgCache", "PgTuner"], + "a memory linked only through a relation must seed both endpoints"); +}); + +// 同一 (实体, 记忆) 被 attr 与关系同时覆盖时只算一条:UNION 去的是对,不是行。 +test("findEntitiesByTags counts a memory once even when attr and relation both cover it", () => { + const { store, service } = makeService(); + const m1 = putMemory(store, service, { title: "a", content: "b", tags: ["缓存"] }); + const m2 = putMemory(store, service, { title: "c", content: "d", tags: ["缓存"] }); + const wide = linkAttr(store, "ZBoth", m1); + store.saveRelation({ from_entity: wide.id, to_entity: wide.id, relation_type: "related_to", memory_id: m1.id }); + linkAttr(store, "ABoth", m2); + + const hits = store.findEntitiesByTags("缓存", { limit: 2 }); + assert.deepEqual(hits.map((h) => h.name).sort(), ["ABoth", "ZBoth"]); + // 两边都是 1 条记忆时按名字序;若 m1 被数成 2 条,ZBoth 会排到前面。 + assert.equal(hits[0].name, "ABoth", + "duplicate coverage of the same memory must not inflate the count"); +}); + +// ---- 分块后计数仍然正确 --------------------------------------------------- +// TAG_ID_CHUNK = 500,这条用例造 600 条命中记忆强制走两块:ZBulk 每块各 300 条 +// (合计 600),ABulk 只有 400 条(全在第一块)。判别点:正确累加时 ZBulk 胜出; +// 若第二块覆盖而不是累加,ZBulk 只剩 300 < 400,字母序靠前的 ABulk 会赢。 +test("findEntitiesByTags accumulates counts across chunk boundaries", () => { + const { store } = makeService(); + const zBulk = store.createEntity({ name: "ZBulk", type: "module" }); + const aBulk = store.createEntity({ name: "ABulk", type: "module" }); + const makeTagged = (i) => store.save({ + type: "project", title: `t${i}`, content: `c${i}`, tags: ["分块测试"], importance: 3 + }); + // 600 条:前 300 条挂 ZBulk,后 300 条挂 ZBulk 的另一批 attr key, + // 其中前 400 条同时挂 ABulk(跨两块)。 + for (let i = 0; i < 600; i++) { + const m = makeTagged(i); + store.saveAttr({ entity_id: zBulk.id, attr_key: `k${i % 2}`, attr_value: "v", memory_id: m.id }); + if (i < 400) store.saveAttr({ entity_id: aBulk.id, attr_key: "k", attr_value: "v", memory_id: m.id }); + } + + const hits = store.findEntitiesByTags("分块测试", { limit: 1 }); + assert.equal(hits.length, 1); + assert.equal(hits[0].name, "ZBulk", + "counts must be summed across chunks, not overwritten by the last batch"); +});