diff --git a/.mex/ROUTER.md b/.mex/ROUTER.md index 548f924d..599ce71e 100644 --- a/.mex/ROUTER.md +++ b/.mex/ROUTER.md @@ -18,7 +18,7 @@ edges: condition: when refreshing the release README, badges, community links, or architecture illustrations - target: patterns/hub-first-run-onboarding.md condition: when adding or changing the Hub first-run tour or its checkout-local completion state -last_updated: 2026-09-28 +last_updated: 2026-10-03 --- # Session Bootstrap @@ -385,10 +385,12 @@ Then read this file fully before doing anything else in this session. appear only when a stable status observation makes the requested operation safe; migration-required or unstable Wiki observations never fabricate a repair action. -- The Wiki CLI's `serviceOptions` carries no code graph, so `wiki validate` - cannot resolve a grounding and `wiki migrate`'s body-hash backfill never runs. - Both degrade silently rather than failing; the validate notice now reports - that the pass had no graph instead of asserting the checkout has none. +- The Wiki CLI's graph-aware commands (validate, rebuild-index, migrate, apply, + regenerate-views) reach the graph only through the repository graph port's + fresh snapshot (#232). A stale or changing graph is not used: the command + runs without it and reports `CODE_GRAPH_UNAVAILABLE`, so `mex graph` must be + current for Wiki grounding verdicts. Synthesis (`wiki build/prepare/propose`) + still opens `graph.db` directly, without that freshness proof. ## Routing Table diff --git a/CHANGELOG.md b/CHANGELOG.md index 827ecc3a..e4cde68f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,12 +20,19 @@ All notable changes to this project will be documented in this file. - `STALE_FILE` no longer counts every repository commit. Its commit signal now counts only commits made since the file last changed that touched a path the file references: its path claims, and the files of the code it grounds (committed `file`, else the node's file in the graph, plus inline `mex://` anchors). A scaffold file that references no paths is judged by its age and `last_updated` alone. Paths that are scaffold knowledge, URLs, and anything that would match the whole repository are not counted. `SETUP.md`, `SYNC.md`, `patterns/README.md` and `patterns/INDEX.md` are text mex ships, kept current by `mex pattern add` and the index-sync check, so they are no longer reported. On mex's own scaffold, `STALE_FILE` findings fell from 16 to 1. The previous counts ran from 50 to 289 commits, and the same files now count 0 to 6 commits to their referenced paths. The day and commit thresholds are unchanged (#237). ### Fixed +- The Wiki commands now use the code graph. `mex wiki validate` always reported `codeGraphAvailable: false` and resolved no grounding, `mex wiki rebuild-index` stored no grounding health, so `--health changed` matched nothing, `mex wiki migrate` never backfilled a `bodyHash`, and `mex wiki apply` could not ground. All of this happened silently, even in a checkout with a fresh `graph.db`. `validate`, `rebuild-index`, `migrate`, `apply` and `regenerate-views` now reach the graph the way the Project Hub does: through one read-only snapshot per command, proved fresh again before anything derived from it is reported. `rebuild-index` publishes its health only after that proof. A graph that is stale, incompatible, corrupt or changing during the pass is not used; the command does its work without it and adds the info notice `CODE_GRAPH_UNAVAILABLE`, which says why and names `mex graph`. A checkout with no graph stays as quiet as before. Three resolution faults that the missing graph had hidden are fixed with it: + - Migration's backfill now writes the body hash the grounding was made against: the baseline `mex ground` cached when its fingerprint is the committed one, otherwise the current body only while the fingerprint still matches, otherwise none. It no longer adopts code that may have drifted. + - A grounding whose node is reached through a re-minted id (#240), or rebound after a move, is fresh when its body is byte-identical. + - A grounding with no `bodyHash` is judged by its code tokens, not by the ids of its callers and callees. + + On a populated Hono scaffold, `wiki validate` and the index went from resolving nothing to the same verdicts as `mex check`: 7 groundings fresh, and both groundings of an edited method changed. Without the last two fixes, 2 entities there and 3 on a second scaffold built before #240 read changed with their code unchanged (#232). - `mex wiki query` now answers natural-language questions. It ANDed every word of the query, stop words included, so "Why does Hono avoid runtime dependencies?" required `why`, `does` and `avoid` and found nothing, although an entity titled *Zero runtime dependencies, Web Standards only* exists. Stop words and question words are now dropped before the every-term search, which otherwise works as before. The rest of the page is filled with entities matching some of the terms, ranked by how much of the query they cover (each term's inverse document frequency, weighted by title, summary or body) and found through a conservative stem, so `dependencies` meets `dependency`. Such an entity needs at least two distinctive terms, and a term found in most of the wiki, usually the project's own name, does not count as one. Within a field, hits are now ordered by relevance (FTS5 `bm25`) after lifecycle and health rather than by title, so the entity most about the words comes first; title still breaks ties. Each hit carries `match` (`all_terms` or `some_terms`) and, for a broader hit, `matchedTerms`; the result carries `terms`, `ignoredTerms` and `unmatchedTerms`. A broader answer adds the info notice `WIKI_QUERY_PARTIAL_MATCH`, and an empty one adds `WIKI_QUERY_NO_MATCH`, which names the terms nothing mentions and suggests keywords, `ROUTER.md` or `mex graph scope` instead of leaving a bare empty list; neither changes the exit code. Hub search uses the same plan and order, with every all-terms hit ahead of the broader ones. On the two evaluation scaffolds, with their unindexed knowledge adopted (#227), natural-language hit@5 went from 0.00 to 0.92 (Hono) and 0.77 (mex), keyword hit@5 to 1.00 on both, keyword hit@1 from 0.50 to 0.75 and from 0.69 to 0.85, exact titles stayed at 1.00, and all six off-topic evaluation queries still return nothing. Of 24 further off-topic questions, 6 return broader hits, each labelled with the terms it matched and the terms nothing does (#235). - A refresh that must extract every TypeScript and JavaScript file in full no longer mis-indexes files outside every `tsconfig`. When such a change reached the program built for them (for example a new plain script, whose declarations the other files can see), the signal to extract in full was taken for a crashed program, and every one of those files was indexed by the fallback parser with different node ids and edges until the next rebuild. On FastAPI, a jump over 80 commits that added one documentation script left 48 nodes and 78 edges different from a clean build; it now matches exactly (#209). - `mex check` no longer reports two false positives seen on clean scaffolds. A brace-expanded script name such as `bun run test:{node,workerd}` is expanded and each script it names is checked against `package.json`; a name that still holds braces after expansion is a placeholder and is skipped, as is one naming more than 64 scripts. A command written in code inside a dependency entry, such as `` **`bun test` / `deno test`** ``, is no longer claimed as a dependency, since a package name never contains whitespace (#231). ### Compatibility +- `CODE_GRAPH_UNAVAILABLE` is a new Wiki diagnostic code, at info severity. `wiki validate` can now report `GROUNDING_STALE`, `GROUNDING_MISSING` and `GROUNDING_UNRESOLVED` on scaffolds where it reported none, all at warning severity, so its exit status is unchanged. The index and graph schemas are unchanged; run `mex wiki rebuild-index` once to record health in an existing index (#232). - Earlier MEX versions cannot read `mh2:` fingerprints. On a Hono scaffold re-encoded to `mh2:`, 0.8.3 reported 10 groundings it could not reconcile as `GROUNDING_GONE`, and its Wiki rejected 11 groundings as malformed and 5 entities as unparseable, breaking 3 relations; capture also treats such a grounding as changed and asks for a review. Upgrade everyone who shares a scaffold before anyone commits new groundings. Graph and Wiki index schemas are unchanged (#233). - A root `type` key is optional and additive. Earlier versions ignore it and leave the file outside the Wiki as before; once migrated it is an ordinary `mex.type`. Adoption may now remove a root `type` key, alongside `grounds_to` and `topics`. `KNOWLEDGE_NOT_ADOPTED` and `KNOWLEDGE_UNTYPED` are new `IssueCode` values and new Wiki diagnostic codes. Graph and Wiki index schemas are unchanged (#227). - `codeHash` is an optional, additive `grounds_to` field; graph and Wiki index schemas are unchanged. Earlier versions read scaffolds that carry it, ignore it, and report every changed body as `GROUNDING_DRIFT` as before. `GROUNDING_COMMENT_DRIFT` is a new `IssueCode` value (#236). diff --git a/src/cli.ts b/src/cli.ts index 452d2076..bb7caf9e 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -613,6 +613,21 @@ function wikiIo(): import("./wiki/cli/commands.js").CommandIo { }; } +/** + * `wikiIo()` plus the code graph, for the commands that resolve groundings + * (#232): validate, rebuild-index, migrate, apply and regenerate-views. + * + * The graph is the repository graph port, the same one the Project Hub hands + * the Wiki: it opens nothing until a command asks for a snapshot, refuses one + * that is missing, stale or incompatible, and never maintains the index. The + * read-only commands keep `wikiIo()` and pay for none of it. + */ +async function groundedWikiIo(): Promise { + const base = wikiIo(); + const { createRepositoryGraphPort } = await import("./graph/application-adapter.js"); + return { ...base, groundingBridge: createRepositoryGraphPort(base.projectRoot ?? resolve(base.scaffoldRoot, "..")) }; +} + /** The §15.1 filters, added to whichever commands they apply to. */ function withReadFilters(command: Command): Command { return command @@ -671,7 +686,7 @@ wikiCommand .option("--json", "emit one enveloped JSON object") .action(async (options) => { const { runValidate } = await import("./wiki/cli/commands.js"); - runValidate(wikiIo(), options); + await runValidate(await groundedWikiIo(), options); }); withReadFilters(wikiCommand.command("graph").description("A bounded slice of the relation graph")).action( @@ -687,7 +702,7 @@ wikiCommand .option("--json", "emit one enveloped JSON object") .action(async (options) => { const { runRebuildIndex } = await import("./wiki/cli/commands.js"); - runRebuildIndex(wikiIo(), options); + await runRebuildIndex(await groundedWikiIo(), options); }); wikiCommand @@ -697,7 +712,7 @@ wikiCommand .option("--json", "emit one enveloped JSON object") .action(async (options) => { const { runRegenerateViews } = await import("./wiki/cli/commands.js"); - runRegenerateViews(wikiIo(), options); + await runRegenerateViews(await groundedWikiIo(), options); }); wikiCommand @@ -707,7 +722,7 @@ wikiCommand .option("--json", "emit one enveloped JSON object") .action(async (options) => { const { runMigrate } = await import("./wiki/cli/commands.js"); - runMigrate(wikiIo(), options); + await runMigrate(await groundedWikiIo(), options); }); wikiCommand @@ -718,7 +733,7 @@ wikiCommand .option("--json", "emit one enveloped JSON object") .action(async (file: string, options) => { const { runApply } = await import("./wiki/cli/commands.js"); - runApply(wikiIo(), file, options); + await runApply(await groundedWikiIo(), file, options); }); /** diff --git a/src/graph/__tests__/fingerprint-encoding.test.ts b/src/graph/__tests__/fingerprint-encoding.test.ts index dc2b30c4..e1bf5c38 100644 --- a/src/graph/__tests__/fingerprint-encoding.test.ts +++ b/src/graph/__tests__/fingerprint-encoding.test.ts @@ -5,6 +5,7 @@ import { createFingerprint, deserializeFingerprint, sameFingerprint, + sameFingerprintCode, serializeFingerprint, } from "../fingerprint.js"; import type { Fingerprint } from "../reconcile.js"; @@ -97,6 +98,22 @@ describe("compact fingerprint encoding (#233)", () => { } }); + it("compares code alone, setting neighbours aside, when asked whether the code changed", () => { + // Renumbered neighbours (#240) and a new caller elsewhere are not this node's code changing. + const renumbered = { ...realistic, neighbors: realistic.neighbors.map((id) => `${id.split(":")[0]}:${"0".repeat(32)}`) }; + const calledMore = { ...realistic, neighbors: [...realistic.neighbors, nodeId("function", "new-caller")] }; + for (const other of [renumbered, calledMore]) { + expect(sameFingerprintCode(legacy(realistic), serializeFingerprint(other))).toBe(true); + expect(sameFingerprint(legacy(realistic), serializeFingerprint(other))).toBe(false); + } + const minhash = [...realistic.minhash]; + minhash[0] = (minhash[0]! + 1) % 0x100000000; + for (const other of [{ ...realistic, minhash }, { ...realistic, tokenCount: realistic.tokenCount + 1 }]) { + expect(sameFingerprintCode(serializeFingerprint(realistic), serializeFingerprint(other))).toBe(false); + } + expect(sameFingerprintCode("mh2:64:not-a-fingerprint", serializeFingerprint(realistic))).toBe(false); + }); + it("compares strings that do not decode only by identity", () => { expect(sameFingerprint("not-a-fingerprint", "not-a-fingerprint")).toBe(true); expect(sameFingerprint("not-a-fingerprint", "also-not")).toBe(false); diff --git a/src/graph/fingerprint.ts b/src/graph/fingerprint.ts index f4f62fed..c147cb1d 100644 --- a/src/graph/fingerprint.ts +++ b/src/graph/fingerprint.ts @@ -160,6 +160,26 @@ export function sameFingerprint(left: string, right: string): boolean { return left === right || canonicalFingerprint(left) === canonicalFingerprint(right); } +/** + * Whether two serialized fingerprints describe the same code: the same MinHash + * signature over the same number of tokens, in either encoding. + * + * The neighbour list is left out on purpose. It names the node's callers and + * callees, so it changes when code *elsewhere* calls this node or when node + * ids are re-minted (#240) — neither of which is this node's code changing. + * Use it to answer "is this still the code?" without a body hash; identity + * questions, which neighbours help settle, keep {@link sameFingerprint}. + */ +export function sameFingerprintCode(left: string, right: string): boolean { + if (sameFingerprint(left, right)) return true; + const a = deserializeFingerprint(left); + const b = deserializeFingerprint(right); + if (a === null || b === null) return false; + return a.tokenCount === b.tokenCount + && a.minhash.length === b.minhash.length + && a.minhash.every((value, index) => value === b.minhash[index]); +} + /** Kind indexes 0–254 name a table entry; 0xff introduces a verbatim id. */ const VERBATIM_NEIGHBOR = 0xff; const NEIGHBOR_ID = /^([a-z][a-z0-9_]*):([0-9a-f]{32})$/; diff --git a/src/wiki/__tests__/diagnostic-coverage.test.ts b/src/wiki/__tests__/diagnostic-coverage.test.ts index 48f63760..9ecb9ede 100644 --- a/src/wiki/__tests__/diagnostic-coverage.test.ts +++ b/src/wiki/__tests__/diagnostic-coverage.test.ts @@ -67,6 +67,7 @@ import { planOperation } from "../operations/plan.js"; import { applyOperation } from "../operations/apply.js"; import { readAuditLog } from "../operations/audit.js"; import type { GroundingResolver } from "../index/write.js"; +import { codeGraphUnavailableDiagnostic } from "../grounding/availability.js"; /** Run `body` against a fresh scratch directory, cleaning up afterwards. */ function inScratch(body: (directory: string) => T): T { @@ -676,6 +677,11 @@ See [the code](mex://function:0123456789abcdef). return validateScaffold({ scaffoldRoot: directory }).diagnostics; }), + CODE_GRAPH_UNAVAILABLE: () => + // A graph that exists but could not be trusted for the pass (#232). A + // missing graph is ordinary and has no notice, so it is not the case here. + [codeGraphUnavailableDiagnostic("stale")!], + KNOWLEDGE_UNTYPED: () => // A domain context file population wrote without a `type`. inScratch((directory) => { diff --git a/src/wiki/cli/__tests__/commands.test.ts b/src/wiki/cli/__tests__/commands.test.ts index 14d4e689..f290d919 100644 --- a/src/wiki/cli/__tests__/commands.test.ts +++ b/src/wiki/cli/__tests__/commands.test.ts @@ -242,7 +242,7 @@ const CODE_GRAPH = { function allCommands( harnessed: Captured, flags: CommandFlags, -): Array<{ command: string; run: () => void }> { +): Array<{ command: string; run: () => void | Promise }> { const { io } = harnessed; return [ { command: "list", run: () => runList(io, flags) }, @@ -270,7 +270,7 @@ function allCommands( } describe("stable JSON envelopes", () => { - it("every implemented command returns the §15.2 shape", () => { + it("every implemented command returns the §15.2 shape", async () => { const cases = allCommands(harness(), { json: true }); // The count is asserted so a command dropped from the list fails here // rather than quietly reducing what this suite covers. @@ -280,13 +280,13 @@ describe("stable JSON envelopes", () => { ); }); - it("emits exactly one enveloped object per command under --json", () => { + it("emits exactly one enveloped object per command under --json", async () => { for (const { command } of allCommands(harness(), { json: true })) { const local = harness(); writeFileSync(join(local.root, "op.json"), "{}", "utf-8"); writeFileSync(join(local.root, "response.json"), "{}", "utf-8"); const entry = allCommands(local, { json: true }).find((candidate) => candidate.command === command)!; - entry.run(); + await entry.run(); expect(local.lines, command).toHaveLength(1); const parsed = JSON.parse(local.lines[0]!) as Record; expect(Object.keys(parsed).sort(), command).toEqual(["data", "diagnostics", "ok", "schemaVersion"]); @@ -297,7 +297,7 @@ describe("stable JSON envelopes", () => { }); describe("no ANSI in JSON", () => { - it("holds for all ten with colour forced on", () => { + it("holds for all ten with colour forced on", async () => { // Built, never written: a literal escape byte in a source file is banned. const escape = String.fromCharCode(0x1b); expect(chalk.level).toBeGreaterThan(0); @@ -308,12 +308,12 @@ describe("no ANSI in JSON", () => { const local = harness(); writeFileSync(join(local.root, "op.json"), "{}", "utf-8"); writeFileSync(join(local.root, "response.json"), "{}", "utf-8"); - allCommands(local, { json: true }).find((candidate) => candidate.command === command)!.run(); + await allCommands(local, { json: true }).find((candidate) => candidate.command === command)!.run(); for (const line of local.lines) expect(line.includes(escape), `${command}: ${line}`).toBe(false); } }); - it("does colour the human path, so the JSON assertion is not vacuous", () => { + it("does colour the human path, so the JSON assertion is not vacuous", async () => { const escape = String.fromCharCode(0x1b); const local = harness(false); runList(local.io, {}); @@ -323,12 +323,12 @@ describe("no ANSI in JSON", () => { }); describe("typed exit statuses", () => { - it("never pairs ok:false with exit 0, across every command", () => { + it("never pairs ok:false with exit 0, across every command", async () => { let checked = 0; for (const { command } of allCommands(harness(), { json: true })) { // No index and no operation file: several of the ten now fail. const local = harness(false); - allCommands(local, { json: true }).find((candidate) => candidate.command === command)!.run(); + await allCommands(local, { json: true }).find((candidate) => candidate.command === command)!.run(); const parsed = JSON.parse(local.lines[0]!) as { ok: boolean }; if (!parsed.ok) expect(local.exit(), command).not.toBe(WIKI_EXIT.ok); else expect(local.exit(), command).toBe(WIKI_EXIT.ok); @@ -337,7 +337,7 @@ describe("typed exit statuses", () => { expect(checked).toBe(14); }); - it("reports a missing index as its own status, distinct from a diagnostic failure", () => { + it("reports a missing index as its own status, distinct from a diagnostic failure", async () => { const local = harness(false); runList(local.io, { json: true }); expect(local.exit()).toBe(WIKI_EXIT.index); @@ -347,33 +347,33 @@ describe("typed exit statuses", () => { expect(parsed.diagnostics[0]?.remediation).toContain("rebuild-index"); }); - it("reports an unreadable operation file as a usage error, not an index one", () => { + it("reports an unreadable operation file as a usage error, not an index one", async () => { const local = harness(); writeFileSync(join(local.root, "op.json"), "{ not json", "utf-8"); - runApply(local.io, join(local.root, "op.json"), { json: true }); + await runApply(local.io, join(local.root, "op.json"), { json: true }); expect(local.exit()).toBe(WIKI_EXIT.usage); }); }); describe("missing-index behaviour", () => { - it("never rebuilds, for any read command", () => { + it("never rebuilds, for any read command", async () => { const reads = ["list", "show", "query", "related", "backlinks", "graph"]; for (const command of reads) { const local = harness(false); - allCommands(local, { json: true }).find((candidate) => candidate.command === command)!.run(); + await allCommands(local, { json: true }).find((candidate) => candidate.command === command)!.run(); expect(readdirSync(local.root), command).not.toContain("wiki.db"); } }); - it("lets validate answer anyway, and still creates nothing", () => { + it("lets validate answer anyway, and still creates nothing", async () => { const local = harness(false); - runValidate(local.io, { json: true }); + await runValidate(local.io, { json: true }); const parsed = JSON.parse(local.lines[0]!) as { ok: boolean; data: { entitiesChecked: number } }; expect(parsed.data.entitiesChecked).toBe(3); expect(readdirSync(local.root)).not.toContain("wiki.db"); }); - it("delete the index, rebuild, and every read command returns — at the command level", () => { + it("delete the index, rebuild, and every read command returns — at the command level", async () => { const local = harness(); runList(local.io, { json: true }); const before = JSON.parse(local.lines[0]!) as { data: { entities: unknown[] } }; @@ -385,13 +385,13 @@ describe("missing-index behaviour", () => { expect((JSON.parse(local.lines[0]!) as { ok: boolean }).ok).toBe(false); local.lines.length = 0; - runRebuildIndex(local.io, { json: true }); + await runRebuildIndex(local.io, { json: true }); expect(local.exit()).toBe(WIKI_EXIT.ok); for (const command of ["list", "show", "query", "related", "backlinks", "graph"]) { const fresh = { ...local, lines: [] as string[] }; const io: CommandIo = { ...local.io, write: (line) => fresh.lines.push(line) }; - allCommands({ ...local, io, lines: fresh.lines }, { json: true }) + await allCommands({ ...local, io, lines: fresh.lines }, { json: true }) .find((candidate) => candidate.command === command)! .run(); expect((JSON.parse(fresh.lines[0]!) as { ok: boolean }).ok, command).toBe(true); @@ -400,7 +400,7 @@ describe("missing-index behaviour", () => { }); describe("bounded output", () => { - it("clamps a limit nobody should be able to ask for, and says the list was cut", () => { + it("clamps a limit nobody should be able to ask for, and says the list was cut", async () => { const local = harness(); runList(local.io, { json: true, limit: 1 }); const parsed = JSON.parse(local.lines[0]!) as { data: { entities: unknown[]; truncated: boolean } }; @@ -410,7 +410,7 @@ describe("bounded output", () => { expect(JSON.stringify(parsed).includes("TRUNCATED")).toBe(false); }); - it("tells a user plainly that a bounded graph is a sample", () => { + it("tells a user plainly that a bounded graph is a sample", async () => { const local = harness(); runGraph(local.io, { limit: 1 }); expect(local.lines.join(" ")).toContain("bounded sample"); @@ -432,22 +432,22 @@ grounds_to: Body. `; - it("does not blame the checkout's code graph for a pass that was never given one", () => { + it("does not blame the checkout's code graph for a pass that was never given one", async () => { // `runValidate` builds its options from `serviceOptions`, which carries no // graph, so this is the branch every CLI run takes. The old wording // asserted there was no graph in the checkout — a fact this command does // not check, and one that was false on the scaffold that surfaced it. const local = harness(true, { "context/grounded.md": GROUNDED }); - runValidate(local.io, {}); + await runValidate(local.io, {}); const notice = local.lines.find((line) => line.includes("grounding checks did not run")); expect(notice).toBeDefined(); expect(notice).toContain("this pass was given no code graph"); expect(notice).not.toContain("no code graph in this checkout"); }); - it("carries the discriminator in JSON, so a CI caller can tell the two causes apart", () => { + it("carries the discriminator in JSON, so a CI caller can tell the two causes apart", async () => { const local = harness(true, { "context/grounded.md": GROUNDED }); - runValidate(local.io, { json: true }); + await runValidate(local.io, { json: true }); const parsed = JSON.parse(local.lines[0]!) as { data: { groundingsUnverified: boolean; codeGraphAvailable: boolean }; }; @@ -457,7 +457,7 @@ Body. }); describe("query filters", () => { - it("mean the same thing wherever they apply", () => { + it("mean the same thing wherever they apply", async () => { expect(filtersFrom({ limit: "25" }).limit).toBe(25); expect(filtersFrom({ limit: 25 }).limit).toBe(25); // A flag that is not a number is dropped rather than becoming NaN, which @@ -467,7 +467,7 @@ describe("query filters", () => { expect(filtersFrom({})).toEqual({}); }); - it("filter list, query and graph identically", () => { + it("filter list, query and graph identically", async () => { const local = harness(); for (const run of [ () => runList(local.io, { json: true, type: "component" }), @@ -590,12 +590,12 @@ describe("mutation cannot bypass review", () => { return prepared.data.groups; } - it("plans and writes nothing without --apply — asserted as a negative", () => { + it("plans and writes nothing without --apply — asserted as a negative", async () => { const local = harness(); const path = operationFile(local.root); const before = readFileSync(join(local.root, "context/architecture.md"), "utf-8"); - runApply(local.io, path, { json: true }); + await runApply(local.io, path, { json: true }); const parsed = JSON.parse(local.lines[0]!) as { ok: boolean; data: { planned: boolean; applied: boolean } }; expect(parsed.data.planned).toBe(true); expect(parsed.data.applied).toBe(false); @@ -603,21 +603,21 @@ describe("mutation cannot bypass review", () => { expect(readdirSync(local.root)).not.toContain("events"); }); - it("writes with --apply, and --dry-run overrides it", () => { + it("writes with --apply, and --dry-run overrides it", async () => { const local = harness(); const path = operationFile(local.root); const before = readFileSync(join(local.root, "context/architecture.md"), "utf-8"); const guardedIo = { ...local.io, enforceInboxSpecBoundary: true }; - runApply(guardedIo, path, { json: true, apply: true, dryRun: true }); + await runApply(guardedIo, path, { json: true, apply: true, dryRun: true }); expect(readFileSync(join(local.root, "context/architecture.md"), "utf-8")).toBe(before); local.lines.length = 0; - runApply(guardedIo, path, { json: true, apply: true }); + await runApply(guardedIo, path, { json: true, apply: true }); expect(readFileSync(join(local.root, "context/architecture.md"), "utf-8")).toContain("status: deprecated"); }); - it("installs the direct Spec guard on real wiki apply before preview or apply can write", () => { + it("installs the direct Spec guard on real wiki apply before preview or apply can write", async () => { const local = harness(); const path = createOperationFile(local.root, "context/direct-spec.md", "spec"); const before = readFileSync(join(local.root, "context/architecture.md"), "utf-8"); @@ -625,7 +625,7 @@ describe("mutation cannot bypass review", () => { for (const flags of [{ json: true }, { json: true, apply: true }]) { local.lines.length = 0; - runApply(guardedIo, path, flags); + await runApply(guardedIo, path, flags); const envelope = JSON.parse(local.lines[0]!) as { ok: boolean; data: { planned: boolean; applied: boolean; changedFiles: string[] }; @@ -641,7 +641,7 @@ describe("mutation cannot bypass review", () => { } }); - it("classifies physical Spec destinations through symlink and case aliases before writing", () => { + it("classifies physical Spec destinations through symlink and case aliases before writing", async () => { const local = harness(); mkdirSync(join(local.root, "specs"), { recursive: true }); symlinkSync("../specs", join(local.root, "context", "spec-alias"), "dir"); @@ -655,7 +655,7 @@ describe("mutation cannot bypass review", () => { for (const destination of cases) { local.lines.length = 0; const path = createOperationFile(local.root, destination); - runApply(guardedIo, path, { json: true, apply: true }); + await runApply(guardedIo, path, { json: true, apply: true }); const envelope = JSON.parse(local.lines[0]!) as { ok: boolean; data: { planned: boolean; applied: boolean; changedFiles: string[] }; @@ -670,12 +670,12 @@ describe("mutation cannot bypass review", () => { } }); - it("refuses a non-Spec inline replacement whose nested relation targets a Spec", () => { + it("refuses a non-Spec inline replacement whose nested relation targets a Spec", async () => { const local = harness(true, { "specs/direct.md": SPEC_DUPLICATES_MD }); const path = inlineReplacementRelationOperationFile(local.root); const before = readFileSync(join(local.root, "context", "architecture.md"), "utf-8"); - runApply( + await runApply( { ...local.io, enforceInboxSpecBoundary: true }, path, { json: true, apply: true }, @@ -695,13 +695,13 @@ describe("mutation cannot bypass review", () => { expect(existsSync(join(local.root, "events"))).toBe(false); }); - it("refuses update and move-out of non-Spec entities physically stored under specs", () => { + it("refuses update and move-out of non-Spec entities physically stored under specs", async () => { const local = harness(true, { "specs/misfiled.md": MISFILED_DECISION_DUPLICATES_MD }); const before = readFileSync(join(local.root, "specs", "misfiled.md"), "utf-8"); for (const type of ["update-entry", "move-entry"] as const) { local.lines.length = 0; - runApply( + await runApply( { ...local.io, enforceInboxSpecBoundary: true }, misfiledExistingOperationFile(local.root, type), { json: true, apply: true }, @@ -721,7 +721,7 @@ describe("mutation cannot bypass review", () => { } }); - it("installs the direct Spec guard on real wiki propose and refuses a mixed batch atomically", () => { + it("installs the direct Spec guard on real wiki propose and refuses a mixed batch atomically", async () => { const local = harness(true, { "specs/direct.md": SPEC_DUPLICATES_MD, "context/conventions.md": CONVENTION_DUPLICATES_MD, @@ -765,7 +765,7 @@ describe("mutation cannot bypass review", () => { expect(existsSync(join(local.root, "events"))).toBe(false); }); - it("keeps non-Spec wiki propose available when the product guard is installed", () => { + it("keeps non-Spec wiki propose available when the product guard is installed", async () => { const local = harness(true, { "context/conventions.md": CONVENTION_DUPLICATES_MD }); const convention = globalGroups(local).find((group) => group.type === "convention")!; const response = join(local.root, "safe-global-response.json"); @@ -800,7 +800,7 @@ describe("mutation cannot bypass review", () => { expect(existsSync(join(local.root, "events"))).toBe(false); }); - it("refuses a mixed propose batch when a non-Spec mutation originates under specs", () => { + it("refuses a mixed propose batch when a non-Spec mutation originates under specs", async () => { const local = harness(true, { "specs/misfiled.md": MISFILED_DECISION_DUPLICATES_MD, "context/conventions.md": CONVENTION_DUPLICATES_MD, @@ -844,7 +844,7 @@ describe("mutation cannot bypass review", () => { }); describe("no command builds its own envelope", () => { - it("holds across the command module", () => { + it("holds across the command module", async () => { // §20.7's parity, structurally: there is one definition, so a command that // assembled the shape by hand would be the second one. Comments stripped, // so prose about the rule is not read as a breach of it. diff --git a/src/wiki/cli/__tests__/grounding.test.ts b/src/wiki/cli/__tests__/grounding.test.ts new file mode 100644 index 00000000..83588075 --- /dev/null +++ b/src/wiki/cli/__tests__/grounding.test.ts @@ -0,0 +1,253 @@ +/** + * The Wiki CLI and the code graph (#232). + * + * The CLI used to hand its services no graph, so `wiki validate` never + * resolved a grounding and `wiki rebuild-index` stored no health in a checkout + * with a fresh `graph.db`. These drive the commands through a bridge with the + * repository graph port's contract: one snapshot per command, a refusal when + * the graph cannot be trusted, and a final freshness proof that can fail after + * the work has run. + */ + +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import type { GroundedNode, GroundingGraph } from "../../grounding/adapter.js"; +import { MexPortError, type MexErrorCode } from "../../../team/contracts/shared.js"; +import { wikiGroundingStatus } from "../../service/read.js"; +import { runRebuildIndex, runValidate, type CommandIo } from "../commands.js"; +import { + publishWithGraph, + readWithGraph, + writeWithGraph, + type WikiCliGroundingBridge, + type WikiCliPreparedPublication, +} from "../grounding.js"; + +const ENTITY = "mx_01KR2E4K002H3ZYA9G0C4XV531"; +const NODE = "function:1a2b3c4d5e6f7a8b"; +const FINGERPRINT = "mh:64:0a0b0c0d"; + +const ENTITY_MD = `--- +mex: + id: ${ENTITY} + type: component + status: promoted + revision: 1 + title: Auth + grounds_to: + - node: ${NODE} + fingerprint: ${FINGERPRINT} + bodyHash: body-1 +--- + +# Auth + +Rotates refresh tokens. +`; + +const roots: string[] = []; + +afterEach(() => { + for (const root of roots.splice(0)) { + try { + rmSync(root, { recursive: true, force: true }); + } catch { + // Windows keeps handles on just-closed SQLite files. + } + } +}); + +function graphWithBody(bodyHash: string): GroundingGraph { + const node: GroundedNode = { id: NODE, bodyHash, filePath: "src/auth.ts", startLine: 1, endLine: 5 }; + return { + getNode: (id) => (id === NODE ? node : null), + getFingerprint: (id) => (id === NODE ? FINGERPRINT : null), + reconcile: () => ({ kind: "GONE" }), + getBaselineSource: () => null, + }; +} + +function refusal(code: MexErrorCode): MexPortError { + return new MexPortError({ title: "Graph unavailable", status: 409, code, detail: "refused" }); +} + +interface FakeBridge extends WikiCliGroundingBridge { + snapshots: number; +} + +/** The repository graph port's contract, with the refusal and final proof under test control. */ +function bridge( + graph: GroundingGraph, + options: { refuse?: MexErrorCode; failFinalProof?: boolean } = {}, +): FakeBridge { + const fake: FakeBridge = { + snapshots: 0, + async withFreshGroundingSnapshot(callback) { + if (options.refuse !== undefined) throw refusal(options.refuse); + fake.snapshots += 1; + const value = await callback(graph); + if (options.failFinalProof === true) throw refusal("OPERATION_INTERRUPTED"); + return value; + }, + async withFreshGroundingPublication(prepare) { + if (options.refuse !== undefined) throw refusal(options.refuse); + fake.snapshots += 1; + const prepared = await prepare(graph); + await prepared.preflight(); + if (options.failFinalProof === true) { + await prepared.discard(); + throw refusal("OPERATION_INTERRUPTED"); + } + return prepared.commit(); + }, + }; + return fake; +} + +function cli(groundingBridge?: WikiCliGroundingBridge): { io: CommandIo; root: string; lines: string[] } { + const root = mkdtempSync(join(tmpdir(), "mex-wiki-cli-graph-")); + roots.push(root); + mkdirSync(join(root, "context"), { recursive: true }); + writeFileSync(join(root, "context", "auth.md"), ENTITY_MD, "utf-8"); + const lines: string[] = []; + return { + root, + lines, + io: { + write: (line) => lines.push(line), + setExitCode: () => undefined, + scaffoldRoot: root, + projectRoot: root, + ...(groundingBridge === undefined ? {} : { groundingBridge }), + }, + }; +} + +interface Envelope { + data: { codeGraphAvailable?: boolean; groundingsUnverified?: boolean }; + diagnostics: Array<{ code: string; severity: string }>; +} + +function envelope(lines: readonly string[]): Envelope { + return JSON.parse(lines[0]!) as Envelope; +} + +function codes(lines: readonly string[]): string[] { + return envelope(lines).diagnostics.map((entry) => entry.code); +} + +describe("wiki validate", () => { + it("resolves groundings against a fresh graph", async () => { + const changed = cli(bridge(graphWithBody("body-2"))); + await runValidate(changed.io, { json: true }); + expect(envelope(changed.lines).data).toMatchObject({ codeGraphAvailable: true, groundingsUnverified: false }); + expect(codes(changed.lines)).toContain("GROUNDING_STALE"); + + const fresh = cli(bridge(graphWithBody("body-1"))); + await runValidate(fresh.io, { json: true }); + expect(codes(fresh.lines)).not.toContain("GROUNDING_STALE"); + }); + + it("says why when the graph refuses a snapshot, and resolves nothing", async () => { + const stale = cli(bridge(graphWithBody("body-2"), { refuse: "INDEX_STALE" })); + await runValidate(stale.io, { json: true }); + expect(envelope(stale.lines).data).toMatchObject({ codeGraphAvailable: false, groundingsUnverified: true }); + expect(codes(stale.lines)).toContain("CODE_GRAPH_UNAVAILABLE"); + expect(codes(stale.lines)).not.toContain("GROUNDING_STALE"); + + const human = cli(bridge(graphWithBody("body-2"), { refuse: "INDEX_STALE" })); + await runValidate(human.io, {}); + expect(human.lines.join("\n")).toContain("the code graph could not be used for this pass"); + }); + + it("treats a checkout with no graph as ordinary, not as a problem", async () => { + const missing = cli(bridge(graphWithBody("body-1"), { refuse: "INDEX_MISSING" })); + await runValidate(missing.io, {}); + const output = missing.lines.join("\n"); + expect(output).toContain("this checkout has no code graph"); + expect(output).not.toContain("CODE_GRAPH_UNAVAILABLE"); + }); + + it("drops verdicts from a snapshot that failed its final freshness proof", async () => { + const moved = cli(bridge(graphWithBody("body-2"), { failFinalProof: true })); + await runValidate(moved.io, { json: true }); + expect(envelope(moved.lines).data.codeGraphAvailable).toBe(false); + expect(codes(moved.lines)).toContain("CODE_GRAPH_UNAVAILABLE"); + expect(codes(moved.lines)).not.toContain("GROUNDING_STALE"); + }); +}); + +describe("wiki rebuild-index", () => { + it("stores grounding health from a fresh graph", async () => { + const local = cli(bridge(graphWithBody("body-2"))); + await runRebuildIndex(local.io, { json: true }); + expect(wikiGroundingStatus({ scaffoldRoot: local.root, id: ENTITY }).data.entities[0]?.health).toBe("changed"); + }); + + it("publishes no health from a graph that changed while the index was built", async () => { + const local = cli(bridge(graphWithBody("body-2"), { failFinalProof: true })); + await runRebuildIndex(local.io, { json: true }); + expect(codes(local.lines)).toContain("CODE_GRAPH_UNAVAILABLE"); + // Still rebuilt — just without a verdict nobody can stand behind. + expect(wikiGroundingStatus({ scaffoldRoot: local.root, id: ENTITY }).data.entities[0]?.health).toBeNull(); + }); + + it("still rebuilds with no graph at all", async () => { + const local = cli(); + await runRebuildIndex(local.io, { json: true }); + expect(codes(local.lines)).not.toContain("CODE_GRAPH_UNAVAILABLE"); + expect(wikiGroundingStatus({ scaffoldRoot: local.root, id: ENTITY }).data.entities[0]?.health).toBeNull(); + }); +}); + +describe("the graph helpers", () => { + it("never runs a completed write twice when the final proof fails after it", async () => { + const graph = graphWithBody("body-1"); + let runs = 0; + const result = await writeWithGraph(bridge(graph, { failFinalProof: true }), (received) => { + runs += 1; + return received; + }); + expect(runs).toBe(1); + expect(result).toEqual({ value: graph, unavailable: "changed" }); + }); + + it("reruns a read without a graph when the final proof fails", async () => { + const seen: Array = []; + const result = await readWithGraph(bridge(graphWithBody("body-1"), { failFinalProof: true }), (graph) => { + seen.push(graph); + return seen.length; + }); + expect(seen.map((graph) => graph === null)).toEqual([false, true]); + expect(result).toEqual({ value: 2, unavailable: "changed" }); + }); + + it("rethrows a failure of the work itself rather than retrying it", async () => { + let runs = 0; + await expect(readWithGraph(bridge(graphWithBody("body-1")), () => { + runs += 1; + throw new Error("boom"); + })).rejects.toThrow("boom"); + expect(runs).toBe(1); + }); + + it("rethrows an unexpected bridge error instead of hiding it as a missing graph", async () => { + const broken: WikiCliGroundingBridge = { + withFreshGroundingSnapshot: async () => { + throw new Error("disk on fire"); + }, + withFreshGroundingPublication: async () => { + throw new Error("disk on fire"); + }, + }; + await expect(readWithGraph(broken, () => 1)).rejects.toThrow("disk on fire"); + const prepared: WikiCliPreparedPublication = { preflight: () => undefined, commit: () => 1, discard: () => undefined }; + await expect(publishWithGraph(broken, () => prepared)).rejects.toThrow("disk on fire"); + }); + + it("reports a caller that supplied no bridge as such, never as a checkout without a graph", async () => { + expect(await readWithGraph(undefined, () => 1)).toEqual({ value: 1, unavailable: "not_supplied" }); + }); +}); diff --git a/src/wiki/cli/commands.ts b/src/wiki/cli/commands.ts index 2bd83e4d..41566fe9 100644 --- a/src/wiki/cli/commands.ts +++ b/src/wiki/cli/commands.ts @@ -41,6 +41,13 @@ import { resolve } from "node:path"; import { tryEnsureSetupIgnoreProtection } from "../../setup/ignore.js"; import type { WikiDiagnostic } from "../model/diagnostic.js"; import { inspectDirectWikiSpecMutation } from "./spec-authoring-boundary.js"; +import { + graphDiagnostics, + publishWithGraph, + readWithGraph, + writeWithGraph, + type WikiCliGroundingBridge, +} from "./grounding.js"; import { envelopeFor, exitCodeFor, @@ -58,7 +65,7 @@ import { wikiList, wikiMigrate, wikiNeighborhood, - wikiRebuildIndex, + wikiPrepareRebuildIndex, wikiRegenerateViews, wikiSearch, wikiSynthesisBuild, @@ -85,6 +92,13 @@ export interface CommandIo { codeGraph?: import("../grounding/adapter.js").SynthesisGraph | null; /** Grounding resolution, for the §12.4 gate. */ graph?: import("../grounding/adapter.js").GroundingGraph | null; + /** + * The code graph for the commands that resolve groundings — validate, + * rebuild-index, migrate, apply and regenerate-views (#232). Reached only + * through one fresh, revocable snapshot per command; absent means this + * invocation has no graph, which every one of them tolerates. + */ + groundingBridge?: WikiCliGroundingBridge | null; /** §12 scope knobs from `wiki.synthesis`. */ synthesisScope?: SynthesisScope; /** Launch an agent with the playbook. Returns false when none is available. */ @@ -256,12 +270,14 @@ export function runBacklinks(io: CommandIo, id: string, flags: CommandFlags): vo }); } -export function runValidate(io: CommandIo, flags: CommandFlags): void { - const result = wikiValidate({ +export async function runValidate(io: CommandIo, flags: CommandFlags): Promise { + const { value: validated, unavailable } = await readWithGraph(io.groundingBridge, (graph) => wikiValidate({ ...serviceOptions(io), ...(io.projectRoot === undefined ? {} : { projectRoot: io.projectRoot }), ...filtersFrom(flags), - }); + graph, + })); + const result = { ...validated, diagnostics: [...validated.diagnostics, ...graphDiagnostics(unavailable)] }; emit(io, result, flags, (data) => { io.write( `${data.filesScanned} file(s), ${data.entitiesChecked} entities — ` + @@ -272,24 +288,20 @@ export function runValidate(io: CommandIo, flags: CommandFlags): void { // Not a diagnostic: it says how much was checked, not that anything is // wrong, and a CI run needs to tell a clean report from an unread one. // - // The flag has two causes and this line used to assert the first of them - // unconditionally, so a reader was sent to inspect a code graph nobody - // had looked at. Two corrections, and the second is the one that was - // costing something: - // - // - When a graph *was* supplied and still produced no verdict, the fault - // is in what the groundings committed, and the reader belongs in the - // scaffold rather than in `mex graph`. - // - When none was supplied, "no code graph in this checkout" is a claim - // about the repository that this command never checked. `wiki - // validate` does not load one — `serviceOptions` does not carry it — - // so on the CLI this is always the branch taken, and it read as a - // diagnosis of a fresh 7 MB `graph.db` sitting right there. Say what - // was actually true of the pass, and name the command that does check. + // Each case worded for what was actually true of the pass: the graph was + // used and still decided nothing, which puts the fault in what the + // groundings committed; the checkout has no graph; the caller gave this + // pass none, which says nothing about the checkout; or the checkout has + // one this pass could not trust, which the CODE_GRAPH_UNAVAILABLE notice + // below explains. io.write(chalk.dim( data.codeGraphAvailable ? "grounding checks did not run — the code graph is present, but no grounding could be compared against it" - : "grounding checks did not run — this pass was given no code graph; `mex check` is the grounding check", + : unavailable === "missing" + ? "grounding checks did not run — this checkout has no code graph; `mex graph` builds one" + : unavailable === "not_supplied" + ? "grounding checks did not run — this pass was given no code graph; `mex check` is the grounding check" + : "grounding checks did not run — the code graph could not be used for this pass", )); } }); @@ -312,7 +324,7 @@ export function runGraph(io: CommandIo, flags: CommandFlags): void { }); } -export function runRebuildIndex(io: CommandIo, flags: CommandFlags): void { +export async function runRebuildIndex(io: CommandIo, flags: CommandFlags): Promise { // The only command that creates `wiki.db`, and it can run in a checkout that // never went through `mex setup` — which is how issue #110's reporter ended // up with an untracked database. Best effort: an unwritable `.gitignore` is @@ -321,7 +333,12 @@ export function runRebuildIndex(io: CommandIo, flags: CommandFlags): void { const protection = tryEnsureSetupIgnoreProtection(projectRoot); if (!protection.ok) io.write(chalk.dim(`could not ignore local MEX data: ${protection.reason}`)); - emit(io, wikiRebuildIndex(serviceOptions(io)), flags, (data) => { + const { value: rebuilt, unavailable } = await publishWithGraph( + io.groundingBridge, + (graph) => wikiPrepareRebuildIndex({ ...serviceOptions(io), graph }), + ); + const result = { ...rebuilt, diagnostics: [...rebuilt.diagnostics, ...graphDiagnostics(unavailable)] }; + emit(io, result, flags, (data) => { io.write(`Indexed ${data.entityCount} entities from ${data.fileCount} file(s) into ${data.indexPath}`); for (const swept of data.sweptTempFiles) io.write(chalk.dim(`removed a crashed build's temp index: ${swept}`)); }); @@ -341,8 +358,12 @@ export function runRebuildIndex(io: CommandIo, flags: CommandFlags): void { * `--dry-run` is the default posture for anything an agent can call, so this * reports what has drifted unless it is told to write. */ -export function runRegenerateViews(io: CommandIo, flags: CommandFlags): void { - const result = wikiRegenerateViews({ ...serviceOptions(io), dryRun: flags.dryRun === true }); +export async function runRegenerateViews(io: CommandIo, flags: CommandFlags): Promise { + const { value: regenerated, unavailable } = await writeWithGraph( + io.groundingBridge, + (graph) => wikiRegenerateViews({ ...serviceOptions(io), dryRun: flags.dryRun === true, graph }), + ); + const result = { ...regenerated, diagnostics: [...regenerated.diagnostics, ...graphDiagnostics(unavailable)] }; emit(io, result, flags, (data) => { if (data.examined.length === 0) { io.write("No generated sections in this scaffold."); @@ -359,8 +380,19 @@ export function runRegenerateViews(io: CommandIo, flags: CommandFlags): void { }); } -export function runMigrate(io: CommandIo, flags: CommandFlags): void { - const result = wikiMigrate({ ...serviceOptions(io), dryRun: flags.dryRun === true }); +export async function runMigrate(io: CommandIo, flags: CommandFlags): Promise { + const dryRun = flags.dryRun === true; + const migrate = (graph: import("../grounding/adapter.js").GroundingGraph | null) => + wikiMigrate({ + ...serviceOptions(io), + dryRun, + graph, + ...(io.projectRoot === undefined ? {} : { projectRoot: io.projectRoot }), + }); + const { value: migrated, unavailable } = dryRun + ? await readWithGraph(io.groundingBridge, migrate) + : await writeWithGraph(io.groundingBridge, migrate); + const result = { ...migrated, diagnostics: [...migrated.diagnostics, ...graphDiagnostics(unavailable)] }; emit(io, result, flags, (data) => { io.write(data.rendered); if (data.report.abstentions.length > 0) { @@ -384,7 +416,7 @@ export function runMigrate(io: CommandIo, flags: CommandFlags): void { * prints the diff and writes nothing, which is the safe outcome when a caller * says nothing at all rather than something a caller has to opt into. */ -export function runApply(io: CommandIo, file: string, flags: CommandFlags): void { +export async function runApply(io: CommandIo, file: string, flags: CommandFlags): Promise { let envelope: unknown; try { envelope = JSON.parse(readFileSync(resolve(file), "utf-8")); @@ -418,10 +450,16 @@ export function runApply(io: CommandIo, file: string, flags: CommandFlags): void return; } - const result = wikiApplyOperation(envelope, { + const write = flags.apply === true && flags.dryRun !== true; + const applyWith = (graph: import("../grounding/adapter.js").GroundingGraph | null) => wikiApplyOperation(envelope, { ...serviceOptions(io), - ...(flags.apply === true && flags.dryRun !== true ? { apply: true } : {}), + ...(write ? { apply: true } : {}), + graph, }); + const { value: applied, unavailable } = write + ? await writeWithGraph(io.groundingBridge, applyWith) + : await readWithGraph(io.groundingBridge, applyWith); + const result = { ...applied, diagnostics: [...applied.diagnostics, ...graphDiagnostics(unavailable)] }; emit(io, result, flags, (data) => { if (data.diff !== null) io.write(data.diff); if (data.applied) io.write(chalk.green(`applied ${data.opId} — ${data.changedFiles.join(", ")}`)); diff --git a/src/wiki/cli/grounding.ts b/src/wiki/cli/grounding.ts new file mode 100644 index 00000000..8bd0a7dc --- /dev/null +++ b/src/wiki/cli/grounding.ts @@ -0,0 +1,186 @@ +/** + * The code graph, as the Wiki CLI may consult it (#232). + * + * The Wiki CLI used to build its service options with no graph at all, so + * `wiki validate` never resolved a grounding, `wiki rebuild-index` stored no + * health, and `wiki migrate` never backfilled a `bodyHash` — each degrading + * silently in a checkout with a fresh `graph.db`. + * + * The graph is reached here exactly as the Project Hub reaches it: through the + * repository graph port's fresh, revocable snapshot, which re-proves database + * and source freshness before anything derived from it escapes. A Wiki command + * never opens `graph.db` itself, never sees a stale snapshot, and never + * triggers graph maintenance. + */ + +import type { GroundingGraph } from "../grounding/adapter.js"; +import { + codeGraphUnavailableDiagnostic, + type CodeGraphUnavailableReason, +} from "../grounding/availability.js"; +import type { WikiDiagnostic } from "../model/diagnostic.js"; +import { MexPortError } from "../../team/contracts/shared.js"; + +/** A two-phase publication prepared while the snapshot is open. */ +export interface WikiCliPreparedPublication { + preflight(): void | Promise; + commit(): T | Promise; + discard(): void | Promise; +} + +/** + * The structural slice of `RepositoryGraphPort` the Wiki CLI uses. + * + * Structural, so `src/wiki/` does not import `src/graph/`; the CLI entry point + * composes the real port in. + */ +export interface WikiCliGroundingBridge { + withFreshGroundingSnapshot(callback: (snapshot: GroundingGraph) => T | Promise): Promise; + withFreshGroundingPublication( + prepare: (snapshot: GroundingGraph) => WikiCliPreparedPublication | Promise>, + ): Promise; +} + +/** + * Why a pass had no graph: the graph's own refusal, or `not_supplied` when + * the caller gave this command no bridge — which says nothing about whether + * the checkout has a graph, and must not be reported as if it did. + */ +export type WikiCliGraphAbsence = CodeGraphUnavailableReason | "not_supplied"; + +/** What a command learned about the graph, beside its own result. */ +export interface WithGraphResult { + value: T; + /** Null when a fresh graph was used; otherwise why the pass had none. */ + unavailable: WikiCliGraphAbsence | null; +} + +const REASONS: Partial> = { + INDEX_MISSING: "missing", + INDEX_STALE: "stale", + MIGRATION_REQUIRED: "rebuild_required", + INDEX_CORRUPT: "corrupt", + OPERATION_INTERRUPTED: "changed", + REVISION_CONFLICT: "changed", +}; + +/** The reason a graph refusal names, or null for an error that is not a refusal. */ +function unavailableReason(error: unknown): CodeGraphUnavailableReason | null { + if (!(error instanceof MexPortError)) return null; + return REASONS[error.problem.code] ?? null; +} + +class WorkFailure { + constructor(readonly cause: unknown) {} +} + +/** + * Run a read against one fresh graph snapshot, or without a graph when the + * graph refuses one. + * + * A read whose snapshot fails its final freshness proof is run again without a + * graph, so no verdict from a graph that moved under it is ever reported. A + * failure of the read itself is rethrown, never retried. + */ +export async function readWithGraph( + bridge: WikiCliGroundingBridge | null | undefined, + work: (graph: GroundingGraph | null) => T, +): Promise> { + if (bridge === null || bridge === undefined) return { value: work(null), unavailable: "not_supplied" }; + try { + const value = await bridge.withFreshGroundingSnapshot((graph) => { + try { + return work(graph); + } catch (error) { + throw new WorkFailure(error); + } + }); + return { value, unavailable: null }; + } catch (error) { + if (error instanceof WorkFailure) throw error.cause; + const reason = unavailableReason(error); + if (reason === null) throw error; + return { value: work(null), unavailable: reason }; + } +} + +/** + * Run a write against one fresh graph snapshot, or without a graph when the + * graph refuses one. + * + * Unlike a read, a write that has already run is never run again: its files + * are on disk. If the snapshot fails its final proof only after the write + * completed, the write's result stands and the pass is reported as `changed`, + * because the graph it consulted no longer describes the source. + */ +export async function writeWithGraph( + bridge: WikiCliGroundingBridge | null | undefined, + work: (graph: GroundingGraph | null) => T, +): Promise> { + if (bridge === null || bridge === undefined) return { value: work(null), unavailable: "not_supplied" }; + let completed: { value: T } | null = null; + try { + const value = await bridge.withFreshGroundingSnapshot((graph) => { + try { + completed = { value: work(graph) }; + return completed.value; + } catch (error) { + throw new WorkFailure(error); + } + }); + return { value, unavailable: null }; + } catch (error) { + if (error instanceof WorkFailure) throw error.cause; + const reason = unavailableReason(error); + if (reason === null) throw error; + const done = completed as { value: T } | null; + if (done !== null) return { value: done.value, unavailable: "changed" }; + return { value: work(null), unavailable: reason }; + } +} + +/** + * Prepare a publication against a fresh snapshot and publish it only after the + * graph's final freshness proof; otherwise publish the no-graph candidate. + * + * The same contract the repository Wiki adapter gives a Hub rebuild, so a CLI + * rebuild cannot publish health from a graph that changed while it was built. + */ +export async function publishWithGraph( + bridge: WikiCliGroundingBridge | null | undefined, + prepare: (graph: GroundingGraph | null) => WikiCliPreparedPublication, +): Promise> { + const withoutGraph = async (reason: WikiCliGraphAbsence): Promise> => { + const prepared = prepare(null); + try { + await prepared.preflight(); + } catch (error) { + await prepared.discard(); + throw error; + } + return { value: await prepared.commit(), unavailable: reason }; + }; + if (bridge === null || bridge === undefined) return withoutGraph("not_supplied"); + try { + const value = await bridge.withFreshGroundingPublication((graph) => { + try { + return prepare(graph); + } catch (error) { + throw new WorkFailure(error); + } + }); + return { value, unavailable: null }; + } catch (error) { + if (error instanceof WorkFailure) throw error.cause; + const reason = unavailableReason(error); + if (reason === null) throw error; + return withoutGraph(reason); + } +} + +/** The notice a fallback adds to a command's diagnostics, if any. */ +export function graphDiagnostics(unavailable: WikiCliGraphAbsence | null): WikiDiagnostic[] { + if (unavailable === null || unavailable === "not_supplied") return []; + const notice = codeGraphUnavailableDiagnostic(unavailable); + return notice === null ? [] : [notice]; +} diff --git a/src/wiki/grounding/__tests__/resolve.test.ts b/src/wiki/grounding/__tests__/resolve.test.ts index dcb3023a..e6b3f434 100644 --- a/src/wiki/grounding/__tests__/resolve.test.ts +++ b/src/wiki/grounding/__tests__/resolve.test.ts @@ -17,6 +17,7 @@ import { resolveGrounding, } from "../resolve.js"; import type { GroundedNode, GroundingGraph } from "../adapter.js"; +import { serializeFingerprint } from "../../../graph/fingerprint.js"; import type { WikiGrounding } from "../../model/grounding.js"; const NODE = "function:1a2b3c4d5e6f7a8b"; @@ -99,6 +100,49 @@ describe("the resolution table", () => { }); }); + it("calls a rebound node with a byte-identical body fresh, whatever its fingerprint", () => { + // The graph answers the declared id with the node a compatibility alias + // recorded for it after ids were re-minted (#240). Its neighbours were + // renumbered, so its fingerprint differs; its body did not. + const aliased = stubGraph({ + nodes: { [NODE]: node(MOVED_NODE, "body-1") }, + fingerprints: { [MOVED_NODE]: "mh:64:ffffffff" }, + }); + expect(resolveGrounding(GROUNDED, aliased)).toEqual({ + state: "fresh", + health: "fresh", + node: NODE, + resolvedNode: MOVED_NODE, + rebound: true, + bodyHash: "body-1", + }); + + // A body that differs is still judged by structure. + const edited = stubGraph({ + nodes: { [NODE]: node(MOVED_NODE, "body-2") }, + fingerprints: { [MOVED_NODE]: "mh:64:ffffffff" }, + }); + expect(resolveGrounding(GROUNDED, edited)).toMatchObject({ state: "stale", health: "changed", resolvedNode: MOVED_NODE }); + }); + + it("judges a grounding with no body hash by its code, not by who calls it", () => { + const code = { minhash: Array.from({ length: 64 }, (_, index) => index * 7919), tokenCount: 40 }; + const committed = serializeFingerprint({ ...code, neighbors: [`function:${"a".repeat(32)}`] }); + const legacyGrounding: WikiGrounding = { node: NODE, fingerprint: committed }; + // Neighbours renumbered (#240) and a new caller: the same code. + const renumbered = serializeFingerprint({ ...code, neighbors: [`function:${"b".repeat(32)}`, `method:${"c".repeat(32)}`] }); + expect(resolveGrounding(legacyGrounding, stubGraph({ + nodes: { [NODE]: node(NODE, "body-9") }, + fingerprints: { [NODE]: renumbered }, + }))).toMatchObject({ state: "fresh", health: "fresh" }); + // A rewritten body changes the sketch, and that is still drift. + const rewritten = serializeFingerprint({ ...code, minhash: code.minhash.map((value) => value + 1), neighbors: [] }); + expect(resolveGrounding(legacyGrounding, stubGraph({ + nodes: { [NODE]: node(NODE, "body-9") }, + fingerprints: { [NODE]: rewritten }, + }))).toMatchObject({ state: "stale", health: "changed" }); + }); + it("compares a rebind by fingerprint, because a rename always moves the body hash", () => { // Measured against a real graph: renaming a symbol changes its body hash, // since the name is part of the body. If a rebind were compared by body diff --git a/src/wiki/grounding/adapter.ts b/src/wiki/grounding/adapter.ts index 03d345a0..fe6653c2 100644 --- a/src/wiki/grounding/adapter.ts +++ b/src/wiki/grounding/adapter.ts @@ -30,7 +30,7 @@ * provoke in a real repository. */ -import { serializeFingerprint, deserializeFingerprint, sameFingerprint } from "../../graph/fingerprint.js"; +import { serializeFingerprint, deserializeFingerprint, sameFingerprint, sameFingerprintCode } from "../../graph/fingerprint.js"; import { FingerprintStore } from "../../graph/fingerprint-store.js"; import { GraphStore } from "../../graph/db/store.js"; import type { GraphEngine } from "../../graph/engine.js"; @@ -46,6 +46,12 @@ import { asGraphDerived, type GraphDerivedGrounding, type WikiGrounding } from " */ export { sameFingerprint }; +/** + * Whether two serialized fingerprints describe the same code, neighbours set + * aside — the structural change signal for a grounding with no body hash. + */ +export { sameFingerprintCode }; + /** What resolution knows about one code node, as the wiki sees it. */ export interface GroundedNode { id: string; diff --git a/src/wiki/grounding/availability.ts b/src/wiki/grounding/availability.ts new file mode 100644 index 00000000..0322a1df --- /dev/null +++ b/src/wiki/grounding/availability.ts @@ -0,0 +1,42 @@ +/** + * Why a pass that resolves groundings ran without a code graph (#232). + * + * The Wiki reads the graph only through one fresh, revocable snapshot, and the + * graph refuses that snapshot in a handful of distinct states. A caller that + * falls back to "no graph" has to say which one, because the remedy differs: + * a checkout with no graph may be one that never has one, while a stale or + * corrupt graph is a problem `mex graph` fixes. + */ + +import { diagnostic, type WikiDiagnostic } from "../model/diagnostic.js"; + +export type CodeGraphUnavailableReason = + | "missing" + | "stale" + | "rebuild_required" + | "corrupt" + | "changed"; + +const MESSAGES: Record, string> = { + stale: "The code graph is out of date with the source, so groundings were not resolved against it.", + rebuild_required: "The code graph was built by an incompatible version of mex, so groundings were not resolved against it.", + corrupt: "The code graph could not be read safely, so groundings were not resolved against it.", + changed: "The code graph or the source changed during this pass, so groundings were not resolved against it.", +}; + +const REMEDIATIONS: Partial> = { + rebuild_required: "Run `mex graph rebuild` to rebuild the code graph, then run this command again.", + corrupt: "Run `mex graph rebuild` to rebuild the code graph, then run this command again.", +}; + +/** + * The notice for a graph that exists but could not be used, or null for one + * that does not exist. A checkout with no graph is an ordinary configuration — + * CI before `mex graph`, a memory-only project — and the commands that care + * already say so in their own output. + */ +export function codeGraphUnavailableDiagnostic(reason: CodeGraphUnavailableReason): WikiDiagnostic | null { + if (reason === "missing") return null; + const remediation = REMEDIATIONS[reason]; + return diagnostic("CODE_GRAPH_UNAVAILABLE", MESSAGES[reason], remediation === undefined ? {} : { remediation }); +} diff --git a/src/wiki/grounding/resolve.ts b/src/wiki/grounding/resolve.ts index 011e4253..a5834638 100644 --- a/src/wiki/grounding/resolve.ts +++ b/src/wiki/grounding/resolve.ts @@ -62,7 +62,7 @@ import { type GroundingResolution, type WikiGrounding, } from "../model/grounding.js"; -import { sameFingerprint, type GroundedNode, type GroundingGraph } from "./adapter.js"; +import { sameFingerprintCode, type GroundedNode, type GroundingGraph } from "./adapter.js"; /** * Resolve one grounding against the local checkout. @@ -183,6 +183,25 @@ function compare( }; } + // **A rebind whose body is byte-identical is fresh.** The graph answers a + // declared id through a compatibility alias when ids are re-minted (#240), + // and reconciliation rebinds a moved symbol; either way, a body equal to the + // committed hash is the code the entity described. Neighbour ids inside + // the fingerprint were renumbered by the re-mint, so comparing structure + // here reported `changed` for code that had not changed by one byte — + // where `mex check`, comparing bodies, rightly called it fresh. A body that + // differs still goes to the fingerprint below, so a rename stays fresh. + if (rebound && grounding.bodyHash !== undefined && grounding.bodyHash === currentBodyHash) { + return { + state: "fresh", + health: "fresh", + node: declaredNode, + resolvedNode: node.id, + rebound, + bodyHash: currentBodyHash, + }; + } + // Structure: either the grounding committed no body hash, or the symbol was // rebound. This sees a symbol being rewritten and misses a constant being // edited, which is why anything mex writes commits a body hash. @@ -196,8 +215,11 @@ function compare( }; } // By value: a grounding committed before #233 spells the same fingerprint in - // the older encoding, and that is not a change. - if (sameFingerprint(currentFingerprint, grounding.fingerprint)) { + // the older encoding, and that is not a change. And by code alone: the + // neighbour list changes when a caller elsewhere is added or node ids are + // re-minted (#240), and reading that as this node's code changing reported + // `changed` for groundings whose every token was the same. + if (sameFingerprintCode(currentFingerprint, grounding.fingerprint)) { return { state: "fresh", health: "fresh", diff --git a/src/wiki/migration/__tests__/grounding.test.ts b/src/wiki/migration/__tests__/grounding.test.ts index eb468a1f..18b8522a 100644 --- a/src/wiki/migration/__tests__/grounding.test.ts +++ b/src/wiki/migration/__tests__/grounding.test.ts @@ -236,6 +236,63 @@ describe("tier 2 — a grounded scaffold, migrated", () => { expect(groundingAt(project, 0)?.bodyHash).toBe(moved?.bodyHash); }, 180_000); + it("backfills the body the grounding was made against, not the code as it is now (#232)", async () => { + const project = createProject(); + await buildGraph(project); + const nodeId = nodeIdOf(project, "rotateRefreshToken"); + runGround(project, nodeId); + const committed = groundingAt(project, 0)!; + const original = withGraph(project, (graph) => graph.getNode(nodeId)!.bodyHash!); + + // A constant edit: the fingerprint cannot see it, the body hash can. + writeFileSync(join(project.root, "src", "auth.ts"), SOURCE.replace("3600", "7200"), "utf-8"); + await buildGraph(project); + const current = withGraph(project, (graph) => graph.getNode(nodeId)!.bodyHash!); + expect(current).not.toBe(original); + + // The baseline `mex ground` cached when the grounding was made. + const db = openGraphDatabase(join(project.root, ".mex", "graph.db")); + try { + new FingerprintStore(db).saveBaseline({ + subject: { kind: "scaffold", id: ".mex/patterns/rotate-tokens.md" }, + nodeId, + source: SOURCE, + bodyHash: original, + fingerprint: committed.fingerprint!, + }); + } finally { + db.close(); + } + + withGraph(project, (graph) => migrateScaffold({ scaffoldRoot: project.scaffoldRoot, graph })); + // The drift since grounding survives migration rather than being adopted. + expect(groundingAt(project, 0)?.bodyHash).toBe(original); + }, 180_000); + + it("writes no bodyHash when neither a cached baseline nor the structure vouches for one (#232)", async () => { + const project = createProject(); + await buildGraph(project); + const nodeId = nodeIdOf(project, "rotateRefreshToken"); + runGround(project, nodeId); + + // A structural rewrite, with no cached baseline to say what it was before. + writeFileSync( + join(project.root, "src", "auth.ts"), + SOURCE.replace( + "return attempts * windowSeconds;", + "if (attempts > 3) throw new Error(\"too many\");\n return attempts * windowSeconds * 2;", + ), + "utf-8", + ); + await buildGraph(project); + expect(nodeIdOf(project, "rotateRefreshToken")).toBe(nodeId); + + withGraph(project, (graph) => migrateScaffold({ scaffoldRoot: project.scaffoldRoot, graph })); + const grounding = groundingAt(project, 0); + expect(grounding?.node).toBe(nodeId); + expect(grounding?.bodyHash).toBeUndefined(); + }, 180_000); + it("moves nothing, and reports it, when no graph is available", async () => { const project = createProject(); await buildGraph(project); diff --git a/src/wiki/migration/legacy.ts b/src/wiki/migration/legacy.ts index dc7a410a..a51b7a8b 100644 --- a/src/wiki/migration/legacy.ts +++ b/src/wiki/migration/legacy.ts @@ -55,7 +55,8 @@ import type { EntityId } from "../model/ids.js"; import type { WikiGrounding } from "../model/grounding.js"; import type { LegacyEdge } from "../markdown/contract.js"; import { rootGroundingsNotInEffect } from "../markdown/grounding-stores.js"; -import type { GroundingGraph } from "../grounding/adapter.js"; +import { join, relative, resolve } from "node:path"; +import { sameFingerprint, type GroundingGraph } from "../grounding/adapter.js"; import type { InventoryFile, ScaffoldInventory } from "./inventory.js"; import { ALREADY_ADOPTED_REASON, type Candidate, type FileClassification } from "./classify.js"; @@ -209,7 +210,11 @@ export function planGroundingMoves( classifications: Map, candidatesFor: (path: string) => Candidate[], graph: GroundingGraph | null, + /** The checkout root `mex ground` keyed its cached baselines under; defaults to the scaffold's parent. */ + projectRoot?: string, ): GroundingPlan { + const project = projectRoot ?? resolve(inventory.root, ".."); + const subjectOf = (path: string): string => relative(project, join(inventory.root, path)).replaceAll("\\", "/"); const moved = new Map(); const absorbed: GroundingPlan["absorbed"] = new Map(); const diagnostics: WikiDiagnostic[] = []; @@ -259,17 +264,47 @@ export function planGroundingMoves( continue; } - moved.set(file.path, groundings.map((grounding) => backfill(grounding, graph))); + moved.set(file.path, groundings.map((grounding) => backfill(grounding, graph, subjectOf(file.path)))); } return { moved, absorbed, diagnostics }; } -/** Add a `bodyHash` the graph can re-derive; never invent one. */ -export function backfill(grounding: WikiGrounding, graph: GroundingGraph | null): WikiGrounding { +/** + * Add the `bodyHash` the grounding was made against, when there is evidence of + * it; never invent one (#232). + * + * The body hash is what every later drift verdict compares against, so the + * one written here has to be the body as it was when the grounding was made, + * not as it is now. Taking the current body unconditionally — the rule before + * the CLI handed migration a graph, when it never ran — would have erased any + * drift since grounding, permanently and silently. Two sources, most direct + * first, the same evidence `mex sync`'s own legacy backfill uses: + * + * 1. The baseline `mex ground` cached for this scaffold file and node, when its + * fingerprint is the one committed — the historical body itself. Read for + * its value, not to decide drift. + * 2. The current body, but only for the declared node itself (not an alias) + * whose fingerprint is still the committed one. A changed structure means + * the code moved on, and adopting it would hide that. + * + * Otherwise the entry stays without one and keeps its structural comparison. + */ +export function backfill( + grounding: WikiGrounding, + graph: GroundingGraph | null, + /** Project-relative path of the scaffold file, the key `mex ground` cached under. */ + subject?: string, +): WikiGrounding { if (grounding.bodyHash !== undefined || graph === null) return grounding; + const historical = subject === undefined ? null : graph.getBaselineSource({ kind: "scaffold", id: subject }, grounding.node); + if (historical !== null && sameFingerprint(historical.fingerprint, grounding.fingerprint)) { + return { ...grounding, bodyHash: historical.bodyHash }; + } const node = graph.getNode(grounding.node); - if (node === null || node.bodyHash === null) return grounding; + if (node === null || node.id !== grounding.node || node.bodyHash === null) return grounding; + const current = graph.getFingerprint(node.id); + if (current === null || !sameFingerprint(current, grounding.fingerprint)) return grounding; return { ...grounding, bodyHash: node.bodyHash }; } diff --git a/src/wiki/migration/migrate.ts b/src/wiki/migration/migrate.ts index af2063ab..9d3f7b98 100644 --- a/src/wiki/migration/migrate.ts +++ b/src/wiki/migration/migrate.ts @@ -33,6 +33,8 @@ import type { AbsorbableRootKey, WikiActor, WikiOperation } from "../model/opera import type { WikiEntityType } from "../model/entity.js"; import type { EntityTypeRegistry } from "../model/entity.js"; import type { GroundingGraph } from "../grounding/adapter.js"; +import { resolveGrounding } from "../grounding/resolve.js"; +import type { GroundingResolver } from "../index/write.js"; import { applyOperation, applyPlannedOperationSequence, type ApplyOptions } from "../operations/apply.js"; import { planOperation, type WikiPatchPlan } from "../operations/plan.js"; import { createParseCache } from "../operations/locate.js"; @@ -66,6 +68,8 @@ export interface MigrateOptions { registry?: EntityTypeRegistry; /** The code graph, for backfilling a moved grounding's `bodyHash`. */ graph?: GroundingGraph | null; + /** The checkout root, which keys the cached grounding baselines backfill reads; defaults to the scaffold's parent. */ + projectRoot?: string; /** Where `wiki.db` lives, when there is one. Absent is normal. */ indexPath?: string; actor?: WikiActor; @@ -346,6 +350,7 @@ function migrationReportFromInventory( classifications, (path) => classifications.get(path)?.candidates ?? [], options.graph ?? null, + options.projectRoot, ); report.groundingsMoved = [...groundings.moved.values()].reduce((sum, list) => sum + list.length, 0) + [...groundings.absorbed.values()].reduce((sum, entry) => sum + entry.count, 0); @@ -497,6 +502,7 @@ function migrateScaffoldHeld(options: MigrateOptions & { maintenanceLease: WikiM ...(options.registry === undefined ? {} : { registry: options.registry }), ...(options.readOnly === undefined ? {} : { readOnly: options.readOnly }), ...(options.graph === undefined || options.graph === null ? {} : { graph: options.graph }), + ...groundingResolverFor(options.graph ?? null), ...(options.indexPath === undefined ? {} : { indexPath: options.indexPath }), ...(options.onFileWritten === undefined ? {} : { onFileWritten: options.onFileWritten }), ...(options.beforeFileRename === undefined ? {} : { beforeFileRename: options.beforeFileRename }), @@ -520,6 +526,7 @@ function migrateScaffoldHeld(options: MigrateOptions & { maintenanceLease: WikiM classifications, (path) => classifications.get(path)?.candidates ?? [], options.graph ?? null, + options.projectRoot, ); report.groundingsAmbiguous = groundingPlan.diagnostics.length; report.diagnostics.push(...groundingPlan.diagnostics); @@ -682,6 +689,11 @@ function diffsFor(options: MigrateOptions, changed: Set): string[] { } /** Render a report the way section 13.6 asks, for a caller that wants text. */ +/** Health for the index refresh after each write, from the graph migration was given (#232). */ +function groundingResolverFor(graph: GroundingGraph | null): { resolveGrounding?: GroundingResolver } { + return graph === null ? {} : { resolveGrounding: (grounding) => resolveGrounding(grounding, graph) }; +} + export function renderMigrationReport(report: MigrationReport): string { const lines: string[] = []; lines.push(report.dryRun ? "Migration dry run" : "Migration applied"); @@ -832,6 +844,7 @@ export function planPinnedMigration(options: MigrateOptions): PinnedMigrationPla classifications, (path) => classifications.get(path)?.candidates ?? [], options.graph ?? null, + options.projectRoot, ); for (const file of inventory.files) { @@ -1034,6 +1047,7 @@ function applyPinnedMigrationHeld( ...(options.registry === undefined ? {} : { registry: options.registry }), ...(options.readOnly === undefined ? {} : { readOnly: options.readOnly }), ...(options.graph === undefined || options.graph === null ? {} : { graph: options.graph }), + ...groundingResolverFor(options.graph ?? null), ...(options.indexPath === undefined ? {} : { indexPath: options.indexPath }), ...(options.onFileWritten === undefined ? {} : { onFileWritten: options.onFileWritten }), ...(options.beforeFileRename === undefined ? {} : { beforeFileRename: options.beforeFileRename }), diff --git a/src/wiki/model/diagnostic.ts b/src/wiki/model/diagnostic.ts index eb2aa42a..64428c2c 100644 --- a/src/wiki/model/diagnostic.ts +++ b/src/wiki/model/diagnostic.ts @@ -206,6 +206,16 @@ export const WIKI_DIAGNOSTICS = { severity: "error", remediation: "Node id and fingerprint must come from live graph output. MEX will not accept caller-supplied values.", }, + /** + * A command that resolves groundings ran without a code graph it could + * trust (#232): the checkout's graph is stale, needs a rebuild, is corrupt, + * or changed during the pass. Info, because the command still did its work; + * it says how much of it was checked, not that the scaffold is wrong. + */ + CODE_GRAPH_UNAVAILABLE: { + severity: "info", + remediation: "Run `mex graph` to bring the code graph up to date, then run this command again.", + }, /** * Groundings split between a root `grounds_to` and the file-level `mex` map * (#226). Info, because both are read and the root ones belong to the diff --git a/src/wiki/service/__tests__/grounding-health.test.ts b/src/wiki/service/__tests__/grounding-health.test.ts new file mode 100644 index 00000000..4d6b3910 --- /dev/null +++ b/src/wiki/service/__tests__/grounding-health.test.ts @@ -0,0 +1,114 @@ +/** + * Index writes record grounding health from the graph they were given (#232). + * + * The write services took a graph to mint and verify groundings, but the index + * writes beside them never saw it, so a rebuild with a fresh graph stored NULL + * health — "nothing looked" — and `--health changed` matched nothing. + */ + +import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import type { GroundedNode, GroundingGraph } from "../../grounding/adapter.js"; +import { wikiGroundingStatus, wikiList } from "../read.js"; +import { wikiPrepareRebuildIndex, wikiRebuildIndex, wikiRefreshIndex } from "../write.js"; + +const ENTITY = "mx_01KR2E4K002H3ZYA9G0C4XV531"; +const NODE = "function:1a2b3c4d5e6f7a8b"; +const FINGERPRINT = "mh:64:0a0b0c0d"; + +function entityText(bodyHash: string): string { + return `--- +mex: + id: ${ENTITY} + type: component + status: promoted + revision: 1 + title: Auth + grounds_to: + - node: ${NODE} + fingerprint: ${FINGERPRINT} + bodyHash: ${bodyHash} +--- + +# Auth + +Rotates refresh tokens. +`; +} + +const roots: string[] = []; + +afterEach(() => { + for (const root of roots.splice(0)) { + try { + rmSync(root, { recursive: true, force: true }); + } catch { + // Windows keeps handles on just-closed SQLite files. + } + } +}); + +function scaffold(bodyHash = "body-1"): string { + const root = mkdtempSync(join(tmpdir(), "mex-wiki-health-")); + roots.push(root); + mkdirSync(join(root, "context"), { recursive: true }); + writeFileSync(join(root, "context", "auth.md"), entityText(bodyHash), "utf-8"); + return root; +} + +function graphWithBody(bodyHash: string): GroundingGraph { + const node: GroundedNode = { id: NODE, bodyHash, filePath: "src/auth.ts", startLine: 1, endLine: 5 }; + return { + getNode: (id) => (id === NODE ? node : null), + getFingerprint: (id) => (id === NODE ? FINGERPRINT : null), + reconcile: () => ({ kind: "GONE" }), + getBaselineSource: () => null, + }; +} + +function healthOf(root: string): string | null | undefined { + return wikiGroundingStatus({ scaffoldRoot: root, id: ENTITY }).data.entities[0]?.health; +} + +describe("index health from the graph a write was given (#232)", () => { + it("a rebuild with no graph stores no verdict", () => { + const root = scaffold(); + wikiRebuildIndex({ scaffoldRoot: root }); + expect(healthOf(root)).toBeNull(); + }); + + it("a rebuild with a graph stores the resolved health, and --health filters on it", () => { + const root = scaffold(); + wikiRebuildIndex({ scaffoldRoot: root, graph: graphWithBody("body-1") }); + expect(healthOf(root)).toBe("fresh"); + + wikiRebuildIndex({ scaffoldRoot: root, graph: graphWithBody("body-2") }); + expect(healthOf(root)).toBe("changed"); + expect(wikiList({ scaffoldRoot: root, health: "changed" }).data.entities.map((entry) => entry.id)).toEqual([ENTITY]); + }); + + it("a prepared rebuild publishes the same health on commit and nothing on discard", () => { + const discarded = scaffold(); + const dropped = wikiPrepareRebuildIndex({ scaffoldRoot: discarded, graph: graphWithBody("body-2") }); + dropped.preflight(); + dropped.discard(); + expect(existsSync(join(discarded, "wiki.db"))).toBe(false); + + const root = scaffold(); + const prepared = wikiPrepareRebuildIndex({ scaffoldRoot: root, graph: graphWithBody("body-2") }); + prepared.preflight(); + const committed = prepared.commit(); + expect(committed.data.entityCount).toBe(1); + expect(healthOf(root)).toBe("changed"); + }); + + it("a refresh resolves the files it re-reads", () => { + const root = scaffold(); + wikiRebuildIndex({ scaffoldRoot: root, graph: graphWithBody("body-1") }); + writeFileSync(join(root, "context", "auth.md"), entityText("body-0"), "utf-8"); + wikiRefreshIndex({ scaffoldRoot: root, changed: ["context/auth.md"], graph: graphWithBody("body-1") }); + expect(healthOf(root)).toBe("changed"); + }); +}); diff --git a/src/wiki/service/write.ts b/src/wiki/service/write.ts index cc844097..f54ff389 100644 --- a/src/wiki/service/write.ts +++ b/src/wiki/service/write.ts @@ -18,8 +18,10 @@ import { existsSync } from "node:fs"; import { diagnostic, hasBlockingDiagnostic, type WikiDiagnostic } from "../model/diagnostic.js"; import type { EntityTypeRegistry } from "../model/entity.js"; import type { GroundingGraph } from "../grounding/adapter.js"; +import { resolveGrounding } from "../grounding/resolve.js"; +import type { GroundingResolver } from "../index/write.js"; import { createHash } from "node:crypto"; -import { rebuildWikiIndex } from "../index/rebuild.js"; +import { prepareWikiRebuild, rebuildWikiIndex, type RebuildOptions, type RebuildResult } from "../index/rebuild.js"; import { refreshWikiIndex } from "../index/refresh.js"; import { payloadHashOf, planOperation, type PlanOptions, type WikiPatchPlan } from "../operations/plan.js"; import { readAuditLog, recordFor } from "../operations/audit.js"; @@ -53,6 +55,25 @@ export interface WikiWriteOptions extends WikiServiceOptions { maintenance?: WikiMaintenanceContext; } +/** + * Grounding health for the index a write publishes, from the graph that write + * was given (#232). + * + * The graph was already threaded through to mint and verify groundings, but + * the index writes beside it never saw it, so every row they wrote stored + * NULL health — "nothing looked" — even when something had. A rebuild, a + * refresh and a post-write refresh all take the same resolver or none, as + * `RebuildOptions.resolveGrounding` requires. + */ +function groundingResolverFor(options: WikiWriteOptions): { resolveGrounding?: GroundingResolver } { + const graph = options.graph ?? null; + return graph === null ? {} : { resolveGrounding: (grounding) => resolveGrounding(grounding, graph) }; +} + +function applyOptionsFrom(options: WikiWriteOptions): ApplyOptions { + return { ...planOptionsFrom(options), ...groundingResolverFor(options) }; +} + function planOptionsFrom(options: WikiWriteOptions): PlanOptions { return { scaffoldRoot: resolve(options.scaffoldRoot), @@ -157,7 +178,7 @@ export function wikiApplyOperation( }; } - const applyOptions: ApplyOptions = planOptionsFrom(options); + const applyOptions: ApplyOptions = applyOptionsFrom(options); const result = options.plan !== undefined && options.expectedPreviewRevision !== undefined ? applyPlannedOperation(options.plan, { ...applyOptions, @@ -241,13 +262,43 @@ export interface RebuildData { * configuration: a command the user runs. */ export function wikiRebuildIndex(options: WikiWriteOptions): ServiceResult { - const result = rebuildWikiIndex({ + return rebuildServiceResult(rebuildWikiIndex(rebuildOptionsFrom(options))); +} + +/** + * `wiki rebuild-index` in two phases, for a caller holding a graph snapshot + * that must prove itself fresh before the index is published (#232). + * + * The candidate is built and preflighted while the snapshot is open; `commit` + * publishes it and `discard` drops it. The same shape the repository adapter + * hands the graph's two-phase publication, so a CLI rebuild and a Hub rebuild + * cannot publish health from a graph that changed under them. + */ +export function wikiPrepareRebuildIndex(options: WikiWriteOptions): { + preflight(): void; + commit(): ServiceResult; + discard(): void; +} { + const prepared = prepareWikiRebuild(rebuildOptionsFrom(options)); + return { + preflight: () => prepared.preflight(), + commit: () => rebuildServiceResult(prepared.commit()), + discard: () => prepared.discard(), + }; +} + +function rebuildOptionsFrom(options: WikiWriteOptions): RebuildOptions { + return { scaffoldRoot: resolve(options.scaffoldRoot), ...(options.indexPath === undefined ? {} : { indexPath: options.indexPath }), ...(options.exclude === undefined ? {} : { exclude: options.exclude }), ...(options.registry === undefined ? {} : { registry: options.registry }), ...(options.maintenance === undefined ? {} : { maintenance: options.maintenance }), - }); + ...groundingResolverFor(options), + }; +} + +function rebuildServiceResult(result: RebuildResult): ServiceResult { return { data: { indexPath: result.indexPath, @@ -298,6 +349,7 @@ export function wikiRefreshIndex( ...(options.exclude === undefined ? {} : { exclude: options.exclude }), ...(options.registry === undefined ? {} : { registry: options.registry }), ...(options.maintenance === undefined ? {} : { maintenance: options.maintenance }), + ...groundingResolverFor(options), }); if (!result.ok) { return { @@ -364,7 +416,7 @@ export interface MigrateData { * on a scaffold it handled correctly. */ export function wikiMigrate( - options: WikiWriteOptions & { dryRun?: boolean; now?: () => string }, + options: WikiWriteOptions & { dryRun?: boolean; now?: () => string; projectRoot?: string }, ): ServiceResult { const migrateOptions = { scaffoldRoot: resolve(options.scaffoldRoot), @@ -374,6 +426,7 @@ export function wikiMigrate( ...(options.graph === undefined ? {} : { graph: options.graph }), ...(options.indexPath === undefined ? {} : { indexPath: options.indexPath }), ...(options.now === undefined ? {} : { now: options.now }), + ...(options.projectRoot === undefined ? {} : { projectRoot: options.projectRoot }), }; const dryRun = options.dryRun === true; const report = dryRun ? planMigration(migrateOptions) : migrateScaffold(migrateOptions); @@ -467,7 +520,7 @@ export function wikiRegenerateViews( }; } - const applied = applyGeneratedViews({ ...planOptionsFrom(options), views }); + const applied = applyGeneratedViews({ ...applyOptionsFrom(options), views }); return { data: { examined, changedFiles: applied.changedFiles, dryRun: false }, diagnostics: [...diagnostics, ...suppressIndexRefresh(applied.diagnostics, options)],