From 9abd80c975771cdf9097ed9db310739ef5e8f12e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 08:31:07 +0000 Subject: [PATCH 1/8] Lanes fill the terminal, like the rows do MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The strip stopped short of the right edge for two independent reasons in one expression: const laneWidth = () => Math.max(10, Math.min(width() - 61, 80)) A hard cap of 80 cells, so past ~150 columns it stopped growing at all — 71 blank columns on a 200-column terminal, 111 on a 240. And a reserve of 61 columns for chrome that actually prints 49, so even below the cap there was a constant 16-column gap. Part of that reserve was stale: it was sized for a mode legend that included "3 calls", and stayed 61 when that mode was removed. The strip is now width() + 2 - LANE_CHROME. The +2 is the "│ " a row draws outside its own width, which a lane label carries inside its 12, so the lane rows end on the same column as the rows and the status line rather than two short. No ceiling: the strip is a window onto an unbounded layout, so more cells is more events visible and less scrolling, and laneOverview already hides the overview track once nothing is off-screen. LANE_CHROME is derived from the label and legend strings, which now live in core/lanes.ts, instead of being written down — that is the actual fix for this class of bug. Four tests pin the invariants it rests on: the legend is the same width whichever mode is selected, the label column is fixed width for every lane name with or without a cue, the suffix is fixed width with or without a cue, and LANE_CHROME equals what a lane row really spends off-strip. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BrwAnTfTmJSmFwVn3DDFxr --- CHANGELOG.md | 11 +++++++++++ DESIGN.md | 10 ++++++++++ src/core/lanes.ts | 24 ++++++++++++++++++++++++ src/tui/route.tsx | 14 ++++++++------ test/lanes.test.ts | 27 ++++++++++++++++++++++++++- 5 files changed, 79 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 33e42d8..2f7ae63 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,16 @@ # Changelog +## Unreleased + +- The timeline lanes fill the terminal, ending on the same column as the rows and the status + line. They were capped at 80 cells and reserved a hardcoded 61 columns for the label and the + mode legend that actually print 49, so a 200-column terminal left ~71 blank columns to the + right of the strip and even a 130-column one left 16. The reserve is now derived from the + strings themselves (`core/lanes.ts#LANE_CHROME`), so shortening the mode legend widens the + strip instead of leaving a gap — which is what went wrong when the "3 calls" mode was + removed and the 61 stayed 61. A wider strip shows more events before scrolling, and the + overview track already hides itself once nothing is off-screen. + ## 0.2.3 — 2026-09-04 - **The inspector shows the whole field now, and pages through it.** It used to cap each field diff --git a/DESIGN.md b/DESIGN.md index f91aef3..d6a5507 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -643,6 +643,16 @@ labeled shows only `L`-labelled rows). `/` filters rows incrementally by role, t name, label, and text — every token must match, like Pi. Folding state resets on filter change (as in Pi) and is otherwise remembered per session in `api.kv`. +**Lane width (0.2.4).** The strip is `width() + 2 - LANE_CHROME`: the terminal, minus the +12-column lane label and the mode legend, plus the two columns a row spends on its `│ ` prefix +*outside* its own width (a lane label carries its own), so all three lane rows end on the same +column as the rows and the status line. `LANE_CHROME` is *measured* from the label and legend +strings in `core/lanes.ts`, never written down — it was a literal `61` against a legend that +printed 49, and it stayed `61` when dropping the "Calls" mode made the legend shorter still. +There is no ceiling on the strip: it is a window onto an unbounded layout, so more cells is +more events visible and less scrolling, and `laneOverview` hides the overview track once +nothing is off-screen. + ### 7.6 Narrow terminals Below 110 columns the inspector is hidden and `i` opens it full-screen; below 80 the diff --git a/src/core/lanes.ts b/src/core/lanes.ts index 154a25b..cdc89a8 100644 --- a/src/core/lanes.ts +++ b/src/core/lanes.ts @@ -42,6 +42,30 @@ function spanOf(m: TranscriptMessage): number { * tokens — `tokens` rides along only for the inspector. */ +/** + * Chrome around the strip on a lane row: the fixed-width label on the left and the mode line + * on the right. Both are fixed width *by construction* — `laneCue` caps at three digits, and + * each mode label is the same length selected or not — so the strip can be sized as + * "everything else" without measuring per frame. + * + * They live here, and `LANE_CHROME` is derived from them rather than written down, because the + * width was a literal `61` that stayed `61` when dropping the "3 calls" mode shortened the + * mode line: the strip quietly gave up ~10 columns and stopped short of the right edge. + */ +export const laneLabel = (name: string, cue = "") => `│ ${name} ${cue}`.padEnd(LANE_LABEL_WIDTH) +export const LANE_LABEL_WIDTH = 12 + +export function laneModeLine(mode: LaneMode): string { + return `${mode === "duration" ? "[1] Duration" : " 1 duration"} · ${mode === "turns" ? "[2] Turns" : " 2 turns"} · 0 off` +} + +export function laneSuffix(cue: string, mode: LaneMode): string { + return `${cue.padStart(4).padEnd(5)}${laneModeLine(mode)}` +} + +/** Columns a lane row spends on anything that is not the strip. */ +export const LANE_CHROME = LANE_LABEL_WIDTH + laneSuffix("", "turns").length + export type LaneEvent = { lane: "input" | "model" | "tools" /** `context` = compaction/branch summary: machine-written context, not the human prompting */ diff --git a/src/tui/route.tsx b/src/tui/route.tsx index 7808a94..90e2abe 100644 --- a/src/tui/route.tsx +++ b/src/tui/route.tsx @@ -16,7 +16,7 @@ import type { Transcript } from "../core/transcript.js" import type { JournalStore } from "../shared/store.js" import { applyCrop, branchLabel, BRANCH_DIALOG, clip as clipTo, copyText, createNamedBranch, describeTail, executeJump, executeUndo, jumpDialogOptions, jumpDialogTitle, mergeBranch, mergeDialogOptions, mergeDialogTitle, mergePickerFigures, MERGE_TRUST, setLabel, UNDO_KEY, type ActionContext, type MergeMode, type SummaryChoice } from "./actions.js" import { decisionSummary, exportDecisions, renderDecision } from "../core/decision.js" -import { layoutEventStrip, overviewTrack, stripIndexFor, windowFor, type LaneMode, type StripCell } from "../core/lanes.js" +import { laneLabel, laneSuffix, layoutEventStrip, overviewTrack, stripIndexFor, windowFor, LANE_CHROME, type LaneMode, type StripCell } from "../core/lanes.js" import { bar, consumers, type Consumer, type ConsumerEntry } from "../core/consumers.js" import { hasEditor } from "./editor.js" import fs from "node:fs" @@ -410,8 +410,12 @@ export function TreeRoute(props: TreeRouteProps) { const helpHeight = () => (panel() === "help" ? Math.min(HELP.length, Math.max(0, size().rows - 12)) : 0) const width = () => Math.max(60, cols() - 4) // ---- lane geometry (the lanes themselves are further down) ---------------- - // 61 = the 12-cell label column + the `N…` cue + the mode legend that follows the Input lane - const laneWidth = () => Math.max(10, Math.min(width() - 61, 80)) + /** The strip fills the terminal, ending on the same column as the rows and the status line. + * The `+ 2` is the `│ ` a row draws *outside* its own width — a lane label carries its own, + * so those two columns come back to the strip. No ceiling: the strip is a window onto an + * unbounded layout, so more cells is more events visible and less scrolling, and the + * overview track hides itself once nothing is off-screen (`laneOverview`). */ + const laneWidth = () => Math.max(10, width() + 2 - LANE_CHROME) const layout = createMemo(() => layoutEventStrip(live() ?? EMPTY_TRANSCRIPT, laneMode(), filter())) /** DESIGN.md §7.6: below 80 columns the strip is the Input lane alone. */ const showAllLanes = () => cols() >= 80 @@ -615,8 +619,6 @@ export function TreeRoute(props: TreeRouteProps) { const laneOffset = () => laneStart() ?? Math.max(0, layout().totalWidth - laneWidth()) const hiddenLeft = createMemo(() => layout().spans.filter((s) => s.end <= laneOffset()).length) const hiddenRight = createMemo(() => layout().spans.filter((s) => s.start >= laneOffset() + laneWidth()).length) - /** `│ Input …12 `: one fixed-width column, so the lanes stay aligned whatever the cues say. */ - const laneLabel = (name: string, cue = "") => `│ ${name} ${cue}`.padEnd(12) const laneCue = (n: number) => (n > 999 ? "999" : String(n)) const cellColor = (cell: StripCell): unknown => { if (cell.error) return t.error @@ -1518,7 +1520,7 @@ export function TreeRoute(props: TreeRouteProps) { {"no input".padEnd(laneWidth())}}> {(r) => {r.text}} - {`${(hiddenRight() > 0 ? `${laneCue(hiddenRight())}…` : "").padStart(4).padEnd(5)}${laneMode() === "duration" ? "[1] Duration" : " 1 duration"} · ${laneMode() === "turns" ? "[2] Turns" : " 2 turns"} · 0 off`} + {laneSuffix(hiddenRight() > 0 ? `${laneCue(hiddenRight())}…` : "", laneMode())} diff --git a/test/lanes.test.ts b/test/lanes.test.ts index 2ff53a2..4b53348 100644 --- a/test/lanes.test.ts +++ b/test/lanes.test.ts @@ -1,5 +1,5 @@ import { describe, expect, test } from "bun:test" -import { eventAllowed, layoutEventStrip, overviewTrack, stripIndexFor, windowFor, type EventLayout } from "../src/core/lanes.js" +import { eventAllowed, laneLabel, laneModeLine, laneSuffix, layoutEventStrip, overviewTrack, stripIndexFor, windowFor, type EventLayout, LANE_CHROME, LANE_LABEL_WIDTH } from "../src/core/lanes.js" import { bar, consumers } from "../src/core/consumers.js" import type { Transcript, TranscriptMessage } from "../src/core/transcript.js" import { assistant, buildFixture, OPEN, user } from "./fixtures/tree.js" @@ -219,3 +219,28 @@ describe("overview track", () => { expect(track.at(-1)).toBe("window") }) }) + +describe("lane chrome", () => { + test("the mode line is the same width whichever mode is selected", () => { + // the strip is sized as "the terminal minus the chrome", so a mode line that changed width + // with the selection would shift every pill sideways on `1`/`2` + expect(laneModeLine("turns").length).toBe(laneModeLine("duration").length) + }) + + test("the label column is fixed width, cue or no cue, longest lane name or shortest", () => { + for (const name of ["Input", "Model", "Tools"]) { + expect(laneLabel(name).length).toBe(LANE_LABEL_WIDTH) + expect(laneLabel(name, "…999").length).toBe(LANE_LABEL_WIDTH) + } + }) + + test("the suffix is fixed width, cue or no cue", () => { + expect(laneSuffix("", "turns").length).toBe(laneSuffix("999…", "duration").length) + }) + + test("LANE_CHROME is what a lane row actually spends off-strip", () => { + // measured, never written down: the old reserve was a literal 61 and stayed 61 when the + // "3 calls" mode went away, so the strip stopped ~10 columns short of the right edge + expect(LANE_CHROME).toBe(laneLabel("Input", "…999").length + laneSuffix("999…", "turns").length) + }) +}) From 25a78d3d6530f13d464227ef4a711645cb8328a5 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 09:41:41 +0000 Subject: [PATCH 2/8] Count the system prompt in the consumers view MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Consumers walked the transcript only, so it omitted the system prompt and its total could not be reconciled with the `ctx …` gauge two lines above it — which reads tokens.input and therefore does include it. On an agent with a large base prompt and an AGENTS.md that is a silently missing 5-15k in the one view whose whole job is "where did my window go". experimental.chat.system.transform is the only place the plugin can see it. The server half snapshots output.system there, before pushing its own note — counting our note as part of the user's prompt would be a small lie in exactly the wrong view — names each part by a shallow text heuristic (AGENTS.md, CLAUDE.md, environment, else base prompt), and writes system-.json. That file is overwritten each request, never appended: it is current state outside the message tree, not a mutation with history, so /undo has nothing to do with it and it must not grow the way the journal does. The TUI reads it on the same poll as the tree. It renders as one `≡ system prompt` bucket with an entry per part, croppable: false and a note, reusing the mechanism `(thinking)` already had. `y` in the consumers panel copies the selected part in full — the only way to read one, since it is not a message and so has no row and no inspector of its own. Absence means unknown, never zero: a session whose prompt we have not seen yet has no bucket rather than a 0. The capture is defensive in every direction and logs what it actually saw (debug "system.captured", parts + chars), which is also how to confirm on a live server that output.system arrives carrying OpenCode's own parts rather than empty — the assumption the whole feature rests on, and one this container cannot check because opencode serve will not bind here. Tool-definition schemas remain uncounted: client.tool.list gives their descriptions but not what the provider is really sent, so that estimate would be rough enough to mislead. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BrwAnTfTmJSmFwVn3DDFxr --- CHANGELOG.md | 10 ++++++++++ DESIGN.md | 25 ++++++++++++++++++++++++ docs/USAGE.md | 2 +- src/core/consumers.ts | 30 ++++++++++++++++++++++++++--- src/core/journal.ts | 22 +++++++++++++++++++++ src/server/index.ts | 45 ++++++++++++++++++++++++++++++++++++++++++- src/shared/store.ts | 32 +++++++++++++++++++++++++++++- src/tui/route.tsx | 27 ++++++++++++++++++++++++-- test/lanes.test.ts | 37 +++++++++++++++++++++++++++++++++++ test/store.test.ts | 33 +++++++++++++++++++++++++++++++ 10 files changed, 255 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2f7ae63..754629e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,16 @@ ## Unreleased +- Consumers (`s`) counts the **system prompt**. It walked the transcript only, so its total + could never be reconciled with the `ctx …` gauge two lines above it, which reads + `tokens.input` and does include the system prompt — a silently missing 5–15k on an agent with + a large base prompt and an `AGENTS.md`. The server half now snapshots what the provider is + really sent (in `experimental.chat.system.transform`, before adding its own note) and the view + shows a `≡ system prompt` bucket with one entry per part — base prompt, `AGENTS.md`, + environment — so you can see that your rules file costs 4k. It is not croppable, and `y` + copies a part in full. A session whose prompt the plugin has not seen yet shows no bucket + rather than a misleading zero. + - The timeline lanes fill the terminal, ending on the same column as the rows and the status line. They were capped at 80 cells and reserved a hardcoded 61 columns for the label and the mode legend that actually print 49, so a 200-column terminal left ~71 blank columns to the diff --git a/DESIGN.md b/DESIGN.md index d6a5507..31f6995 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -635,6 +635,31 @@ Three changes, in the order they matter: `y` stays the answer for actually *reading* a large payload — copy it somewhere with search and folding. The scroller is for "there were twelve more lines and I want to glance at them". +**The system prompt (0.2.4).** Consumers walked the transcript only, so it omitted the system +prompt and its total could not be reconciled with the `ctx …` gauge above it — which reads +`tokens.input` and therefore *does* include it. On an agent with a large base prompt and an +`AGENTS.md` that is a silently missing 5–15k in the one view whose job is "where did my window +go". + +`experimental.chat.system.transform` is the only place the plugin can see it. The server half +snapshots `output.system` there **before pushing its own note** (counting our note as part of +the user's prompt would be a small lie in exactly the wrong view), names each part by a shallow +text heuristic (`AGENTS.md`, `CLAUDE.md`, `environment`, …, else `base prompt`), and writes it +to `system-.json` — *overwritten* each request, not appended: it is current state +outside the message tree, not a mutation with history, so `/undo` has nothing to do with it and +it must not grow the way the journal does. The TUI reads it on the same poll as the tree. + +It appears as one `≡ system prompt` bucket with an entry per part, `croppable: false` and the +note `sent whole every request · not croppable (y copies a part)` — the same mechanism +`(thinking)` already uses. `y` in the consumers panel copies the selected part in full, which is +the only way to read one: it is not a message, so it has no row and no inspector of its own. + +Two honest limits. It is mostly *diagnostic* — OpenCode's base prompt cannot be cropped, though +`AGENTS.md` being 4k is a lever the user owns. And absence means **unknown**, never zero: a +session whose prompt we have not seen yet simply has no bucket, rather than reporting 0. +Tool-definition schemas are still uncounted; `client.tool.list` gives their descriptions but not +what the provider is really sent, so that estimate would be rough enough to mislead. + ### 7.5 Filters and search (from Pi) `f` cycles `default → no-tools → user-only → labeled → all` (default hides diff --git a/docs/USAGE.md b/docs/USAGE.md index 03e7b2f..6c7dd9e 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -111,7 +111,7 @@ appended to the trunk as a normal message.* | `c` `space` `a` `t` `⏎` | crop mode: mark (`space` alone enters it on a croppable row), auto-mark (≥10k tokens, older than 2 turns), result⇄turn, apply | | `u` (`x`) | undo | | `D` `E` | decisions panel, export `ctree-decisions.md` | -| `s` | consumers: what is filling the context (`⏎` opens a bucket, `space` marks one entry for crop) | +| `s` | consumers: what is filling the context (`⏎` opens a bucket, `space` marks one entry for crop, `y` copies one). Includes a `≡ system prompt` bucket broken down by part (base prompt, `AGENTS.md`, …) once the plugin has seen one request for the session — it is not croppable, but it is counted, so the total reconciles with the `ctx …` gauge | | `i` | inspector pane on/off (auto-hidden under 110 columns) | | `i` `I` `PgUp` `PgDn` | inspector in the side pane / full screen; page through a long payload or result. The pane shows every line it has, sized to your terminal, with `12–40 of 118` at the foot when there is more; `y` copies the untruncated text. Below 110 columns the side pane does not fit, so `i` opens full screen directly | | `1 2` `0` | timeline lanes, x-axis by duration / one cell per event; `0` off. `│` marks a turn boundary, and the lanes show whatever the `f` filter shows — so `f` → `tools-only` is the "what did I run" view in both the rows and the lanes | diff --git a/src/core/consumers.ts b/src/core/consumers.ts index c2de33e..8eb9fe3 100644 --- a/src/core/consumers.ts +++ b/src/core/consumers.ts @@ -17,7 +17,7 @@ export type ConsumerEntry = { export type Consumer = { source: string - kind: "tool" | "assistant" | "user" | "decision" | "summary" | "reasoning" + kind: "tool" | "assistant" | "user" | "decision" | "summary" | "reasoning" | "system" tokens: number count: number /** share of this transcript's own total */ @@ -32,8 +32,22 @@ export type Consumer = { const THINKING = "(thinking)" const THINKING_NOTE = "provider reasoning · not croppable" +const SYSTEM = "≡ system prompt" +const SYSTEM_NOTE = "sent whole every request · not croppable (y copies a part)" -export function consumers(transcript: Transcript, opts: { cropped?: Set; limit?: number } = {}): Consumer[] { +export function consumers( + transcript: Transcript, + opts: { + cropped?: Set + limit?: number + /** + * The system parts the provider is really sent, captured by the server half. Without it + * this view silently omits them, and its total cannot be reconciled with the `ctx …` gauge + * — which reads `tokens.input` and so *does* include them (DESIGN.md §7.4). + */ + system?: { name: string; text: string }[] + } = {}, +): Consumer[] { const acc = new Map() const add = (source: string, kind: Consumer["kind"], tokens: number, message: TranscriptMessage, part: StepPart) => { const c = acc.get(source) ?? { source, kind, tokens: 0, count: 0, share: 0, entries: [] } @@ -62,6 +76,16 @@ export function consumers(transcript: Transcript, opts: { cropped?: Set; } else if (p.type === "reasoning") add(THINKING, "reasoning", estimateTokens(p.text ?? ""), m, p) } } + // one bucket, one entry per part, so "AGENTS.md is 4k" is visible next to "bash is 30k" + for (const part of opts.system ?? []) { + const c = acc.get(SYSTEM) ?? { source: SYSTEM, kind: "system" as const, tokens: 0, count: 0, share: 0, entries: [] } + const tokens = estimateTokens(part.text) + c.tokens += tokens + c.count += 1 + c.entries.push({ messageID: "", tokens, preview: `${part.name}: ${part.text.slice(0, 120).replace(/\s+/g, " ").trim()}`, croppable: false }) + acc.set(SYSTEM, c) + } + const total = [...acc.values()].reduce((s, c) => s + c.tokens, 0) || 1 const limit = opts.limit !== undefined && opts.limit > 0 ? opts.limit : undefined return [...acc.values()] @@ -69,7 +93,7 @@ export function consumers(transcript: Transcript, opts: { cropped?: Set; ...c, share: c.tokens / total, ...(limit === undefined ? {} : { shareOfWindow: c.tokens / limit }), - ...(c.kind === "reasoning" ? { note: THINKING_NOTE } : {}), + ...(c.kind === "reasoning" ? { note: THINKING_NOTE } : c.kind === "system" ? { note: SYSTEM_NOTE } : {}), entries: c.entries.sort((a, b) => b.tokens - a.tokens), })) .sort((a, b) => b.tokens - a.tokens) diff --git a/src/core/journal.ts b/src/core/journal.ts index a1f89ce..737f87b 100644 --- a/src/core/journal.ts +++ b/src/core/journal.ts @@ -131,6 +131,28 @@ export type JournalEntry = z.infer export type JournalEntryType = JournalEntry["type"] /** Parse one JSONL line into a validated journal entry, or `undefined` if it is malformed. */ +/** + * What the provider is actually sent as its system prompt, captured by the server half in + * `experimental.chat.system.transform` (DESIGN.md §7.4). It is *not* a journal entry: it has + * no history, it is not a mutation the user made, and `/undo` has nothing to do with it — the + * store keeps it in its own per-session file, overwritten each request. + */ +export const SystemPart = z.object({ + /** A name for the part, guessed from its own text — providers hand us an unlabelled array. */ + name: z.string(), + chars: z.number(), + text: z.string(), +}) +export type SystemPart = z.infer + +export const SystemSnapshot = z.object({ + v: z.literal(1), + /** When the provider was last sent this; the prompt can change between requests. */ + ts: z.number(), + parts: z.array(SystemPart), +}) +export type SystemSnapshot = z.infer + export function parseJournalLine(line: string): JournalEntry | undefined { const trimmed = line.trim() if (!trimmed) return undefined diff --git a/src/server/index.ts b/src/server/index.ts index 7b72baa..75043c7 100644 --- a/src/server/index.ts +++ b/src/server/index.ts @@ -23,6 +23,43 @@ import { cacheShare, contextSizeOf, formatK, type MinimalMessage as MinimalToken import { parseForkTitle } from "../core/adopt.js" import { adoptNativeForks } from "../shared/adopt.js" +/** + * Name the parts of an unlabelled system prompt so the consumers view can say *which* part + * costs what — "my AGENTS.md is 4k" is a lever the user can actually pull, where OpenCode's + * base prompt is not. + * + * The heuristics are deliberately shallow and never throw: an unrecognised part is "system + * prompt" plus its index, which is still worth counting. + */ +function nameSystemPart(text: string, index: number): string { + const head = text.slice(0, 400).toLowerCase() + if (head.includes("agents.md")) return "AGENTS.md" + if (head.includes("claude.md")) return "CLAUDE.md" + if (/\bcontext notes:/.test(head)) return "context-tree note" + if (head.includes("") || head.includes("working directory")) return "environment" + if (head.includes("today's date") || head.includes("current date")) return "date" + return index === 0 ? "base prompt" : `system prompt ${index + 1}` +} + +/** + * Record the system parts for a session. Best effort in every direction: if the host ever hands + * us an empty array (we append to it, so it arrives carrying OpenCode's own parts — the debug + * line below is how to confirm that on a live server), nothing is written and every reader + * treats the absence as "unknown", never as "zero". + */ +function captureSystem(store: JournalStore, sessionID: string, system: readonly string[]): void { + try { + const parts = system + .map((text, i) => ({ name: nameSystemPart(text, i), chars: text.length, text })) + .filter((p) => p.chars > 0) + debug("system.captured", { sessionID, parts: parts.length, chars: parts.reduce((n, p) => n + p.chars, 0) }) + if (parts.length === 0) return + store.writeSystem(sessionID, { v: 1, ts: Date.now(), parts }) + } catch (e) { + debug("system.capture.failed", { error: e instanceof Error ? e.message : String(e) }) + } +} + export const server: Plugin = async ({ worktree, client, directory }, options) => { // same option parsing as the TUI half, so both write to the same place (docs/USAGE.md) const mode: StorageMode = options?.["storage"] === "global" ? "global" : "local" @@ -255,9 +292,15 @@ export const server: Plugin = async ({ worktree, client, directory }, options) = ) }, - // DESIGN.md §6.8: a system note so the model reads ◆ / ✂ markers correctly + // DESIGN.md §6.8: a system note so the model reads ◆ / ✂ markers correctly, and §7.4: + // this is the one place the plugin can see what the provider is really sent as its system + // prompt, so it is also where the consumers view gets the bucket it was missing. "experimental.chat.system.transform": async ({ sessionID }, output) => { if (!sessionID || !(await journal).stateForSession(sessionID)) return + // snapshot what OpenCode assembled, BEFORE our own note joins it — the note is ours, and + // counting it as part of the user's prompt would be a small lie in the one view that + // exists to say where the context went + captureSystem(await journal, sessionID, output.system) output.system.push( "Context notes: messages starting with ◆ are decision records confirmed by the user — treat them as settled facts. Tool results reading [cropped: …] or turns reading [dropped turn …] were removed from your context on purpose to save space; if you need one back, ask the user to restore it (they can with /undo in the context tree).", ) diff --git a/src/shared/store.ts b/src/shared/store.ts index 81e55a2..9b21cce 100644 --- a/src/shared/store.ts +++ b/src/shared/store.ts @@ -7,7 +7,7 @@ */ import fs from "node:fs" import path from "node:path" -import { foldJournal, parseJournal, type JournalActor, type JournalEntry, type TreeState } from "../core/journal.js" +import { foldJournal, parseJournal, SystemSnapshot, type JournalActor, type JournalEntry, type TreeState } from "../core/journal.js" export type StorageMode = "local" | "global" @@ -54,6 +54,36 @@ export class JournalStore { if (!fs.existsSync(gitignorePath)) fs.writeFileSync(gitignorePath, "*\n") } + /** + * Where a session's system-prompt snapshot lives. One file per session, **overwritten** each + * request rather than appended: it is the current state of something outside the message + * tree, not a mutation with history, so it must not grow the way the journal does. + */ + private systemPath(sessionID: string): string { + return path.join(this.baseDir, `system-${sessionID}.json`) + } + + /** Record the system parts the provider is actually being sent (server half, at request + * time). Best effort: a session whose prompt we never see simply has no snapshot, and every + * reader treats that as "unknown" rather than "zero". */ + writeSystem(sessionID: string, snapshot: SystemSnapshot): void { + this.ensureDir() + const tmp = `${this.systemPath(sessionID)}.${process.pid}.${Date.now()}.tmp` + fs.writeFileSync(tmp, `${JSON.stringify(snapshot, null, 2)}\n`) + fs.renameSync(tmp, this.systemPath(sessionID)) + } + + /** The last snapshot for a session, or undefined when we have never seen its prompt. */ + readSystem(sessionID: string): SystemSnapshot | undefined { + try { + const raw = JSON.parse(fs.readFileSync(this.systemPath(sessionID), "utf8")) as unknown + const parsed = SystemSnapshot.safeParse(raw) + return parsed.success ? parsed.data : undefined + } catch { + return undefined + } + } + private journalPath(treeId: string): string { return path.join(this.baseDir, `${treeId}.jsonl`) } diff --git a/src/tui/route.tsx b/src/tui/route.tsx index 90e2abe..d7aab5b 100644 --- a/src/tui/route.tsx +++ b/src/tui/route.tsx @@ -796,7 +796,15 @@ export function TreeRoute(props: TreeRouteProps) { createEffect(on(() => `${current()?.id ?? ""}:${showInspectorFull()}`, () => setInspectorTop(0))) // ---- consumers ------------------------------------------------------------- - const consumerRows = createMemo(() => (live() ? consumers(live()!, { cropped: alreadyCropped(), limit: contextLimit() }) : [])) + /** The system parts the server half captured for this session, if it has seen a request yet. + * Absent means "we have not seen this session's prompt", never "it costs nothing", so the + * bucket simply does not appear rather than reporting a misleading zero. */ + const systemParts = createMemo(() => { + tick() // the server half rewrites the snapshot on each request; follow the same poll the tree does + if (!sessionID) return undefined + return store.readSystem(sessionID)?.parts + }) + const consumerRows = createMemo(() => (live() ? consumers(live()!, { cropped: alreadyCropped(), limit: contextLimit(), system: systemParts() }) : [])) /** Buckets plus the entries of every expanded one, flattened so ↑↓ walks both. */ type ConsumerLine = { bucket: Consumer; entry?: ConsumerEntry } const consumerLines = createMemo((): ConsumerLine[] => @@ -850,6 +858,21 @@ export function TreeRoute(props: TreeRouteProps) { } function copySelected() { + // in the consumers panel `y` copies the selected entry — the only way to read a system + // part in full, since it is not a message and has no row of its own + if (panel() === "consumers") { + const line = consumerLine() + const part = line?.entry && line.bucket.kind === "system" ? systemParts()?.find((p) => line.entry!.preview.startsWith(`${p.name}:`)) : undefined + const text = part?.text ?? line?.entry?.preview ?? "" + if (!text) return + try { + const { hint } = copyText(api, text, directory) + notify(`copied ${text.length} chars → ${hint}`) + } catch (e) { + notify(`copy failed: ${e instanceof Error ? e.message : String(e)}`) + } + return + } const row = current() if (!row || row.kind === "branch" || row.kind === "separator") return const tr = row.sessionID === sessionID ? live() : others()[row.sessionID] @@ -1370,7 +1393,7 @@ export function TreeRoute(props: TreeRouteProps) { { name: "ctree.inspector_up", hidden: true, enabled: inspectorOpen, run: () => scrollInspector(-1) }, { name: "ctree.inspector_down", hidden: true, enabled: inspectorOpen, run: () => scrollInspector(1) }, { name: "ctree.consumers", hidden: true, enabled: () => !inCrop(), run: () => setPanel(panel() === "consumers" ? "tree" : "consumers") }, - { name: "ctree.copy", hidden: true, enabled: treeIdle, run: () => copySelected() }, + { name: "ctree.copy", hidden: true, enabled: () => treeIdle() || panel() === "consumers", run: () => copySelected() }, { name: "ctree.mode_duration", hidden: true, enabled: treePanel, run: () => setLane("duration") }, { name: "ctree.mode_turns", hidden: true, enabled: treePanel, run: () => setLane("turns") }, { name: "ctree.lanes_off", hidden: true, enabled: treePanel, run: () => { setLanesOn(false); api.kv.set("ctree.lanesOn", false) } }, diff --git a/test/lanes.test.ts b/test/lanes.test.ts index 4b53348..2580370 100644 --- a/test/lanes.test.ts +++ b/test/lanes.test.ts @@ -28,6 +28,43 @@ describe("consumers", () => { }) }) + test("the system prompt is a bucket, so the view reconciles with the ctx gauge", () => { + // without it this view walks the transcript only, and silently omits a chunk the header's + // `ctx …` (tokens.input) does include + const system = [ + { name: "base prompt", text: "x".repeat(8000) }, + { name: "AGENTS.md", text: "y".repeat(4000) }, + ] + const withSystem = consumers(open, { system }) + const without = consumers(open) + const bucket = withSystem.find((c) => c.kind === "system")! + expect(bucket.source).toBe("≡ system prompt") + expect(bucket.tokens).toBe(3000) // (8000 + 4000) / 4 + expect(bucket.count).toBe(2) + expect(withSystem.reduce((n, c) => n + c.tokens, 0)).toBe(without.reduce((n, c) => n + c.tokens, 0) + 3000) + }) + + test("its parts are separate entries, biggest first, and none is croppable", () => { + const system = [ + { name: "base prompt", text: "x".repeat(400) }, + { name: "AGENTS.md", text: "y".repeat(4000) }, + ] + const bucket = consumers(open, { system }).find((c) => c.kind === "system")! + expect(bucket.entries.map((e) => e.preview.split(":")[0])).toEqual(["AGENTS.md", "base prompt"]) + expect(bucket.entries.every((e) => !e.croppable)).toBe(true) + expect(bucket.note).toContain("not croppable") + }) + + test("no snapshot means no bucket — absent is 'unknown', never 'zero'", () => { + expect(consumers(open).some((c) => c.kind === "system")).toBe(false) + expect(consumers(open, { system: [] }).some((c) => c.kind === "system")).toBe(false) + }) + + test("shares still sum to 1 once the system prompt is in", () => { + const cs = consumers(open, { system: [{ name: "base prompt", text: "x".repeat(8000) }] }) + expect(cs.reduce((n, c) => n + c.share, 0)).toBeCloseTo(1, 5) + }) + const T = (messages: TranscriptMessage[]): Transcript => ({ sessionID: "s", title: "strip", status: "available", messages }) /** cells that belong to any lane, per index — the axis is shared, so this must never exceed 1 */ diff --git a/test/store.test.ts b/test/store.test.ts index 4b7d81a..24eabcb 100644 --- a/test/store.test.ts +++ b/test/store.test.ts @@ -101,3 +101,36 @@ describe("JournalStore registry", () => { expect(store.stateFor(treeId).labels["msg_1"]?.label).toBe("x") }) }) + +describe("system prompt snapshot", () => { + const store = () => new JournalStore({ worktree: fs.mkdtempSync(path.join(os.tmpdir(), "ctree-sys-")) }) + + test("round-trips, and is overwritten rather than appended", () => { + const s = store() + s.writeSystem("ses_a", { v: 1, ts: 1, parts: [{ name: "base prompt", chars: 3, text: "abc" }] }) + s.writeSystem("ses_a", { v: 1, ts: 2, parts: [{ name: "AGENTS.md", chars: 2, text: "hi" }] }) + // the prompt changes between requests and has no history: the file must not grow + expect(s.readSystem("ses_a")).toEqual({ v: 1, ts: 2, parts: [{ name: "AGENTS.md", chars: 2, text: "hi" }] }) + }) + + test("a session we have never seen a request for reads as undefined, not empty", () => { + expect(store().readSystem("ses_never")).toBeUndefined() + }) + + test("a corrupt or foreign file reads as undefined instead of throwing", () => { + const s = store() + s.writeSystem("ses_b", { v: 1, ts: 1, parts: [] }) + fs.writeFileSync(path.join(s.dir, "system-ses_b.json"), "{not json") + expect(s.readSystem("ses_b")).toBeUndefined() + fs.writeFileSync(path.join(s.dir, "system-ses_b.json"), JSON.stringify({ v: 2, ts: 1, parts: [] })) + expect(s.readSystem("ses_b")).toBeUndefined() + }) + + test("snapshots are per session and do not collide", () => { + const s = store() + s.writeSystem("ses_a", { v: 1, ts: 1, parts: [{ name: "base prompt", chars: 1, text: "a" }] }) + s.writeSystem("ses_b", { v: 1, ts: 1, parts: [{ name: "base prompt", chars: 1, text: "b" }] }) + expect(s.readSystem("ses_a")!.parts[0]!.text).toBe("a") + expect(s.readSystem("ses_b")!.parts[0]!.text).toBe("b") + }) +}) From 8adb02757f7c142146f835c72607950c31ab505a Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 09:43:53 +0000 Subject: [PATCH 3/8] Capture the system prompt on plain sessions too, and prove it end to end MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The capture was gated on the session already being in a tree, but a session is only registered by a branch, a fork or adoption — so a plain session never captured its prompt and the consumers bucket silently never appeared. That is the common case, not the edge one. The capture is now unconditional; only the "Context notes:" push keeps the tree guard, since that note explains ◆/✂ markers that exist only in managed sessions. Since this hook runs on every request, an in-memory per-session shape check skips the write when nothing changed. Adds the e2e that settles the assumption the feature rests on — that output.system arrives carrying OpenCode's own parts rather than empty. It drives the real TUI on a session that never branches and asserts the snapshot exists with real content, that our own note is NOT in it (the capture runs before the push), and that "≡ system prompt" reaches the consumers view. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BrwAnTfTmJSmFwVn3DDFxr --- src/server/index.ts | 22 +++++++++++++++++----- test/e2e/tui.test.ts | 43 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 5 deletions(-) diff --git a/src/server/index.ts b/src/server/index.ts index 75043c7..224e29e 100644 --- a/src/server/index.ts +++ b/src/server/index.ts @@ -47,14 +47,20 @@ function nameSystemPart(text: string, index: number): string { * line below is how to confirm that on a live server), nothing is written and every reader * treats the absence as "unknown", never as "zero". */ +/** Last shape written per session, so an unchanged prompt costs no filesystem write at all — + * this hook runs on every single request. */ +const lastSystem = new Map() + function captureSystem(store: JournalStore, sessionID: string, system: readonly string[]): void { try { const parts = system .map((text, i) => ({ name: nameSystemPart(text, i), chars: text.length, text })) .filter((p) => p.chars > 0) - debug("system.captured", { sessionID, parts: parts.length, chars: parts.reduce((n, p) => n + p.chars, 0) }) - if (parts.length === 0) return + const shape = parts.map((p) => `${p.name}:${p.chars}`).join("|") + debug("system.captured", { sessionID, parts: parts.length, chars: parts.reduce((n, p) => n + p.chars, 0), unchanged: lastSystem.get(sessionID) === shape }) + if (parts.length === 0 || lastSystem.get(sessionID) === shape) return store.writeSystem(sessionID, { v: 1, ts: Date.now(), parts }) + lastSystem.set(sessionID, shape) } catch (e) { debug("system.capture.failed", { error: e instanceof Error ? e.message : String(e) }) } @@ -296,11 +302,17 @@ export const server: Plugin = async ({ worktree, client, directory }, options) = // this is the one place the plugin can see what the provider is really sent as its system // prompt, so it is also where the consumers view gets the bucket it was missing. "experimental.chat.system.transform": async ({ sessionID }, output) => { - if (!sessionID || !(await journal).stateForSession(sessionID)) return - // snapshot what OpenCode assembled, BEFORE our own note joins it — the note is ours, and + if (!sessionID) return + // Snapshot what OpenCode assembled, BEFORE our own note joins it — the note is ours, and // counting it as part of the user's prompt would be a small lie in the one view that - // exists to say where the context went + // exists to say where the context went. + // + // Deliberately NOT gated on the session being in a tree, unlike the note below: a session + // is only registered by a branch/fork/adoption, so gating here would mean a plain session + // never captured its prompt and the consumers bucket silently never appeared — which is + // the common case, not the edge one. captureSystem(await journal, sessionID, output.system) + if (!(await journal).stateForSession(sessionID)) return output.system.push( "Context notes: messages starting with ◆ are decision records confirmed by the user — treat them as settled facts. Tool results reading [cropped: …] or turns reading [dropped turn …] were removed from your context on purpose to save space; if you need one back, ask the user to restore it (they can with /undo in the context tree).", ) diff --git a/test/e2e/tui.test.ts b/test/e2e/tui.test.ts index cbc0da4..0ed2b7a 100644 --- a/test/e2e/tui.test.ts +++ b/test/e2e/tui.test.ts @@ -231,6 +231,49 @@ describe.skipIf(!e2e)("tui e2e: built plugin", () => { } }, 320_000) + test("the server captures the real system prompt; consumers shows it as a bucket", async () => { + const m = await startMock({ tool: false }) + const proj = await createProject({ mockPort: m.port }) + await installPlugins({ projectDir: proj.dir, server: [path.join(REPO_ROOT, "dist", "server.js")], tui: [path.join(REPO_ROOT, "dist", "tui.js")] }) + try { + const text = await runTui({ + projectDir: proj.dir, + keys: [ + // a plain session that never branches: the case the capture must not miss + ["Ask anything", 1, "hello\r"], + ["mock reply", 8, "/tree"], + ["Context tree", 0.5, "\r"], + ["Context tree ·", 2, "s"], + ["what is filling|system prompt|consumers", 3, "\x03"], + ["", 1, "\x03"], + ], + timeoutSec: 180, + cols: 130, + rows: 34, + exitWhenDone: true, + }) + + // 1. the assumption the whole feature rests on: `output.system` arrives carrying + // OpenCode's own parts, so there is something real to snapshot + const dir = path.join(proj.dir, ".opencode", "context-tree") + const snap = readdirSync(dir).find((f) => f.startsWith("system-") && f.endsWith(".json")) + expect(snap).toBeDefined() + const parsed = JSON.parse(readFileSync(path.join(dir, snap!), "utf8")) as { v: number; parts: { name: string; chars: number; text: string }[] } + expect(parsed.v).toBe(1) + expect(parsed.parts.length).toBeGreaterThan(0) + expect(parsed.parts.reduce((n, p) => n + p.chars, 0)).toBeGreaterThan(200) + + // 2. our own note is NOT in the snapshot: it is captured before we push it + expect(parsed.parts.some((p) => p.text.startsWith("Context notes:"))).toBe(false) + + // 3. and it reaches the consumers view + expect(text).toContain("≡ system prompt") + } finally { + await m.stop() + await proj.cleanup() + } + }, 300_000) + test("/tree opens the context tree route with rows and a context header", async () => { const text = await runTui({ projectDir: project.dir, From 8e28af4818aaa945b69847b657c9911b3881ae44 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 09:47:48 +0000 Subject: [PATCH 4/8] e2e: say why the system snapshot is missing instead of ENOENT MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first run of this test failed with a bare scandir ENOENT on the context-tree directory, which says only "it did not happen" — not whether the hook never fired or fired with an empty output.system. The test now runs the TUI with CTREE_DEBUG and, when the directory is absent, fails with the capture's own log lines. That distinguishes the two causes on the spot: a "system.captured" line with parts: 0 means the array arrives empty, and no line at all means the hook does not fire in this OpenCode version. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BrwAnTfTmJSmFwVn3DDFxr --- test/e2e/tui.test.ts | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/test/e2e/tui.test.ts b/test/e2e/tui.test.ts index 0ed2b7a..e9a2f1e 100644 --- a/test/e2e/tui.test.ts +++ b/test/e2e/tui.test.ts @@ -4,7 +4,7 @@ */ import { afterAll, beforeAll, describe, expect, test } from "bun:test" import path from "node:path" -import { readFileSync, readdirSync } from "node:fs" +import { existsSync, readFileSync, readdirSync } from "node:fs" import { createProject, installPlugins, REPO_ROOT, runTui, runTuiScreens, startMock, type StartedMock } from "./harness.js" const e2e = process.env["CTREE_E2E"] === "1" @@ -236,8 +236,10 @@ describe.skipIf(!e2e)("tui e2e: built plugin", () => { const proj = await createProject({ mockPort: m.port }) await installPlugins({ projectDir: proj.dir, server: [path.join(REPO_ROOT, "dist", "server.js")], tui: [path.join(REPO_ROOT, "dist", "tui.js")] }) try { + const log = path.join(proj.dir, "ctree-debug.log") const text = await runTui({ projectDir: proj.dir, + env: { CTREE_DEBUG: log }, keys: [ // a plain session that never branches: the case the capture must not miss ["Ask anything", 1, "hello\r"], @@ -255,7 +257,9 @@ describe.skipIf(!e2e)("tui e2e: built plugin", () => { // 1. the assumption the whole feature rests on: `output.system` arrives carrying // OpenCode's own parts, so there is something real to snapshot + const debugLog = existsSync(log) ? readFileSync(log, "utf8") : "(no debug log)" const dir = path.join(proj.dir, ".opencode", "context-tree") + if (!existsSync(dir)) throw new Error(`no context-tree dir. debug log:\n${debugLog.split("\n").filter((l) => l.includes("system")).join("\n") || debugLog.slice(0, 2000)}`) const snap = readdirSync(dir).find((f) => f.startsWith("system-") && f.endsWith(".json")) expect(snap).toBeDefined() const parsed = JSON.parse(readFileSync(path.join(dir, snap!), "utf8")) as { v: number; parts: { name: string; chars: number; text: string }[] } From 12feb2a51bcb2d4c20934522f93a6adac47099fb Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 09:51:21 +0000 Subject: [PATCH 5/8] debug: log when the server half loads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Did the server plugin load at all?" is the first question in any bug report about it, and there was no way to answer it — every other debug line sits behind a hook that only fires for sessions already in a tree, so an empty log meant nothing. No-op unless CTREE_DEBUG is set. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BrwAnTfTmJSmFwVn3DDFxr --- src/server/index.ts | 1 + 1 file changed, 1 insertion(+) diff --git a/src/server/index.ts b/src/server/index.ts index 224e29e..dcd5e44 100644 --- a/src/server/index.ts +++ b/src/server/index.ts @@ -67,6 +67,7 @@ function captureSystem(store: JournalStore, sessionID: string, system: readonly } export const server: Plugin = async ({ worktree, client, directory }, options) => { + debug("server.loaded", { worktree, directory }) // same option parsing as the TUI half, so both write to the same place (docs/USAGE.md) const mode: StorageMode = options?.["storage"] === "global" ? "global" : "local" // awaiting an SDK call in the plugin factory deadlocks the server (plugin init blocks From 732a579b79d561216063847a4d9bc4d6e0b28b3a Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 10:25:45 +0000 Subject: [PATCH 6/8] HANDOVER: what is verified, what is not, and the one command that settles it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The system-prompt capture could not be verified in the web container: its opencode never runs the plugin's server half at all, which was established by reproducing the crop e2e failure on clean main in a separate worktree. This file records that finding, marks 8adb027 as do-not-merge until one command is run, and reads out the three possible outcomes — including that a bad result would mean the plugin's existing "Context notes:" system push has never been delivered. Delete before merging. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BrwAnTfTmJSmFwVn3DDFxr --- HANDOVER.md | 160 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 HANDOVER.md diff --git a/HANDOVER.md b/HANDOVER.md new file mode 100644 index 0000000..9daa76a --- /dev/null +++ b/HANDOVER.md @@ -0,0 +1,160 @@ +# Handover: `claude/pi-context-tree-workflow-sjuutd` + +You are taking over from a Claude Code **web** session. Its container cannot run the +plugin's server half at all (see §3), so one of the five commits on this branch is +**unverified** and must not be merged until you run one command (§4). + +Delete this file before merging. + +--- + +## 1. State + +- Repo `navbytes/opencode-tree`, branch **`claude/pi-context-tree-workflow-sjuutd`**, + 5 commits on top of `main` (`2266966`). All pushed. No PR opened yet. +- `v0.2.3` is released. The changelog has an `## Unreleased` section for this work. +- Locally green: `bun run typecheck`, `bun test` (**289 pass, 0 fail**), `bun run build`. + +``` +12feb2a debug: log when the server half loads +8e28af4 e2e: say why the system snapshot is missing instead of ENOENT +8adb027 Capture the system prompt on plain sessions too, and prove it end to end +25a78d3 Count the system prompt in the consumers view +9abd80c Lanes fill the terminal, like the rows do +``` + +## 2. What the commits do, and how far each is trusted + +### `9abd80c` — lanes fill the terminal. **Verified. Merge with confidence.** + +The three lane rows stopped short of the right edge for two reasons in one expression: + +```ts +const laneWidth = () => Math.max(10, Math.min(width() - 61, 80)) +``` + +A hard **cap of 80** cells (71 blank columns at 200 cols, 111 at 240), and a **reserve of +61** for chrome that prints 49 — stale, because it was sized for a mode legend that still +said `· 3 calls`. + +Now `width() + 2 - LANE_CHROME`. The `+ 2` matters: a row draws its `│ ` prefix *outside* +its padded width while a lane label carries its own inside its 12, so without it the lanes +land two columns short of the rows. `LANE_CHROME` is now **measured** from the label and +legend strings in `core/lanes.ts`, not written down — that's the fix for the bug class, and +4 tests pin the invariants it needs (legend same width in both modes, label fixed width for +every lane name with and without a cue, etc.). + +### `25a78d3` — `≡ system prompt` bucket in consumers. **Pure layer verified; the wiring is not.** + +Consumers walked the transcript only, so its total could never reconcile with the `ctx …` +gauge two lines above it, which reads `tokens.input` and *does* include the system prompt — +a silently missing 5–15k on an agent with a big base prompt and an `AGENTS.md`. + +8 tests cover `consumers()` directly: bucket accounting, one entry per part sorted +biggest-first, none croppable, shares still summing to 1, and absent-vs-empty (no snapshot +shows **no bucket**, never a misleading `0`). Store round-trip/overwrite/corrupt-file/ +per-session isolation are covered too. + +What is **not** covered: whether the snapshot the TUI reads ever gets written. That is §4. + +### `8adb027` — capture on plain sessions + the e2e. **UNVERIFIED. Do not merge yet.** + +Two things: + +1. A **real bug fix**, sound regardless of §4: the capture was gated on + `stateForSession(sessionID)`, but a session is only registered by a branch/fork/adoption + — so on a plain session that never branches (the common case) the capture would never + have fired and the bucket would silently never appear. The capture is now ungated; the + `Context notes:` push below it keeps its gate, because that note only makes sense for + sessions the plugin manages. +2. The e2e that is supposed to prove the whole thing, which could not run here. + +### `8e28af4`, `12feb2a` — diagnostics. Keep. + +The e2e now fails with the capture's own debug lines instead of a bare `ENOENT`, and the +server plugin logs `server.loaded`. These are what §4 reads. + +## 3. Why the web session could not verify it — read this before you debug anything + +**The plugin's server half does not run in that container.** Established, not assumed: + +- `opencode serve` never binds there. A bare `opencode serve` in an empty directory with + **no plugin installed** was started and killed at timeout having printed nothing. +- The `test/e2e/server.test.ts` suite fails 5/9 with `ConnectionRefused`, in isolation. +- Decisively: the **crop** TUI e2e (`crop in the tree hides a tool result…`), which requires + `experimental.chat.messages.transform` to run, **fails identically on clean `main`** at + the same assertion (`tui.test.ts:71`, `[cropped: bash` never reaching the provider). It + was run in a separate worktree at `2266966` with none of this branch's changes. + +So the pty-driven TUI e2e boots a real OpenCode whose **server-side plugin never executes**. +Four diagnostic runs were spent before this was pinned down; they measured nothing. + +**Corollary:** if the crop e2e also fails on your machine, stop and fix the harness first — +nothing server-side can be verified until it passes, and this branch is not the cause. + +## 4. The one thing to run + +```sh +bun install +CTREE_E2E=1 CTREE_DEBUG=/tmp/ctree.log bun test --timeout 400000 \ + -t "captures the real system prompt" test/e2e/tui.test.ts +``` + +First run downloads `opencode-ai@1.18.26` into `harness/` (~3 min, and it counts against +the test's own timeout — a first run may time out; just run it again). + +The test drives a real TUI on a **plain session that never branches**, then asserts: + +1. `.opencode/context-tree/system-.json` exists with ≥1 part and >200 chars, +2. our own `Context notes:` is **not** in the snapshot (captured before we push it), +3. `≡ system prompt` appears on screen after pressing `s` in `/tree`. + +### Reading the outcome from `/tmp/ctree.log` + +| What you see | Meaning | Do | +|---|---|---| +| `system.captured` with `parts: N>0`, test green | The assumption holds | Merge all 5. Open a PR; delete this file. | +| `system.captured` with `parts: 0` | `output.system` arrives **empty** | See §5 — this is a shipped bug, and `25a78d3`/`8adb027` need rethinking | +| No `system.captured`, but `server.loaded` present | The hook never fires on this path | Same as above | +| Neither line | The server half isn't loading on your machine either | Harness problem — run the crop test as a control | + +## 5. The pre-existing bug this may expose + +The whole feature rests on `experimental.chat.system.transform` handing us +`output: { system: string[] }` **already carrying OpenCode's own parts**, so appending adds +our note. The hook name is correct — it is present in the 1.18.26 binary (verified by +`grep`ing it). But: + +- **Nothing has ever tested that this hook fires.** `test/e2e/server.test.ts:52` asserts the + provider receives *a* system message; nothing asserts our `Context notes:` string is in it. +- If §4 shows `parts: 0` or no call at all, then the plugin's `Context notes:` — which tells + the model how to read `◆` records and `[cropped: …]` stubs — **has never been delivered**, + and `applyCrops` has been shipping stubs the model was never told how to interpret. + +That would be a bigger bug than the feature that surfaced it. Worth a focused test either +way: assert `Context notes:` reaches the mock provider's system message on a session that +*has* branched (the note is gated on tree membership, so a plain session legitimately won't +have it — don't be fooled by that). + +## 6. Known gaps, deliberately left + +- **Tool-definition schemas are still uncounted.** `client.tool.list` gives descriptions but + not what the provider actually receives, so the estimate would be rough enough to mislead. + That is the last unattributed chunk between consumers' total and the `ctx` gauge. +- **Snapshot files grow with sessions** — one small JSON per session that ever made a + request, in the gitignored plugin dir, overwritten in place and skipped entirely when the + prompt shape is unchanged (in-memory hash per session). Bounded per session, unbounded in + session count. Prune if it ever matters. +- **Part naming is a shallow heuristic** (`AGENTS.md`, `CLAUDE.md`, `environment`, `date`, + else `base prompt`) over the first 400 chars. It never throws; a miss just reads + `system prompt N`. +- **`y` is the only way to read a system part.** It is not a message, so it has no row and + no inspector. `y` in the consumers panel copies the selected part in full. + +## 7. If you need to change direction + +If §4 comes back bad, the fallback that needs no hook: derive the system prompt's *size* +(not its text) by subtracting everything else from an assistant message's `tokens.input`. +That gives one number for a `≡ system + tools` bucket with no per-part breakdown and no +copyable text — much weaker, but it cannot be wrong about the total, and it would also cover +the tool definitions from §6. From a0bea6700fb50a881e999ce0a866e4262e54b2af Mon Sep 17 00:00:00 2001 From: Naveen Kumar Date: Fri, 4 Sep 2026 19:22:17 +0800 Subject: [PATCH 7/8] Review fixes: y copied the wrong system part, and snapshots went stale MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three real defects found reviewing the branch before opening it. `y` handed you the wrong text. `nameSystemPart` is a substring heuristic, so a global and a project `AGENTS.md` both come back "AGENTS.md" — and the copy path looked its part up *by name*, in original order, while the entries are shown sorted biggest-first. Selecting the 4k part copied the 12k part, silently. Entries now carry the index of the part they came from. The "has this prompt changed?" key was `name:chars`, so a change of equal length never rewrote the file: the `` part's date rolling 2026-09-04 → 2026-09-05 is the same length and the same name, and the snapshot would serve yesterday's text for the life of the server process. It hashes the content now. `system-${sessionID}.json` put a host-supplied id straight into a path we write, where a `/` or `..` escapes the plugin dir. Refused rather than sanitised. Also: the snapshot read was an eager memo doing a readFileSync — a thrown and caught ENOENT, in the common no-snapshot case — on every poll tick whether or not the consumers panel was ever opened; it is gated on the panel now. Dropped the pretty-printing that doubled every snapshot on disk. Four consumers tests were indented as if inside their `describe` but sat outside it. And the e2e that proves the capture end to end now asserts against the pyte-rendered screen: the raw pty stream drops cells from an incrementally repainted panel, so it read `≡ yem prompt` and failed a working feature. --- src/core/consumers.ts | 7 +++++-- src/server/index.ts | 6 +++++- src/shared/store.ts | 5 ++++- src/tui/route.tsx | 7 +++++-- test/e2e/server.test.ts | 30 ++++++++++++++++++++++++++++++ test/e2e/tui.test.ts | 11 +++++++---- test/lanes.test.ts | 14 +++++++++++++- 7 files changed, 69 insertions(+), 11 deletions(-) diff --git a/src/core/consumers.ts b/src/core/consumers.ts index 8eb9fe3..930cf48 100644 --- a/src/core/consumers.ts +++ b/src/core/consumers.ts @@ -13,6 +13,9 @@ export type ConsumerEntry = { preview: string /** only a completed tool result can be stubbed by crop's result mode (core/crop.ts) */ croppable: boolean + /** system entries only: which captured part this is. Names are a heuristic and two parts can + * share one (a global and a project `AGENTS.md`), so `y` must not find its text by name. */ + systemIndex?: number } export type Consumer = { @@ -77,12 +80,12 @@ export function consumers( } } // one bucket, one entry per part, so "AGENTS.md is 4k" is visible next to "bash is 30k" - for (const part of opts.system ?? []) { + for (const [i, part] of (opts.system ?? []).entries()) { const c = acc.get(SYSTEM) ?? { source: SYSTEM, kind: "system" as const, tokens: 0, count: 0, share: 0, entries: [] } const tokens = estimateTokens(part.text) c.tokens += tokens c.count += 1 - c.entries.push({ messageID: "", tokens, preview: `${part.name}: ${part.text.slice(0, 120).replace(/\s+/g, " ").trim()}`, croppable: false }) + c.entries.push({ messageID: "", tokens, preview: `${part.name}: ${part.text.slice(0, 120).replace(/\s+/g, " ").trim()}`, croppable: false, systemIndex: i }) acc.set(SYSTEM, c) } diff --git a/src/server/index.ts b/src/server/index.ts index dcd5e44..f9fa3fa 100644 --- a/src/server/index.ts +++ b/src/server/index.ts @@ -6,6 +6,7 @@ * messages OpenCode sends to the model, in place. Tree-shaping entries it does * not create itself (squash merges, labels, summaries) come from the TUI half. */ +import crypto from "node:crypto" import fs from "node:fs" import path from "node:path" import type { Plugin } from "@opencode-ai/plugin" @@ -56,7 +57,10 @@ function captureSystem(store: JournalStore, sessionID: string, system: readonly const parts = system .map((text, i) => ({ name: nameSystemPart(text, i), chars: text.length, text })) .filter((p) => p.chars > 0) - const shape = parts.map((p) => `${p.name}:${p.chars}`).join("|") + // hash the text, not just its length: the `` part's date rolls 2026-09-04 → 2026-09-05 + // without changing a single character count, and a length-only key would serve that stale + // text from disk for the life of the process. + const shape = crypto.createHash("sha1").update(parts.map((p) => `${p.name}\n${p.text}`).join("\n\n")).digest("hex") debug("system.captured", { sessionID, parts: parts.length, chars: parts.reduce((n, p) => n + p.chars, 0), unchanged: lastSystem.get(sessionID) === shape }) if (parts.length === 0 || lastSystem.get(sessionID) === shape) return store.writeSystem(sessionID, { v: 1, ts: Date.now(), parts }) diff --git a/src/shared/store.ts b/src/shared/store.ts index 9b21cce..37d04d7 100644 --- a/src/shared/store.ts +++ b/src/shared/store.ts @@ -60,6 +60,9 @@ export class JournalStore { * tree, not a mutation with history, so it must not grow the way the journal does. */ private systemPath(sessionID: string): string { + // the id reaches us from the host and lands in a path we write: a `/` or `..` in it would + // escape baseDir, so anything but an id-shaped string is refused rather than sanitised + if (!/^[A-Za-z0-9_-]+$/.test(sessionID)) throw new Error(`refusing to use unsafe sessionID in a path: ${JSON.stringify(sessionID)}`) return path.join(this.baseDir, `system-${sessionID}.json`) } @@ -69,7 +72,7 @@ export class JournalStore { writeSystem(sessionID: string, snapshot: SystemSnapshot): void { this.ensureDir() const tmp = `${this.systemPath(sessionID)}.${process.pid}.${Date.now()}.tmp` - fs.writeFileSync(tmp, `${JSON.stringify(snapshot, null, 2)}\n`) + fs.writeFileSync(tmp, `${JSON.stringify(snapshot)}\n`) fs.renameSync(tmp, this.systemPath(sessionID)) } diff --git a/src/tui/route.tsx b/src/tui/route.tsx index d7aab5b..3bd79a9 100644 --- a/src/tui/route.tsx +++ b/src/tui/route.tsx @@ -801,7 +801,9 @@ export function TreeRoute(props: TreeRouteProps) { * bucket simply does not appear rather than reporting a misleading zero. */ const systemParts = createMemo(() => { tick() // the server half rewrites the snapshot on each request; follow the same poll the tree does - if (!sessionID) return undefined + // only the consumers panel reads this, and the memo is eager: without the gate every poll + // tick pays a readFileSync — a thrown-and-caught ENOENT, in the common no-snapshot case + if (!sessionID || panel() !== "consumers") return undefined return store.readSystem(sessionID)?.parts }) const consumerRows = createMemo(() => (live() ? consumers(live()!, { cropped: alreadyCropped(), limit: contextLimit(), system: systemParts() }) : [])) @@ -862,7 +864,8 @@ export function TreeRoute(props: TreeRouteProps) { // part in full, since it is not a message and has no row of its own if (panel() === "consumers") { const line = consumerLine() - const part = line?.entry && line.bucket.kind === "system" ? systemParts()?.find((p) => line.entry!.preview.startsWith(`${p.name}:`)) : undefined + const idx = line?.bucket.kind === "system" ? line.entry?.systemIndex : undefined + const part = idx === undefined ? undefined : systemParts()?.[idx] const text = part?.text ?? line?.entry?.preview ?? "" if (!text) return try { diff --git a/test/e2e/server.test.ts b/test/e2e/server.test.ts index 76bf745..1467750 100644 --- a/test/e2e/server.test.ts +++ b/test/e2e/server.test.ts @@ -24,6 +24,15 @@ function unwrap(res: { data?: T; error?: unknown }): T { return res.data } +/** Every system message the provider was sent, joined — `output.system` parts may arrive as + * separate messages or as one, and structured content is stringified rather than assumed. */ +function systemText(req: { body: { messages: { role: string; content: unknown }[] } }): string { + return req.body.messages + .filter((m) => m.role === "system") + .map((m) => (typeof m.content === "string" ? m.content : JSON.stringify(m.content))) + .join("\n") +} + async function installSpikePlugin(projectDir: string): Promise { const pluginsDir = path.join(projectDir, ".opencode", "plugins") await mkdir(pluginsDir, { recursive: true }) @@ -312,6 +321,27 @@ describe.skipIf(!e2e)("server e2e: built plugin headless /ctree commands", () => expect(readFileSync(path.join(server.dir, "ctree-decisions.md"), "utf8")).toContain("# Decisions") }, 180_000) + // DESIGN.md §6.8. `applyCrops` ships `[cropped: …]` stubs and `◆` records to the model; this + // note is the only thing that tells it how to read them, and nothing proved the note survived + // `experimental.chat.system.transform` into the actual provider request. It is gated on tree + // membership, so a plain session legitimately has none — branch first, then look. + test("the ◆/crop system note reaches the provider once the session is in a tree", async () => { + const session = unwrap(await server.client.session.create({ query: { directory: server.dir }, body: { title: "system-note" } })) + + mock.clearRequests() + unwrap(await server.client.session.prompt({ path: { id: session.id }, query: { directory: server.dir }, body: { parts: [{ type: "text", text: "seed turn" }] } })) + expect(systemText(mock.requests().at(-1)!)).not.toContain("Context notes:") + + unwrap(await server.client.session.command({ path: { id: session.id }, query: { directory: server.dir }, body: { command: "ctree", arguments: "branch note-check mock/mock-b" } })) + + mock.clearRequests() + unwrap(await server.client.session.prompt({ path: { id: session.id }, query: { directory: server.dir }, body: { parts: [{ type: "text", text: "second turn" }] } })) + const sys = systemText(mock.requests().at(-1)!) + expect(sys).toContain("Context notes:") + // and it is appended to OpenCode's own prompt, not sent instead of it + expect(sys.length).toBeGreaterThan(500) + }, 180_000) + test("a native session.fork (no plugin command) is adopted into the tree", async () => { const session = unwrap(await server.client.session.create({ query: { directory: server.dir }, body: { title: "native-fork" } })) unwrap(await server.client.session.prompt({ path: { id: session.id }, query: { directory: server.dir }, body: { parts: [{ type: "text", text: "seed turn" }] } })) diff --git a/test/e2e/tui.test.ts b/test/e2e/tui.test.ts index e9a2f1e..c0824d7 100644 --- a/test/e2e/tui.test.ts +++ b/test/e2e/tui.test.ts @@ -237,7 +237,7 @@ describe.skipIf(!e2e)("tui e2e: built plugin", () => { await installPlugins({ projectDir: proj.dir, server: [path.join(REPO_ROOT, "dist", "server.js")], tui: [path.join(REPO_ROOT, "dist", "tui.js")] }) try { const log = path.join(proj.dir, "ctree-debug.log") - const text = await runTui({ + const { screens } = await runTuiScreens({ projectDir: proj.dir, env: { CTREE_DEBUG: log }, keys: [ @@ -246,7 +246,7 @@ describe.skipIf(!e2e)("tui e2e: built plugin", () => { ["mock reply", 8, "/tree"], ["Context tree", 0.5, "\r"], ["Context tree ·", 2, "s"], - ["what is filling|system prompt|consumers", 3, "\x03"], + ["what's filling|consumers", 3, "\x03"], ["", 1, "\x03"], ], timeoutSec: 180, @@ -270,8 +270,11 @@ describe.skipIf(!e2e)("tui e2e: built plugin", () => { // 2. our own note is NOT in the snapshot: it is captured before we push it expect(parsed.parts.some((p) => p.text.startsWith("Context notes:"))).toBe(false) - // 3. and it reaches the consumers view - expect(text).toContain("≡ system prompt") + // 3. and it reaches the consumers view. Assert against the pyte-rendered screen, not + // the raw stream: the panel is painted cell-by-cell, so no label lands there whole. + const consumers = screens.find((s) => s.screen.includes("what's filling the context")) + if (!consumers) throw new Error(`consumers panel never rendered. screens: ${screens.map((s) => s.label).join(" | ")}`) + expect(consumers.screen).toContain("≡ system prompt") } finally { await m.stop() await proj.cleanup() diff --git a/test/lanes.test.ts b/test/lanes.test.ts index 2580370..fdf67c9 100644 --- a/test/lanes.test.ts +++ b/test/lanes.test.ts @@ -26,7 +26,6 @@ describe("consumers", () => { expect(c[0]!.tokens).toBeLessThan(consumers(open)[0]!.tokens) expect(bar(0.5, 10)).toBe("▰▰▰▰▰▱▱▱▱▱") }) -}) test("the system prompt is a bucket, so the view reconciles with the ctx gauge", () => { // without it this view walks the transcript only, and silently omits a chunk the header's @@ -65,6 +64,19 @@ describe("consumers", () => { expect(cs.reduce((n, c) => n + c.share, 0)).toBeCloseTo(1, 5) }) + // names are a heuristic, so two parts can share one; `y` must still copy the part you picked + test("each system entry points at its own part, even when two share a name", () => { + const system = [ + { name: "AGENTS.md", text: "global ".repeat(600) }, + { name: "AGENTS.md", text: "project ".repeat(200) }, + ] + const entries = consumers(open, { system }).find((c) => c.kind === "system")!.entries + expect(entries.map((e) => e.systemIndex)).toEqual([0, 1]) + // sorted biggest-first, so the index must not be positional + expect(system[entries[0]!.systemIndex!]!.text.startsWith("global")).toBe(true) + }) +}) + const T = (messages: TranscriptMessage[]): Transcript => ({ sessionID: "s", title: "strip", status: "available", messages }) /** cells that belong to any lane, per index — the axis is shared, so this must never exceed 1 */ From 78fee46c2f52debbb0ea4d8ec335653c193dba8a Mon Sep 17 00:00:00 2001 From: Naveen Kumar Date: Fri, 4 Sep 2026 19:22:21 +0800 Subject: [PATCH 8/8] Remove the handover: its open questions are answered and its commits are verified --- HANDOVER.md | 160 ---------------------------------------------------- 1 file changed, 160 deletions(-) delete mode 100644 HANDOVER.md diff --git a/HANDOVER.md b/HANDOVER.md deleted file mode 100644 index 9daa76a..0000000 --- a/HANDOVER.md +++ /dev/null @@ -1,160 +0,0 @@ -# Handover: `claude/pi-context-tree-workflow-sjuutd` - -You are taking over from a Claude Code **web** session. Its container cannot run the -plugin's server half at all (see §3), so one of the five commits on this branch is -**unverified** and must not be merged until you run one command (§4). - -Delete this file before merging. - ---- - -## 1. State - -- Repo `navbytes/opencode-tree`, branch **`claude/pi-context-tree-workflow-sjuutd`**, - 5 commits on top of `main` (`2266966`). All pushed. No PR opened yet. -- `v0.2.3` is released. The changelog has an `## Unreleased` section for this work. -- Locally green: `bun run typecheck`, `bun test` (**289 pass, 0 fail**), `bun run build`. - -``` -12feb2a debug: log when the server half loads -8e28af4 e2e: say why the system snapshot is missing instead of ENOENT -8adb027 Capture the system prompt on plain sessions too, and prove it end to end -25a78d3 Count the system prompt in the consumers view -9abd80c Lanes fill the terminal, like the rows do -``` - -## 2. What the commits do, and how far each is trusted - -### `9abd80c` — lanes fill the terminal. **Verified. Merge with confidence.** - -The three lane rows stopped short of the right edge for two reasons in one expression: - -```ts -const laneWidth = () => Math.max(10, Math.min(width() - 61, 80)) -``` - -A hard **cap of 80** cells (71 blank columns at 200 cols, 111 at 240), and a **reserve of -61** for chrome that prints 49 — stale, because it was sized for a mode legend that still -said `· 3 calls`. - -Now `width() + 2 - LANE_CHROME`. The `+ 2` matters: a row draws its `│ ` prefix *outside* -its padded width while a lane label carries its own inside its 12, so without it the lanes -land two columns short of the rows. `LANE_CHROME` is now **measured** from the label and -legend strings in `core/lanes.ts`, not written down — that's the fix for the bug class, and -4 tests pin the invariants it needs (legend same width in both modes, label fixed width for -every lane name with and without a cue, etc.). - -### `25a78d3` — `≡ system prompt` bucket in consumers. **Pure layer verified; the wiring is not.** - -Consumers walked the transcript only, so its total could never reconcile with the `ctx …` -gauge two lines above it, which reads `tokens.input` and *does* include the system prompt — -a silently missing 5–15k on an agent with a big base prompt and an `AGENTS.md`. - -8 tests cover `consumers()` directly: bucket accounting, one entry per part sorted -biggest-first, none croppable, shares still summing to 1, and absent-vs-empty (no snapshot -shows **no bucket**, never a misleading `0`). Store round-trip/overwrite/corrupt-file/ -per-session isolation are covered too. - -What is **not** covered: whether the snapshot the TUI reads ever gets written. That is §4. - -### `8adb027` — capture on plain sessions + the e2e. **UNVERIFIED. Do not merge yet.** - -Two things: - -1. A **real bug fix**, sound regardless of §4: the capture was gated on - `stateForSession(sessionID)`, but a session is only registered by a branch/fork/adoption - — so on a plain session that never branches (the common case) the capture would never - have fired and the bucket would silently never appear. The capture is now ungated; the - `Context notes:` push below it keeps its gate, because that note only makes sense for - sessions the plugin manages. -2. The e2e that is supposed to prove the whole thing, which could not run here. - -### `8e28af4`, `12feb2a` — diagnostics. Keep. - -The e2e now fails with the capture's own debug lines instead of a bare `ENOENT`, and the -server plugin logs `server.loaded`. These are what §4 reads. - -## 3. Why the web session could not verify it — read this before you debug anything - -**The plugin's server half does not run in that container.** Established, not assumed: - -- `opencode serve` never binds there. A bare `opencode serve` in an empty directory with - **no plugin installed** was started and killed at timeout having printed nothing. -- The `test/e2e/server.test.ts` suite fails 5/9 with `ConnectionRefused`, in isolation. -- Decisively: the **crop** TUI e2e (`crop in the tree hides a tool result…`), which requires - `experimental.chat.messages.transform` to run, **fails identically on clean `main`** at - the same assertion (`tui.test.ts:71`, `[cropped: bash` never reaching the provider). It - was run in a separate worktree at `2266966` with none of this branch's changes. - -So the pty-driven TUI e2e boots a real OpenCode whose **server-side plugin never executes**. -Four diagnostic runs were spent before this was pinned down; they measured nothing. - -**Corollary:** if the crop e2e also fails on your machine, stop and fix the harness first — -nothing server-side can be verified until it passes, and this branch is not the cause. - -## 4. The one thing to run - -```sh -bun install -CTREE_E2E=1 CTREE_DEBUG=/tmp/ctree.log bun test --timeout 400000 \ - -t "captures the real system prompt" test/e2e/tui.test.ts -``` - -First run downloads `opencode-ai@1.18.26` into `harness/` (~3 min, and it counts against -the test's own timeout — a first run may time out; just run it again). - -The test drives a real TUI on a **plain session that never branches**, then asserts: - -1. `.opencode/context-tree/system-.json` exists with ≥1 part and >200 chars, -2. our own `Context notes:` is **not** in the snapshot (captured before we push it), -3. `≡ system prompt` appears on screen after pressing `s` in `/tree`. - -### Reading the outcome from `/tmp/ctree.log` - -| What you see | Meaning | Do | -|---|---|---| -| `system.captured` with `parts: N>0`, test green | The assumption holds | Merge all 5. Open a PR; delete this file. | -| `system.captured` with `parts: 0` | `output.system` arrives **empty** | See §5 — this is a shipped bug, and `25a78d3`/`8adb027` need rethinking | -| No `system.captured`, but `server.loaded` present | The hook never fires on this path | Same as above | -| Neither line | The server half isn't loading on your machine either | Harness problem — run the crop test as a control | - -## 5. The pre-existing bug this may expose - -The whole feature rests on `experimental.chat.system.transform` handing us -`output: { system: string[] }` **already carrying OpenCode's own parts**, so appending adds -our note. The hook name is correct — it is present in the 1.18.26 binary (verified by -`grep`ing it). But: - -- **Nothing has ever tested that this hook fires.** `test/e2e/server.test.ts:52` asserts the - provider receives *a* system message; nothing asserts our `Context notes:` string is in it. -- If §4 shows `parts: 0` or no call at all, then the plugin's `Context notes:` — which tells - the model how to read `◆` records and `[cropped: …]` stubs — **has never been delivered**, - and `applyCrops` has been shipping stubs the model was never told how to interpret. - -That would be a bigger bug than the feature that surfaced it. Worth a focused test either -way: assert `Context notes:` reaches the mock provider's system message on a session that -*has* branched (the note is gated on tree membership, so a plain session legitimately won't -have it — don't be fooled by that). - -## 6. Known gaps, deliberately left - -- **Tool-definition schemas are still uncounted.** `client.tool.list` gives descriptions but - not what the provider actually receives, so the estimate would be rough enough to mislead. - That is the last unattributed chunk between consumers' total and the `ctx` gauge. -- **Snapshot files grow with sessions** — one small JSON per session that ever made a - request, in the gitignored plugin dir, overwritten in place and skipped entirely when the - prompt shape is unchanged (in-memory hash per session). Bounded per session, unbounded in - session count. Prune if it ever matters. -- **Part naming is a shallow heuristic** (`AGENTS.md`, `CLAUDE.md`, `environment`, `date`, - else `base prompt`) over the first 400 chars. It never throws; a miss just reads - `system prompt N`. -- **`y` is the only way to read a system part.** It is not a message, so it has no row and - no inspector. `y` in the consumers panel copies the selected part in full. - -## 7. If you need to change direction - -If §4 comes back bad, the fallback that needs no hook: derive the system prompt's *size* -(not its text) by subtracting everything else from an assistant message's `tokens.input`. -That gives one number for a `≡ system + tools` bucket with no per-part breakdown and no -copyable text — much weaker, but it cannot be wrong about the total, and it would also cover -the tool definitions from §6.