diff --git a/CHANGELOG.md b/CHANGELOG.md index 33e42d8..754629e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,26 @@ # Changelog +## Unreleased + +- Consumers (`s`) counts the **system prompt**. It walked the transcript only, so its total + could never be reconciled with the `ctx …` gauge two lines above it, which reads + `tokens.input` and does include the system prompt — a silently missing 5–15k on an agent with + a large base prompt and an `AGENTS.md`. The server half now snapshots what the provider is + really sent (in `experimental.chat.system.transform`, before adding its own note) and the view + shows a `≡ system prompt` bucket with one entry per part — base prompt, `AGENTS.md`, + environment — so you can see that your rules file costs 4k. It is not croppable, and `y` + copies a part in full. A session whose prompt the plugin has not seen yet shows no bucket + rather than a misleading zero. + +- The timeline lanes fill the terminal, ending on the same column as the rows and the status + line. They were capped at 80 cells and reserved a hardcoded 61 columns for the label and the + mode legend that actually print 49, so a 200-column terminal left ~71 blank columns to the + right of the strip and even a 130-column one left 16. The reserve is now derived from the + strings themselves (`core/lanes.ts#LANE_CHROME`), so shortening the mode legend widens the + strip instead of leaving a gap — which is what went wrong when the "3 calls" mode was + removed and the 61 stayed 61. A wider strip shows more events before scrolling, and the + overview track already hides itself once nothing is off-screen. + ## 0.2.3 — 2026-09-04 - **The inspector shows the whole field now, and pages through it.** It used to cap each field diff --git a/DESIGN.md b/DESIGN.md index f91aef3..31f6995 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -635,6 +635,31 @@ Three changes, in the order they matter: `y` stays the answer for actually *reading* a large payload — copy it somewhere with search and folding. The scroller is for "there were twelve more lines and I want to glance at them". +**The system prompt (0.2.4).** Consumers walked the transcript only, so it omitted the system +prompt and its total could not be reconciled with the `ctx …` gauge above it — which reads +`tokens.input` and therefore *does* include it. On an agent with a large base prompt and an +`AGENTS.md` that is a silently missing 5–15k in the one view whose job is "where did my window +go". + +`experimental.chat.system.transform` is the only place the plugin can see it. The server half +snapshots `output.system` there **before pushing its own note** (counting our note as part of +the user's prompt would be a small lie in exactly the wrong view), names each part by a shallow +text heuristic (`AGENTS.md`, `CLAUDE.md`, `environment`, …, else `base prompt`), and writes it +to `system-.json` — *overwritten* each request, not appended: it is current state +outside the message tree, not a mutation with history, so `/undo` has nothing to do with it and +it must not grow the way the journal does. The TUI reads it on the same poll as the tree. + +It appears as one `≡ system prompt` bucket with an entry per part, `croppable: false` and the +note `sent whole every request · not croppable (y copies a part)` — the same mechanism +`(thinking)` already uses. `y` in the consumers panel copies the selected part in full, which is +the only way to read one: it is not a message, so it has no row and no inspector of its own. + +Two honest limits. It is mostly *diagnostic* — OpenCode's base prompt cannot be cropped, though +`AGENTS.md` being 4k is a lever the user owns. And absence means **unknown**, never zero: a +session whose prompt we have not seen yet simply has no bucket, rather than reporting 0. +Tool-definition schemas are still uncounted; `client.tool.list` gives their descriptions but not +what the provider is really sent, so that estimate would be rough enough to mislead. + ### 7.5 Filters and search (from Pi) `f` cycles `default → no-tools → user-only → labeled → all` (default hides @@ -643,6 +668,16 @@ labeled shows only `L`-labelled rows). `/` filters rows incrementally by role, t name, label, and text — every token must match, like Pi. Folding state resets on filter change (as in Pi) and is otherwise remembered per session in `api.kv`. +**Lane width (0.2.4).** The strip is `width() + 2 - LANE_CHROME`: the terminal, minus the +12-column lane label and the mode legend, plus the two columns a row spends on its `│ ` prefix +*outside* its own width (a lane label carries its own), so all three lane rows end on the same +column as the rows and the status line. `LANE_CHROME` is *measured* from the label and legend +strings in `core/lanes.ts`, never written down — it was a literal `61` against a legend that +printed 49, and it stayed `61` when dropping the "Calls" mode made the legend shorter still. +There is no ceiling on the strip: it is a window onto an unbounded layout, so more cells is +more events visible and less scrolling, and `laneOverview` hides the overview track once +nothing is off-screen. + ### 7.6 Narrow terminals Below 110 columns the inspector is hidden and `i` opens it full-screen; below 80 the diff --git a/docs/USAGE.md b/docs/USAGE.md index 03e7b2f..6c7dd9e 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -111,7 +111,7 @@ appended to the trunk as a normal message.* | `c` `space` `a` `t` `⏎` | crop mode: mark (`space` alone enters it on a croppable row), auto-mark (≥10k tokens, older than 2 turns), result⇄turn, apply | | `u` (`x`) | undo | | `D` `E` | decisions panel, export `ctree-decisions.md` | -| `s` | consumers: what is filling the context (`⏎` opens a bucket, `space` marks one entry for crop) | +| `s` | consumers: what is filling the context (`⏎` opens a bucket, `space` marks one entry for crop, `y` copies one). Includes a `≡ system prompt` bucket broken down by part (base prompt, `AGENTS.md`, …) once the plugin has seen one request for the session — it is not croppable, but it is counted, so the total reconciles with the `ctx …` gauge | | `i` | inspector pane on/off (auto-hidden under 110 columns) | | `i` `I` `PgUp` `PgDn` | inspector in the side pane / full screen; page through a long payload or result. The pane shows every line it has, sized to your terminal, with `12–40 of 118` at the foot when there is more; `y` copies the untruncated text. Below 110 columns the side pane does not fit, so `i` opens full screen directly | | `1 2` `0` | timeline lanes, x-axis by duration / one cell per event; `0` off. `│` marks a turn boundary, and the lanes show whatever the `f` filter shows — so `f` → `tools-only` is the "what did I run" view in both the rows and the lanes | diff --git a/src/core/consumers.ts b/src/core/consumers.ts index c2de33e..930cf48 100644 --- a/src/core/consumers.ts +++ b/src/core/consumers.ts @@ -13,11 +13,14 @@ export type ConsumerEntry = { preview: string /** only a completed tool result can be stubbed by crop's result mode (core/crop.ts) */ croppable: boolean + /** system entries only: which captured part this is. Names are a heuristic and two parts can + * share one (a global and a project `AGENTS.md`), so `y` must not find its text by name. */ + systemIndex?: number } export type Consumer = { source: string - kind: "tool" | "assistant" | "user" | "decision" | "summary" | "reasoning" + kind: "tool" | "assistant" | "user" | "decision" | "summary" | "reasoning" | "system" tokens: number count: number /** share of this transcript's own total */ @@ -32,8 +35,22 @@ export type Consumer = { const THINKING = "(thinking)" const THINKING_NOTE = "provider reasoning · not croppable" +const SYSTEM = "≡ system prompt" +const SYSTEM_NOTE = "sent whole every request · not croppable (y copies a part)" -export function consumers(transcript: Transcript, opts: { cropped?: Set; limit?: number } = {}): Consumer[] { +export function consumers( + transcript: Transcript, + opts: { + cropped?: Set + limit?: number + /** + * The system parts the provider is really sent, captured by the server half. Without it + * this view silently omits them, and its total cannot be reconciled with the `ctx …` gauge + * — which reads `tokens.input` and so *does* include them (DESIGN.md §7.4). + */ + system?: { name: string; text: string }[] + } = {}, +): Consumer[] { const acc = new Map() const add = (source: string, kind: Consumer["kind"], tokens: number, message: TranscriptMessage, part: StepPart) => { const c = acc.get(source) ?? { source, kind, tokens: 0, count: 0, share: 0, entries: [] } @@ -62,6 +79,16 @@ export function consumers(transcript: Transcript, opts: { cropped?: Set; } else if (p.type === "reasoning") add(THINKING, "reasoning", estimateTokens(p.text ?? ""), m, p) } } + // one bucket, one entry per part, so "AGENTS.md is 4k" is visible next to "bash is 30k" + for (const [i, part] of (opts.system ?? []).entries()) { + const c = acc.get(SYSTEM) ?? { source: SYSTEM, kind: "system" as const, tokens: 0, count: 0, share: 0, entries: [] } + const tokens = estimateTokens(part.text) + c.tokens += tokens + c.count += 1 + c.entries.push({ messageID: "", tokens, preview: `${part.name}: ${part.text.slice(0, 120).replace(/\s+/g, " ").trim()}`, croppable: false, systemIndex: i }) + acc.set(SYSTEM, c) + } + const total = [...acc.values()].reduce((s, c) => s + c.tokens, 0) || 1 const limit = opts.limit !== undefined && opts.limit > 0 ? opts.limit : undefined return [...acc.values()] @@ -69,7 +96,7 @@ export function consumers(transcript: Transcript, opts: { cropped?: Set; ...c, share: c.tokens / total, ...(limit === undefined ? {} : { shareOfWindow: c.tokens / limit }), - ...(c.kind === "reasoning" ? { note: THINKING_NOTE } : {}), + ...(c.kind === "reasoning" ? { note: THINKING_NOTE } : c.kind === "system" ? { note: SYSTEM_NOTE } : {}), entries: c.entries.sort((a, b) => b.tokens - a.tokens), })) .sort((a, b) => b.tokens - a.tokens) diff --git a/src/core/journal.ts b/src/core/journal.ts index a1f89ce..737f87b 100644 --- a/src/core/journal.ts +++ b/src/core/journal.ts @@ -131,6 +131,28 @@ export type JournalEntry = z.infer export type JournalEntryType = JournalEntry["type"] /** Parse one JSONL line into a validated journal entry, or `undefined` if it is malformed. */ +/** + * What the provider is actually sent as its system prompt, captured by the server half in + * `experimental.chat.system.transform` (DESIGN.md §7.4). It is *not* a journal entry: it has + * no history, it is not a mutation the user made, and `/undo` has nothing to do with it — the + * store keeps it in its own per-session file, overwritten each request. + */ +export const SystemPart = z.object({ + /** A name for the part, guessed from its own text — providers hand us an unlabelled array. */ + name: z.string(), + chars: z.number(), + text: z.string(), +}) +export type SystemPart = z.infer + +export const SystemSnapshot = z.object({ + v: z.literal(1), + /** When the provider was last sent this; the prompt can change between requests. */ + ts: z.number(), + parts: z.array(SystemPart), +}) +export type SystemSnapshot = z.infer + export function parseJournalLine(line: string): JournalEntry | undefined { const trimmed = line.trim() if (!trimmed) return undefined diff --git a/src/core/lanes.ts b/src/core/lanes.ts index 154a25b..cdc89a8 100644 --- a/src/core/lanes.ts +++ b/src/core/lanes.ts @@ -42,6 +42,30 @@ function spanOf(m: TranscriptMessage): number { * tokens — `tokens` rides along only for the inspector. */ +/** + * Chrome around the strip on a lane row: the fixed-width label on the left and the mode line + * on the right. Both are fixed width *by construction* — `laneCue` caps at three digits, and + * each mode label is the same length selected or not — so the strip can be sized as + * "everything else" without measuring per frame. + * + * They live here, and `LANE_CHROME` is derived from them rather than written down, because the + * width was a literal `61` that stayed `61` when dropping the "3 calls" mode shortened the + * mode line: the strip quietly gave up ~10 columns and stopped short of the right edge. + */ +export const laneLabel = (name: string, cue = "") => `│ ${name} ${cue}`.padEnd(LANE_LABEL_WIDTH) +export const LANE_LABEL_WIDTH = 12 + +export function laneModeLine(mode: LaneMode): string { + return `${mode === "duration" ? "[1] Duration" : " 1 duration"} · ${mode === "turns" ? "[2] Turns" : " 2 turns"} · 0 off` +} + +export function laneSuffix(cue: string, mode: LaneMode): string { + return `${cue.padStart(4).padEnd(5)}${laneModeLine(mode)}` +} + +/** Columns a lane row spends on anything that is not the strip. */ +export const LANE_CHROME = LANE_LABEL_WIDTH + laneSuffix("", "turns").length + export type LaneEvent = { lane: "input" | "model" | "tools" /** `context` = compaction/branch summary: machine-written context, not the human prompting */ diff --git a/src/server/index.ts b/src/server/index.ts index 7b72baa..f9fa3fa 100644 --- a/src/server/index.ts +++ b/src/server/index.ts @@ -6,6 +6,7 @@ * messages OpenCode sends to the model, in place. Tree-shaping entries it does * not create itself (squash merges, labels, summaries) come from the TUI half. */ +import crypto from "node:crypto" import fs from "node:fs" import path from "node:path" import type { Plugin } from "@opencode-ai/plugin" @@ -23,7 +24,54 @@ import { cacheShare, contextSizeOf, formatK, type MinimalMessage as MinimalToken import { parseForkTitle } from "../core/adopt.js" import { adoptNativeForks } from "../shared/adopt.js" +/** + * Name the parts of an unlabelled system prompt so the consumers view can say *which* part + * costs what — "my AGENTS.md is 4k" is a lever the user can actually pull, where OpenCode's + * base prompt is not. + * + * The heuristics are deliberately shallow and never throw: an unrecognised part is "system + * prompt" plus its index, which is still worth counting. + */ +function nameSystemPart(text: string, index: number): string { + const head = text.slice(0, 400).toLowerCase() + if (head.includes("agents.md")) return "AGENTS.md" + if (head.includes("claude.md")) return "CLAUDE.md" + if (/\bcontext notes:/.test(head)) return "context-tree note" + if (head.includes("") || head.includes("working directory")) return "environment" + if (head.includes("today's date") || head.includes("current date")) return "date" + return index === 0 ? "base prompt" : `system prompt ${index + 1}` +} + +/** + * Record the system parts for a session. Best effort in every direction: if the host ever hands + * us an empty array (we append to it, so it arrives carrying OpenCode's own parts — the debug + * line below is how to confirm that on a live server), nothing is written and every reader + * treats the absence as "unknown", never as "zero". + */ +/** Last shape written per session, so an unchanged prompt costs no filesystem write at all — + * this hook runs on every single request. */ +const lastSystem = new Map() + +function captureSystem(store: JournalStore, sessionID: string, system: readonly string[]): void { + try { + const parts = system + .map((text, i) => ({ name: nameSystemPart(text, i), chars: text.length, text })) + .filter((p) => p.chars > 0) + // hash the text, not just its length: the `` part's date rolls 2026-09-04 → 2026-09-05 + // without changing a single character count, and a length-only key would serve that stale + // text from disk for the life of the process. + const shape = crypto.createHash("sha1").update(parts.map((p) => `${p.name}\n${p.text}`).join("\n\n")).digest("hex") + debug("system.captured", { sessionID, parts: parts.length, chars: parts.reduce((n, p) => n + p.chars, 0), unchanged: lastSystem.get(sessionID) === shape }) + if (parts.length === 0 || lastSystem.get(sessionID) === shape) return + store.writeSystem(sessionID, { v: 1, ts: Date.now(), parts }) + lastSystem.set(sessionID, shape) + } catch (e) { + debug("system.capture.failed", { error: e instanceof Error ? e.message : String(e) }) + } +} + export const server: Plugin = async ({ worktree, client, directory }, options) => { + debug("server.loaded", { worktree, directory }) // same option parsing as the TUI half, so both write to the same place (docs/USAGE.md) const mode: StorageMode = options?.["storage"] === "global" ? "global" : "local" // awaiting an SDK call in the plugin factory deadlocks the server (plugin init blocks @@ -255,9 +303,21 @@ export const server: Plugin = async ({ worktree, client, directory }, options) = ) }, - // DESIGN.md §6.8: a system note so the model reads ◆ / ✂ markers correctly + // DESIGN.md §6.8: a system note so the model reads ◆ / ✂ markers correctly, and §7.4: + // this is the one place the plugin can see what the provider is really sent as its system + // prompt, so it is also where the consumers view gets the bucket it was missing. "experimental.chat.system.transform": async ({ sessionID }, output) => { - if (!sessionID || !(await journal).stateForSession(sessionID)) return + if (!sessionID) return + // Snapshot what OpenCode assembled, BEFORE our own note joins it — the note is ours, and + // counting it as part of the user's prompt would be a small lie in the one view that + // exists to say where the context went. + // + // Deliberately NOT gated on the session being in a tree, unlike the note below: a session + // is only registered by a branch/fork/adoption, so gating here would mean a plain session + // never captured its prompt and the consumers bucket silently never appeared — which is + // the common case, not the edge one. + captureSystem(await journal, sessionID, output.system) + if (!(await journal).stateForSession(sessionID)) return output.system.push( "Context notes: messages starting with ◆ are decision records confirmed by the user — treat them as settled facts. Tool results reading [cropped: …] or turns reading [dropped turn …] were removed from your context on purpose to save space; if you need one back, ask the user to restore it (they can with /undo in the context tree).", ) diff --git a/src/shared/store.ts b/src/shared/store.ts index 81e55a2..37d04d7 100644 --- a/src/shared/store.ts +++ b/src/shared/store.ts @@ -7,7 +7,7 @@ */ import fs from "node:fs" import path from "node:path" -import { foldJournal, parseJournal, type JournalActor, type JournalEntry, type TreeState } from "../core/journal.js" +import { foldJournal, parseJournal, SystemSnapshot, type JournalActor, type JournalEntry, type TreeState } from "../core/journal.js" export type StorageMode = "local" | "global" @@ -54,6 +54,39 @@ export class JournalStore { if (!fs.existsSync(gitignorePath)) fs.writeFileSync(gitignorePath, "*\n") } + /** + * Where a session's system-prompt snapshot lives. One file per session, **overwritten** each + * request rather than appended: it is the current state of something outside the message + * tree, not a mutation with history, so it must not grow the way the journal does. + */ + private systemPath(sessionID: string): string { + // the id reaches us from the host and lands in a path we write: a `/` or `..` in it would + // escape baseDir, so anything but an id-shaped string is refused rather than sanitised + if (!/^[A-Za-z0-9_-]+$/.test(sessionID)) throw new Error(`refusing to use unsafe sessionID in a path: ${JSON.stringify(sessionID)}`) + return path.join(this.baseDir, `system-${sessionID}.json`) + } + + /** Record the system parts the provider is actually being sent (server half, at request + * time). Best effort: a session whose prompt we never see simply has no snapshot, and every + * reader treats that as "unknown" rather than "zero". */ + writeSystem(sessionID: string, snapshot: SystemSnapshot): void { + this.ensureDir() + const tmp = `${this.systemPath(sessionID)}.${process.pid}.${Date.now()}.tmp` + fs.writeFileSync(tmp, `${JSON.stringify(snapshot)}\n`) + fs.renameSync(tmp, this.systemPath(sessionID)) + } + + /** The last snapshot for a session, or undefined when we have never seen its prompt. */ + readSystem(sessionID: string): SystemSnapshot | undefined { + try { + const raw = JSON.parse(fs.readFileSync(this.systemPath(sessionID), "utf8")) as unknown + const parsed = SystemSnapshot.safeParse(raw) + return parsed.success ? parsed.data : undefined + } catch { + return undefined + } + } + private journalPath(treeId: string): string { return path.join(this.baseDir, `${treeId}.jsonl`) } diff --git a/src/tui/route.tsx b/src/tui/route.tsx index 7808a94..3bd79a9 100644 --- a/src/tui/route.tsx +++ b/src/tui/route.tsx @@ -16,7 +16,7 @@ import type { Transcript } from "../core/transcript.js" import type { JournalStore } from "../shared/store.js" import { applyCrop, branchLabel, BRANCH_DIALOG, clip as clipTo, copyText, createNamedBranch, describeTail, executeJump, executeUndo, jumpDialogOptions, jumpDialogTitle, mergeBranch, mergeDialogOptions, mergeDialogTitle, mergePickerFigures, MERGE_TRUST, setLabel, UNDO_KEY, type ActionContext, type MergeMode, type SummaryChoice } from "./actions.js" import { decisionSummary, exportDecisions, renderDecision } from "../core/decision.js" -import { layoutEventStrip, overviewTrack, stripIndexFor, windowFor, type LaneMode, type StripCell } from "../core/lanes.js" +import { laneLabel, laneSuffix, layoutEventStrip, overviewTrack, stripIndexFor, windowFor, LANE_CHROME, type LaneMode, type StripCell } from "../core/lanes.js" import { bar, consumers, type Consumer, type ConsumerEntry } from "../core/consumers.js" import { hasEditor } from "./editor.js" import fs from "node:fs" @@ -410,8 +410,12 @@ export function TreeRoute(props: TreeRouteProps) { const helpHeight = () => (panel() === "help" ? Math.min(HELP.length, Math.max(0, size().rows - 12)) : 0) const width = () => Math.max(60, cols() - 4) // ---- lane geometry (the lanes themselves are further down) ---------------- - // 61 = the 12-cell label column + the `N…` cue + the mode legend that follows the Input lane - const laneWidth = () => Math.max(10, Math.min(width() - 61, 80)) + /** The strip fills the terminal, ending on the same column as the rows and the status line. + * The `+ 2` is the `│ ` a row draws *outside* its own width — a lane label carries its own, + * so those two columns come back to the strip. No ceiling: the strip is a window onto an + * unbounded layout, so more cells is more events visible and less scrolling, and the + * overview track hides itself once nothing is off-screen (`laneOverview`). */ + const laneWidth = () => Math.max(10, width() + 2 - LANE_CHROME) const layout = createMemo(() => layoutEventStrip(live() ?? EMPTY_TRANSCRIPT, laneMode(), filter())) /** DESIGN.md §7.6: below 80 columns the strip is the Input lane alone. */ const showAllLanes = () => cols() >= 80 @@ -615,8 +619,6 @@ export function TreeRoute(props: TreeRouteProps) { const laneOffset = () => laneStart() ?? Math.max(0, layout().totalWidth - laneWidth()) const hiddenLeft = createMemo(() => layout().spans.filter((s) => s.end <= laneOffset()).length) const hiddenRight = createMemo(() => layout().spans.filter((s) => s.start >= laneOffset() + laneWidth()).length) - /** `│ Input …12 `: one fixed-width column, so the lanes stay aligned whatever the cues say. */ - const laneLabel = (name: string, cue = "") => `│ ${name} ${cue}`.padEnd(12) const laneCue = (n: number) => (n > 999 ? "999" : String(n)) const cellColor = (cell: StripCell): unknown => { if (cell.error) return t.error @@ -794,7 +796,17 @@ export function TreeRoute(props: TreeRouteProps) { createEffect(on(() => `${current()?.id ?? ""}:${showInspectorFull()}`, () => setInspectorTop(0))) // ---- consumers ------------------------------------------------------------- - const consumerRows = createMemo(() => (live() ? consumers(live()!, { cropped: alreadyCropped(), limit: contextLimit() }) : [])) + /** The system parts the server half captured for this session, if it has seen a request yet. + * Absent means "we have not seen this session's prompt", never "it costs nothing", so the + * bucket simply does not appear rather than reporting a misleading zero. */ + const systemParts = createMemo(() => { + tick() // the server half rewrites the snapshot on each request; follow the same poll the tree does + // only the consumers panel reads this, and the memo is eager: without the gate every poll + // tick pays a readFileSync — a thrown-and-caught ENOENT, in the common no-snapshot case + if (!sessionID || panel() !== "consumers") return undefined + return store.readSystem(sessionID)?.parts + }) + const consumerRows = createMemo(() => (live() ? consumers(live()!, { cropped: alreadyCropped(), limit: contextLimit(), system: systemParts() }) : [])) /** Buckets plus the entries of every expanded one, flattened so ↑↓ walks both. */ type ConsumerLine = { bucket: Consumer; entry?: ConsumerEntry } const consumerLines = createMemo((): ConsumerLine[] => @@ -848,6 +860,22 @@ export function TreeRoute(props: TreeRouteProps) { } function copySelected() { + // in the consumers panel `y` copies the selected entry — the only way to read a system + // part in full, since it is not a message and has no row of its own + if (panel() === "consumers") { + const line = consumerLine() + const idx = line?.bucket.kind === "system" ? line.entry?.systemIndex : undefined + const part = idx === undefined ? undefined : systemParts()?.[idx] + const text = part?.text ?? line?.entry?.preview ?? "" + if (!text) return + try { + const { hint } = copyText(api, text, directory) + notify(`copied ${text.length} chars → ${hint}`) + } catch (e) { + notify(`copy failed: ${e instanceof Error ? e.message : String(e)}`) + } + return + } const row = current() if (!row || row.kind === "branch" || row.kind === "separator") return const tr = row.sessionID === sessionID ? live() : others()[row.sessionID] @@ -1368,7 +1396,7 @@ export function TreeRoute(props: TreeRouteProps) { { name: "ctree.inspector_up", hidden: true, enabled: inspectorOpen, run: () => scrollInspector(-1) }, { name: "ctree.inspector_down", hidden: true, enabled: inspectorOpen, run: () => scrollInspector(1) }, { name: "ctree.consumers", hidden: true, enabled: () => !inCrop(), run: () => setPanel(panel() === "consumers" ? "tree" : "consumers") }, - { name: "ctree.copy", hidden: true, enabled: treeIdle, run: () => copySelected() }, + { name: "ctree.copy", hidden: true, enabled: () => treeIdle() || panel() === "consumers", run: () => copySelected() }, { name: "ctree.mode_duration", hidden: true, enabled: treePanel, run: () => setLane("duration") }, { name: "ctree.mode_turns", hidden: true, enabled: treePanel, run: () => setLane("turns") }, { name: "ctree.lanes_off", hidden: true, enabled: treePanel, run: () => { setLanesOn(false); api.kv.set("ctree.lanesOn", false) } }, @@ -1518,7 +1546,7 @@ export function TreeRoute(props: TreeRouteProps) { {"no input".padEnd(laneWidth())}}> {(r) => {r.text}} - {`${(hiddenRight() > 0 ? `${laneCue(hiddenRight())}…` : "").padStart(4).padEnd(5)}${laneMode() === "duration" ? "[1] Duration" : " 1 duration"} · ${laneMode() === "turns" ? "[2] Turns" : " 2 turns"} · 0 off`} + {laneSuffix(hiddenRight() > 0 ? `${laneCue(hiddenRight())}…` : "", laneMode())} diff --git a/test/e2e/server.test.ts b/test/e2e/server.test.ts index 76bf745..1467750 100644 --- a/test/e2e/server.test.ts +++ b/test/e2e/server.test.ts @@ -24,6 +24,15 @@ function unwrap(res: { data?: T; error?: unknown }): T { return res.data } +/** Every system message the provider was sent, joined — `output.system` parts may arrive as + * separate messages or as one, and structured content is stringified rather than assumed. */ +function systemText(req: { body: { messages: { role: string; content: unknown }[] } }): string { + return req.body.messages + .filter((m) => m.role === "system") + .map((m) => (typeof m.content === "string" ? m.content : JSON.stringify(m.content))) + .join("\n") +} + async function installSpikePlugin(projectDir: string): Promise { const pluginsDir = path.join(projectDir, ".opencode", "plugins") await mkdir(pluginsDir, { recursive: true }) @@ -312,6 +321,27 @@ describe.skipIf(!e2e)("server e2e: built plugin headless /ctree commands", () => expect(readFileSync(path.join(server.dir, "ctree-decisions.md"), "utf8")).toContain("# Decisions") }, 180_000) + // DESIGN.md §6.8. `applyCrops` ships `[cropped: …]` stubs and `◆` records to the model; this + // note is the only thing that tells it how to read them, and nothing proved the note survived + // `experimental.chat.system.transform` into the actual provider request. It is gated on tree + // membership, so a plain session legitimately has none — branch first, then look. + test("the ◆/crop system note reaches the provider once the session is in a tree", async () => { + const session = unwrap(await server.client.session.create({ query: { directory: server.dir }, body: { title: "system-note" } })) + + mock.clearRequests() + unwrap(await server.client.session.prompt({ path: { id: session.id }, query: { directory: server.dir }, body: { parts: [{ type: "text", text: "seed turn" }] } })) + expect(systemText(mock.requests().at(-1)!)).not.toContain("Context notes:") + + unwrap(await server.client.session.command({ path: { id: session.id }, query: { directory: server.dir }, body: { command: "ctree", arguments: "branch note-check mock/mock-b" } })) + + mock.clearRequests() + unwrap(await server.client.session.prompt({ path: { id: session.id }, query: { directory: server.dir }, body: { parts: [{ type: "text", text: "second turn" }] } })) + const sys = systemText(mock.requests().at(-1)!) + expect(sys).toContain("Context notes:") + // and it is appended to OpenCode's own prompt, not sent instead of it + expect(sys.length).toBeGreaterThan(500) + }, 180_000) + test("a native session.fork (no plugin command) is adopted into the tree", async () => { const session = unwrap(await server.client.session.create({ query: { directory: server.dir }, body: { title: "native-fork" } })) unwrap(await server.client.session.prompt({ path: { id: session.id }, query: { directory: server.dir }, body: { parts: [{ type: "text", text: "seed turn" }] } })) diff --git a/test/e2e/tui.test.ts b/test/e2e/tui.test.ts index cbc0da4..c0824d7 100644 --- a/test/e2e/tui.test.ts +++ b/test/e2e/tui.test.ts @@ -4,7 +4,7 @@ */ import { afterAll, beforeAll, describe, expect, test } from "bun:test" import path from "node:path" -import { readFileSync, readdirSync } from "node:fs" +import { existsSync, readFileSync, readdirSync } from "node:fs" import { createProject, installPlugins, REPO_ROOT, runTui, runTuiScreens, startMock, type StartedMock } from "./harness.js" const e2e = process.env["CTREE_E2E"] === "1" @@ -231,6 +231,56 @@ describe.skipIf(!e2e)("tui e2e: built plugin", () => { } }, 320_000) + test("the server captures the real system prompt; consumers shows it as a bucket", async () => { + const m = await startMock({ tool: false }) + const proj = await createProject({ mockPort: m.port }) + await installPlugins({ projectDir: proj.dir, server: [path.join(REPO_ROOT, "dist", "server.js")], tui: [path.join(REPO_ROOT, "dist", "tui.js")] }) + try { + const log = path.join(proj.dir, "ctree-debug.log") + const { screens } = await runTuiScreens({ + projectDir: proj.dir, + env: { CTREE_DEBUG: log }, + keys: [ + // a plain session that never branches: the case the capture must not miss + ["Ask anything", 1, "hello\r"], + ["mock reply", 8, "/tree"], + ["Context tree", 0.5, "\r"], + ["Context tree ·", 2, "s"], + ["what's filling|consumers", 3, "\x03"], + ["", 1, "\x03"], + ], + timeoutSec: 180, + cols: 130, + rows: 34, + exitWhenDone: true, + }) + + // 1. the assumption the whole feature rests on: `output.system` arrives carrying + // OpenCode's own parts, so there is something real to snapshot + const debugLog = existsSync(log) ? readFileSync(log, "utf8") : "(no debug log)" + const dir = path.join(proj.dir, ".opencode", "context-tree") + if (!existsSync(dir)) throw new Error(`no context-tree dir. debug log:\n${debugLog.split("\n").filter((l) => l.includes("system")).join("\n") || debugLog.slice(0, 2000)}`) + const snap = readdirSync(dir).find((f) => f.startsWith("system-") && f.endsWith(".json")) + expect(snap).toBeDefined() + const parsed = JSON.parse(readFileSync(path.join(dir, snap!), "utf8")) as { v: number; parts: { name: string; chars: number; text: string }[] } + expect(parsed.v).toBe(1) + expect(parsed.parts.length).toBeGreaterThan(0) + expect(parsed.parts.reduce((n, p) => n + p.chars, 0)).toBeGreaterThan(200) + + // 2. our own note is NOT in the snapshot: it is captured before we push it + expect(parsed.parts.some((p) => p.text.startsWith("Context notes:"))).toBe(false) + + // 3. and it reaches the consumers view. Assert against the pyte-rendered screen, not + // the raw stream: the panel is painted cell-by-cell, so no label lands there whole. + const consumers = screens.find((s) => s.screen.includes("what's filling the context")) + if (!consumers) throw new Error(`consumers panel never rendered. screens: ${screens.map((s) => s.label).join(" | ")}`) + expect(consumers.screen).toContain("≡ system prompt") + } finally { + await m.stop() + await proj.cleanup() + } + }, 300_000) + test("/tree opens the context tree route with rows and a context header", async () => { const text = await runTui({ projectDir: project.dir, diff --git a/test/lanes.test.ts b/test/lanes.test.ts index 2ff53a2..fdf67c9 100644 --- a/test/lanes.test.ts +++ b/test/lanes.test.ts @@ -1,5 +1,5 @@ import { describe, expect, test } from "bun:test" -import { eventAllowed, layoutEventStrip, overviewTrack, stripIndexFor, windowFor, type EventLayout } from "../src/core/lanes.js" +import { eventAllowed, laneLabel, laneModeLine, laneSuffix, layoutEventStrip, overviewTrack, stripIndexFor, windowFor, type EventLayout, LANE_CHROME, LANE_LABEL_WIDTH } from "../src/core/lanes.js" import { bar, consumers } from "../src/core/consumers.js" import type { Transcript, TranscriptMessage } from "../src/core/transcript.js" import { assistant, buildFixture, OPEN, user } from "./fixtures/tree.js" @@ -26,6 +26,55 @@ describe("consumers", () => { expect(c[0]!.tokens).toBeLessThan(consumers(open)[0]!.tokens) expect(bar(0.5, 10)).toBe("▰▰▰▰▰▱▱▱▱▱") }) + + test("the system prompt is a bucket, so the view reconciles with the ctx gauge", () => { + // without it this view walks the transcript only, and silently omits a chunk the header's + // `ctx …` (tokens.input) does include + const system = [ + { name: "base prompt", text: "x".repeat(8000) }, + { name: "AGENTS.md", text: "y".repeat(4000) }, + ] + const withSystem = consumers(open, { system }) + const without = consumers(open) + const bucket = withSystem.find((c) => c.kind === "system")! + expect(bucket.source).toBe("≡ system prompt") + expect(bucket.tokens).toBe(3000) // (8000 + 4000) / 4 + expect(bucket.count).toBe(2) + expect(withSystem.reduce((n, c) => n + c.tokens, 0)).toBe(without.reduce((n, c) => n + c.tokens, 0) + 3000) + }) + + test("its parts are separate entries, biggest first, and none is croppable", () => { + const system = [ + { name: "base prompt", text: "x".repeat(400) }, + { name: "AGENTS.md", text: "y".repeat(4000) }, + ] + const bucket = consumers(open, { system }).find((c) => c.kind === "system")! + expect(bucket.entries.map((e) => e.preview.split(":")[0])).toEqual(["AGENTS.md", "base prompt"]) + expect(bucket.entries.every((e) => !e.croppable)).toBe(true) + expect(bucket.note).toContain("not croppable") + }) + + test("no snapshot means no bucket — absent is 'unknown', never 'zero'", () => { + expect(consumers(open).some((c) => c.kind === "system")).toBe(false) + expect(consumers(open, { system: [] }).some((c) => c.kind === "system")).toBe(false) + }) + + test("shares still sum to 1 once the system prompt is in", () => { + const cs = consumers(open, { system: [{ name: "base prompt", text: "x".repeat(8000) }] }) + expect(cs.reduce((n, c) => n + c.share, 0)).toBeCloseTo(1, 5) + }) + + // names are a heuristic, so two parts can share one; `y` must still copy the part you picked + test("each system entry points at its own part, even when two share a name", () => { + const system = [ + { name: "AGENTS.md", text: "global ".repeat(600) }, + { name: "AGENTS.md", text: "project ".repeat(200) }, + ] + const entries = consumers(open, { system }).find((c) => c.kind === "system")!.entries + expect(entries.map((e) => e.systemIndex)).toEqual([0, 1]) + // sorted biggest-first, so the index must not be positional + expect(system[entries[0]!.systemIndex!]!.text.startsWith("global")).toBe(true) + }) }) const T = (messages: TranscriptMessage[]): Transcript => ({ sessionID: "s", title: "strip", status: "available", messages }) @@ -219,3 +268,28 @@ describe("overview track", () => { expect(track.at(-1)).toBe("window") }) }) + +describe("lane chrome", () => { + test("the mode line is the same width whichever mode is selected", () => { + // the strip is sized as "the terminal minus the chrome", so a mode line that changed width + // with the selection would shift every pill sideways on `1`/`2` + expect(laneModeLine("turns").length).toBe(laneModeLine("duration").length) + }) + + test("the label column is fixed width, cue or no cue, longest lane name or shortest", () => { + for (const name of ["Input", "Model", "Tools"]) { + expect(laneLabel(name).length).toBe(LANE_LABEL_WIDTH) + expect(laneLabel(name, "…999").length).toBe(LANE_LABEL_WIDTH) + } + }) + + test("the suffix is fixed width, cue or no cue", () => { + expect(laneSuffix("", "turns").length).toBe(laneSuffix("999…", "duration").length) + }) + + test("LANE_CHROME is what a lane row actually spends off-strip", () => { + // measured, never written down: the old reserve was a literal 61 and stayed 61 when the + // "3 calls" mode went away, so the strip stopped ~10 columns short of the right edge + expect(LANE_CHROME).toBe(laneLabel("Input", "…999").length + laneSuffix("999…", "turns").length) + }) +}) diff --git a/test/store.test.ts b/test/store.test.ts index 4b7d81a..24eabcb 100644 --- a/test/store.test.ts +++ b/test/store.test.ts @@ -101,3 +101,36 @@ describe("JournalStore registry", () => { expect(store.stateFor(treeId).labels["msg_1"]?.label).toBe("x") }) }) + +describe("system prompt snapshot", () => { + const store = () => new JournalStore({ worktree: fs.mkdtempSync(path.join(os.tmpdir(), "ctree-sys-")) }) + + test("round-trips, and is overwritten rather than appended", () => { + const s = store() + s.writeSystem("ses_a", { v: 1, ts: 1, parts: [{ name: "base prompt", chars: 3, text: "abc" }] }) + s.writeSystem("ses_a", { v: 1, ts: 2, parts: [{ name: "AGENTS.md", chars: 2, text: "hi" }] }) + // the prompt changes between requests and has no history: the file must not grow + expect(s.readSystem("ses_a")).toEqual({ v: 1, ts: 2, parts: [{ name: "AGENTS.md", chars: 2, text: "hi" }] }) + }) + + test("a session we have never seen a request for reads as undefined, not empty", () => { + expect(store().readSystem("ses_never")).toBeUndefined() + }) + + test("a corrupt or foreign file reads as undefined instead of throwing", () => { + const s = store() + s.writeSystem("ses_b", { v: 1, ts: 1, parts: [] }) + fs.writeFileSync(path.join(s.dir, "system-ses_b.json"), "{not json") + expect(s.readSystem("ses_b")).toBeUndefined() + fs.writeFileSync(path.join(s.dir, "system-ses_b.json"), JSON.stringify({ v: 2, ts: 1, parts: [] })) + expect(s.readSystem("ses_b")).toBeUndefined() + }) + + test("snapshots are per session and do not collide", () => { + const s = store() + s.writeSystem("ses_a", { v: 1, ts: 1, parts: [{ name: "base prompt", chars: 1, text: "a" }] }) + s.writeSystem("ses_b", { v: 1, ts: 1, parts: [{ name: "base prompt", chars: 1, text: "b" }] }) + expect(s.readSystem("ses_a")!.parts[0]!.text).toBe("a") + expect(s.readSystem("ses_b")!.parts[0]!.text).toBe("b") + }) +})