Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ node_modules/
.DS_Store
*.log
package-lock.json
graft/.cache/
50 changes: 35 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,15 +217,15 @@ It cannot be combined with

backpass reads the local transcript stores of seven harnesses directly. No API, no upload.

| Harness | Store | Repo tie |
| -------------- | ---------------------------------------------- | --------------------------------------------------- |
| **claude** | `~/.claude/projects/<munged-cwd>/<uuid>.jsonl` | per-line `cwd` |
| **codex** | `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl` | `cwd` + recorded `git.repository_url` |
| **pi** | standalone and BB-managed Pi JSONL stores | session-header `cwd` |
| **opencode** | `~/.local/share/opencode/opencode.db` (sqlite) | `session.directory` / `session_v2.directory` |
| **grok** | `~/.grok/sessions/<encoded-cwd>/<uuid>/` | `summary.json` `cwd` + `git_remotes` |
| **cursor CLI** | `~/.cursor/chats/<md5(cwd)>/<uuid>/` | `meta.json` `cwd` |
| **hermes** | `~/.hermes/state.db` (sqlite) | session cwd, with CLI prompt / ACP config fallbacks |
| Harness | Store | Repo tie |
| -------------- | ----------------------------------------------- | --------------------------------------------------- |
| **claude** | `~/.claude/projects/<munged-cwd>/<uuid>.jsonl` | per-line `cwd` |
| **codex** | `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl` | `cwd` + recorded `git.repository_url` |
| **pi** | standalone, OMP, and BB-managed Pi JSONL stores | session-header `cwd` |
| **opencode** | `~/.local/share/opencode/opencode.db` (sqlite) | `session.directory` / `session_v2.directory` |
| **grok** | `~/.grok/sessions/<encoded-cwd>/<uuid>/` | `summary.json` `cwd` + `git_remotes` |
| **cursor CLI** | `~/.cursor/chats/<md5(cwd)>/<uuid>/` | `meta.json` `cwd` |
| **hermes** | `~/.hermes/state.db` (sqlite) | session cwd, with CLI prompt / ACP config fallbacks |

Claude collection covers `$CLAUDE_CONFIG_DIR/projects` alongside the default store, so a
relocated config dir does not hide its sessions. The variable is read from backpass's own
Expand All @@ -238,6 +238,25 @@ sessions under `~/.bb/pi-bridge-sessions/`. It also honors `PI_CODING_AGENT_DIR`
set in backpass's environment. When roots overlap, backpass scans every applicable layout
and reads each JSONL file once.

OMP (Oh My Pi) collection is **off by default**. Set `discovery.includeOmp` to `true` in
`.backpassrc.json` or your [personal config](#configuration) to also read
`~/.omp/agent/sessions/` through the Pi adapter:

```json
{ "discovery": { "includeOmp": true } }
```

Then run `backpass scan --harness pi` (or a normal `backpass` run). The setting also
applies to configured SSH hosts. For `--scope user`, set it in `user.discovery` and
include `"pi"` in `user.discovery.harnesses`; user scope otherwise collects only Claude
and Codex. Explicit Pi store environment overrides still work without this setting;
it controls only the additional default OMP store, not the harness backpass invokes.

OMP nests subagent JSONL files below each parent session, and a subagent's own subagents
one level further down. Backpass analyzes each file separately, but uses the root session as
their shared corroboration source; a session and its subagents cannot count as independent
sessions.

OpenCode collection reads both store layouts: OpenCode 1.x (`session`, `message`, `part`) and OpenCode 2.x (`session_v2`, `session_message`).
For 2.x, session activity uses the later of the session's update time and its newest message's update time.
An upgraded store keeps its 1.x tables beside the copies in `session_v2`, so a session found in both is read from `session_v2`.
Expand Down Expand Up @@ -289,12 +308,12 @@ OpenCode sessions with no recorded messages, such as unused agent probes, are no

Every remaining session is labelled **interactive** or **non-interactive** (`src/interaction.js`).
Codex `codex exec` / `originator: codex_exec`, Claude SDK, GitHub, action, and CI
entrypoints, OpenCode child sessions (`parent_id`), and a cwd with a `.no-mistakes` path
segment are non-interactive. Hermes gateway, cron, and WhatsApp sessions are classified the
same way if they leak past collection's source filter. A no-mistakes pipeline run is just one
kind of non-interactive session, not its own category. Missing harness metadata defaults to
interactive. `backpass scan`, the proposal, and apply all print the mix so relevance is never
silently computed against a robot-skewed pool.
entrypoints, OpenCode child sessions (`parent_id`), OMP subagent transcripts, and a cwd
with a `.no-mistakes` path segment are non-interactive. Hermes gateway, cron, and WhatsApp
sessions are classified the same way if they leak past collection's source filter. A
no-mistakes pipeline run is just one kind of non-interactive session, not its own category.
Missing harness metadata defaults to interactive. `backpass scan`, the proposal, and apply
all print the mix so relevance is never silently computed against a robot-skewed pool.

```sh
backpass scan --since 7d --strict
Expand Down Expand Up @@ -812,6 +831,7 @@ CLI flags on top:
"since": "30d",
"worktreeGlobs": [],
"cloneRoots": [],
"includeOmp": false,
"minUserTurns": 2
},
"jobs": 4
Expand Down
3 changes: 2 additions & 1 deletion src/acpx.js
Original file line number Diff line number Diff line change
Expand Up @@ -906,8 +906,9 @@ function recoverPiUsage({ promptFile, cwd, startedAt }) {
.filter((c) => c.mtimeMs >= since)
.sort((a, b) => b.mtimeMs - a.mtimeMs);

const scanContext = piStore.createScanContext();
for (const candidate of candidates) {
const descriptor = piStore.classify(candidate);
const descriptor = piStore.classify(candidate, { scanContext });
if (!descriptor || !wanted.has(descriptor.cwd)) continue;
const entries = readJsonl(candidate.path);
const firstUser = entries.find((e) => e.type === "message" && e.message?.role === "user");
Expand Down
6 changes: 5 additions & 1 deletion src/analyze.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import { renderOpenGapIndex } from "./gap-ledger.js";
import { evidenceKey, isEvidenceFresh, safeFileName } from "./state.js";
import { emitProgress } from "./progress.js";
import { UserError, color, info, warn } from "./logger.js";
import { transcriptIdentity } from "./transcript.js";
import { corroborationIdentityOf, transcriptIdentity } from "./transcript.js";

/**
* Stage 1 of the pipeline (design section 3): one cheap model call per transcript,
Expand Down Expand Up @@ -410,6 +410,10 @@ export async function analyzeTranscripts({
harness: transcript.harness,
id: transcript.id,
identity: transcriptIdentity(transcript),
parentSessionId: transcript.parentSessionId || null,
corroborationIdentity: corroborationIdentityOf(transcript),
corroborationNativeId: transcript.corroborationNativeId || null,
corroborationStartedAt: transcript.corroborationStartedAt ?? null,
path: transcript.path,
mtimeMs: transcript.mtimeMs,
bytes: transcript.bytes,
Expand Down
51 changes: 36 additions & 15 deletions src/commands/propose.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
import { consolidateGapLedger } from "../consolidate.js";
import { foldEvidence } from "../fold.js";
import { ledgerGapObservations, pruneGapLedger, recordGapObservations } from "../gap-ledger.js";
import {
ledgerGapObservations,
normalizeGapLedgerSessions,
pruneGapLedger,
recordGapObservations,
} from "../gap-ledger.js";
import { synthesizeProposal } from "../synthesize.js";
import { ProposalViolation } from "../proposal.js";
import { formatCorpusMix, INTERACTIVE, NON_INTERACTIVE } from "../interaction.js";
Expand All @@ -13,7 +18,7 @@ import { printUsage } from "./usage.js";
import { closeRemoteDiscovery, discoverForRun } from "./scan.js";
import { capTranscripts } from "../sample.js";
import { isEvidenceFresh } from "../state.js";
import { transcriptIdentity } from "../transcript.js";
import { corroborationIdentityOf, transcriptIdentity } from "../transcript.js";
import { pruneHostCache } from "../discovery/cache.js";

/**
Expand All @@ -29,11 +34,12 @@ import { pruneHostCache } from "../discovery/cache.js";
* cap remain on disk. Folding those records would inflate `analyzedSessions` beyond the
* sampled corpus or score positional instruction aliases against an index they never saw.
* Legacy records stay excluded until ordinary discovery and analysis backfill them.
* Current discovery's corroboration fields are overlaid on admitted records, so a
* subagent analyzed before its parent appeared still folds under the parent's identity.
*/
export async function foldForRun(ctx, memoryFile, memoryHash, skills = [], transcripts = [], { route = null } = {}) {
const { state, minGapEvidence, gapLedgerMaxAge } = ctx.config;
const selectedByIdentity = new Map(transcripts.map((transcript) => [transcriptIdentity(transcript), transcript]));
const selected = new Set(selectedByIdentity.keys());
const evidence = state.listEvidence();
const identitiesByLegacyId = new Map();
for (const record of evidence) {
Expand All @@ -42,26 +48,41 @@ export async function foldForRun(ctx, memoryFile, memoryHash, skills = [], trans
if (!identitiesByLegacyId.has(legacyId)) identitiesByLegacyId.set(legacyId, new Set());
identitiesByLegacyId.get(legacyId).add(transcriptIdentity(record.transcript));
}
const selectedGapSessions = new Set(selected);
const selectedGapSessions = new Set(selectedByIdentity.keys());
const legacyIds = new Set();
for (const transcript of transcripts) {
selectedGapSessions.add(corroborationIdentityOf(transcript));
const identities = identitiesByLegacyId.get(transcript.id);
if (identities?.size === 1 && identities.has(transcriptIdentity(transcript))) {
selectedGapSessions.add(transcript.id);
legacyIds.add(transcript.id);
}
}
const relevant = evidence.filter((e) => {
const currentTranscript = selectedByIdentity.get(transcriptIdentity(e.transcript));
return (
e.memoryPath === memoryFile.path &&
e.memoryHash === memoryHash &&
(e.transcript?.interaction === INTERACTIVE || e.transcript?.interaction === NON_INTERACTIVE) &&
currentTranscript &&
isEvidenceFresh(e, currentTranscript, memoryHash)
);
});
const relevant = [];
for (const record of evidence) {
const currentTranscript = selectedByIdentity.get(transcriptIdentity(record.transcript));
if (
record.memoryPath !== memoryFile.path ||
record.memoryHash !== memoryHash ||
(record.transcript?.interaction !== INTERACTIVE && record.transcript?.interaction !== NON_INTERACTIVE) ||
!currentTranscript ||
!isEvidenceFresh(record, currentTranscript, memoryHash)
) {
continue;
}
const transcript = {
...record.transcript,
parentSessionId: currentTranscript.parentSessionId || null,
corroborationIdentity: corroborationIdentityOf(currentTranscript),
corroborationNativeId: currentTranscript.corroborationNativeId || null,
corroborationStartedAt: currentTranscript.corroborationStartedAt ?? null,
};
relevant.push({ ...record, transcript });
}

const ledger = state.readGapLedger();
recordGapObservations(ledger, relevant, { skills });
normalizeGapLedgerSessions(ledger, transcripts, { legacyIds });
recordGapObservations(ledger, relevant, { skills, legacyIds });
// Consolidate after recording, so the pass sees this run's sightings too: two
// sessions coining the same brand-new gap in one parallel fan-out can only line up
// here. One bounded judged call; a failure degrades to lexical identity and the run
Expand Down
5 changes: 5 additions & 0 deletions src/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ export const DEFAULT_CONFIG = {
* is refused by name. Absolute paths; `~` is expanded.
*/
opencodeStores: [],
/** Include Oh My Pi's default session store in Pi discovery; opt-in only. */
includeOmp: false,
minUserTurns: 2,
includeCursorIde: false,
},
Expand Down Expand Up @@ -467,6 +469,9 @@ function validate(config, { kind = "project", repoRoot = null } = {}) {
}
config.discovery.harnesses = config.discovery.harnesses.filter((h) => known.has(h));
parseSince(config.discovery.since);
if (typeof config.discovery.includeOmp !== "boolean") {
throw new UserError("config.discovery.includeOmp must be a boolean");
}
if (!Array.isArray(config.discovery.cloneRoots) || config.discovery.cloneRoots.some((p) => typeof p !== "string")) {
throw new UserError("config.discovery.cloneRoots must be an array of paths");
}
Expand Down
Loading
Loading