Skip to content

feat(compaction): Prepare a summary in a background job - #2606

Merged
thomasjiangcy merged 2 commits into
mainfrom
compaction-background-job
Nov 3, 2026
Merged

thomasjiangcy merged 2 commits into
mainfrom
compaction-background-job

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Targets main (rebased after #2602 merged as 436e67c). This is workstream F. G is #2605 (swap + tool-result trimming). F and G ship together: without G's swap, F prepares summaries that no Turn uses.

Uses the #2600 compaction_requested / compaction_prepared schemas unchanged. Those already shipped in 0.274.0, so no reader release has to go first.

Mark (end of a completed Turn)

requestBackgroundCompaction runs in executeAgentRun after the agent loop succeeds. It does not run on error, timeout, or auth pause. It reads the last assistant message's provider-reported input (input + cacheRead + cacheWrite). It requests a summary when that input is at or above the mark. The mark is getAgentBackgroundCompactionMarkTokens = 70% of getAgentContextCompactionTriggerTokens, so it uses the same budget as blocking compaction.

Above the mark, it loads the current History and stops in any of these cases:

  • a request for this History version is still pending
  • a ready result for this version was prepared after this execution slice started (the next Turn swaps it in)
  • 2 attempts already failed for this version: a failed result, no result after 10 min, or a ready result that a Turn passed over

A Turn swaps a ready summary in before its first model request, and a swap makes a new History version. So a ready result of the current version that predates the slice start was discarded by #2605 (over_target, split_tool_call, profile_changed). It no longer blocks a new request with a later mark. There is no Model profile check on requests: a handoff already makes a new History version, and the request records the Turn's profile.

Otherwise it does the following, in order:

  1. Saves a request snapshot in the State adapter with the queue retention TTL: system prompt, tools (name, label, description, parameters, constrainedSampling), thinking level, model id, privacy, and input size.
  2. Appends compaction_requested with markSeq = last model-context seq, using an idempotency key and activity: "preserve".
  3. Sends a signed message {conversationId, historyVersion, markSeq}.

A failure here is logged and does not fail the Turn. If the send fails after the append, the request times out and counts as one failed attempt.

Job

Topic junior_context_compaction, route /api/internal/context/compactions. Nitro adds a Vercel queue trigger. The app wires a dev consumer, a queue-worker consumer, and a createApp({ backgroundCompactionQueue }) override. The agent gets the queue through AgentEnvironment, as with conversationWorkQueue.

The job:

  • Projects the events through markSeq.
  • Builds a Pi Agent from the snapshot, with the same options as the live agent.
  • Calls feat(compaction): Summarize with the agent model and cached request prefix #2602's buildAgentModelRequest + requestContextSummary through the traced stream (gen_ai.prompt.name: junior.context_compaction).
  • Writes compaction_prepared exactly once, with ready + summary or failed + a fixed label. Labels: timeout, snapshot_missing, provider_<kind>, summary_error, empty, max_tokens, missing_summary, tool_call. Exception text is never stored.
  • Writes nothing when the History was replaced (stale) or the request already has a result (duplicate).
  • Deletes the snapshot after it writes a result.

Model failures become events. Queue retries (max 3) cover storage errors only.

Telemetry

agent.context_compaction span with app.compaction.mode: background. It runs in its own trace and adds nothing to app.turn.compaction_wait_ms. Attributes: app.compaction.outcome, mark_seq, history_version, input.size, queue_wait_ms, ready_ms, summary_chars, and error.type. The Turn side logs context_compaction.background.requested. All of this is documented in TELEMETRY.md.

Tests

  • tests/integration/background-compaction-events.test.ts: uses the real event store and State adapter. The only fake is a model streamFn, the same pattern as the context-summary unit test. It covers:
    • below the mark vs. at the mark
    • the summary request's system prompt, tools JSON, and messages match buildAgentModelRequest on the live agent, with only the instruction appended
    • the ready path, and that a ready result prepared during the Turn blocks further requests
    • a ready result that a later Turn passed over allows a new request, and passed-over results count toward the attempt limit
    • a repeated delivery writes nothing
    • the fixed failure label and the stop after two failures
    • no write after the History was replaced
  • Nitro unit test: the new function rule and trigger.
  • Eval fixture: jobs run in process and are awaited, like plugin tasks. No new eval: the dashboard API leaves out these events, so an eval cannot see them yet. G's swap gives evals something to check.

Typecheck, lint (incl. depcruise and test-architecture), and file-length pass. The full local vitest run showed 18 timeouts in agent integration tests. The same files also time out on the base branch in this sandbox (checked with the change stashed), so those failures come from the sandbox, not this change.

For reviewers

  • Turn latency: above the mark, the Turn waits on one history load, a State write, an append, and a queue send before it returns. Below the mark there is no I/O.
  • Passed-over summaries: F tells that feat(compaction): Swap in prepared background summaries #2605 discarded a summary by comparing compaction_prepared.createdAtMs (job clock) with the slice start (Turn clock), not by reading G's discard reason. Clock skew can cost one extra request or one extra Turn of waiting. This relies on G swapping a ready summary at the start of every slice.
  • Snapshot size: the system prompt and tools JSON can be about 100 KB per request in Redis, with a 1 h TTL.
  • Eval cost: compaction evals with a 50k window now also run a background summary call (mark ≈ 31.5k).

--

View Session [Sentry]

@vercel

vercel Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
junior-docs Ready Ready Preview Nov 3, 2026 2:12pm UTC

Request Review

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

1 screenshot change — 1 changed · 0 added · 0 removed

Review screenshots in Frameshift

Settings Mcp · Desktop
Settings Mcp · Desktop
Changed

When a completed Turn's last model request is at or above the mark (70% of
the compaction threshold, provider-reported input), the Turn writes
compaction_requested and sends a signed queue job. The job repeats the
request prefix of the last agent request and writes compaction_prepared
with a summary or a fixed failure label.
A Turn swaps a ready summary in before its first model request, which makes
a new History version. A ready summary of the current version that predates
the Turn was discarded (over target, split tool call, or another Model
profile), so it no longer blocks a new request. It counts toward the attempt
limit. Drop the Model profile guard on requests: a handoff already makes a
new History version, and the request records the Turn's profile.
@sentry-junior
sentry-junior Bot force-pushed the compaction-background-job branch from 4a67c89 to 28dc7cb Compare November 3, 2026 14:11
@thomasjiangcy
thomasjiangcy merged commit 36af417 into main Nov 3, 2026
50 of 63 checks passed
@thomasjiangcy
thomasjiangcy deleted the compaction-background-job branch November 3, 2026 14:19

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 28dc7cb. Configure here.

}
const markSeq = projectConversationEvents(events, {
defaultProfile: botConfig.defaultProfile,
}).seqs.at(-1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong markSeq after history replacement

High Severity

markSeq is taken from projectConversationEvents(...).seqs, which after a compaction or handoff stores sourceEventSeq from the old History. The job then cuts with maxSeq against current event.seq. When the last persisted model message still belongs to that replacement — the usual case, because the final assistant is not checkpointed — the job stops before the replacement event and summarizes an empty or truncated History.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 28dc7cb. Configure here.

if (result.outcome === "failed") {
setSpanAttributes({ "error.type": result.errorType });
}
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale prepared event after replacement

Medium Severity

The job treats a replaced History as stale only before the summary call. writePrepared appends to whatever History is current and does not re-check the version. A long-running job can therefore write a compaction_prepared for the old version into the new History.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 28dc7cb. Configure here.

thomasjiangcy pushed a commit that referenced this pull request Nov 3, 2026
…istory (#2611)

Fixes two Cursor Bugbot findings from #2606 that landed on main. They
were reported on its last head (`28dc7cbd3`), and the PR merged before
the fix was pushed.

**Mark after a replacement (high).** The request took `markSeq` from the
projection seqs. Messages kept by a `compaction` or `handoff`
replacement carry the seqs of their older source events. The job
projects with `maxSeq: markSeq`, so it could stop before the replacement
event and summarize empty or truncated history. The mark now does not
fall before the replacement event. The swap still finds the cut
correctly: kept messages have seqs below the mark, and later native
events have seqs above it.

**Result after a replacement during the job (medium).** The job checked
the History version only before the summary call. A replacement during
the call let it append a `compaction_prepared` event for the old version
into the new current history. The job now checks the version again
before it writes and returns `stale` on a mismatch. A replacement
between that check and the append is still possible, but the swap and
the request check both match `historyVersion`, so that event is never
used. No schema or event store API change.

Both new integration cases in `background-compaction-events.test.ts`
fail without the fix.

<!-- junior-request-attribution:start -->
via @thomasjiangcy.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:agent%3A1612e7f5f3380de0ab261fa0175f20ba -->

--

[View
Session](https://junior-prod.sentry.dev/conversations/agent%3A1612e7f5f3380de0ab261fa0175f20ba)
[[Sentry]](https://sentry.sentry.io/explore/conversations/agent%3A1612e7f5f3380de0ab261fa0175f20ba/?project=4510944073809921)

<!-- junior-session-footer:end -->

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>

This branch was successfully deployed

1 active deployment
Preview – junior-docs — 28dc7cbd Deployed Nov 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant