Skip to content

ScaleDown proxy mode: real per-turn token savings + true compaction (v0.5.0) - #9

Merged
varchanaiyer merged 1 commit into
mainfrom
feat/proxy-mode
Jun 28, 2026
Merged

varchanaiyer merged 1 commit into
mainfrom
feat/proxy-mode

Conversation

@soham96

@soham96 soham96 commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Why

Hooks can only append to the context window — they can't rewrite the history Claude re-sends each turn, and PreCompact can't replace Claude's summary (anthropics/claude-code#24965). So the old pre-compact.ts (built on a non-existent messages_to_compact contract) never ran, and a hook-injected summary would only cost tokens. Real savings require sitting in the request path.

What

A local proxy — dietcode claude (wrapper, recommended) or dietcode proxy (foreground) — that Claude Code talks to via ANTHROPIC_BASE_URL. On each /v1/messages request it rewrites the outgoing payload via progressive compaction:

[ system + tools ]            untouched (keeps Anthropic's prompt cache warm)
[ running ScaleDown summary ] stands in for older turns; extended only at compaction steps
[ last N turns verbatim ]     live working set
  • Not per-call — a per-session running summary is reused; ScaleDown is called only at a compaction step when (summary + tail) crosses SCALEDOWN_PROXY_COMPACT_THRESHOLD (default 50k). Most turns make zero ScaleDown calls.
  • Replaces native compaction — smaller forwarded payload ⇒ Claude's auto-compact effectively never fires.
  • Reversible — compacted turns are stashed; a marker tells Claude to call the new sd_retrieve tool for the original.
  • Cache-safe & fail-open — summary block is byte-stable between steps; any error forwards the original request unchanged. Response streamed back untouched.
  • Claude Code only — Codex/Cursor installs are unaffected (different wire format), keeping their existing hooks/rules.

Also in this PR

  • setup: opt-in (recommended) claude → dietcode claude alias; removed on dietcode uninstall. No recursion (wrapper spawns the real binary).
  • Status line shows DietCode compaction off when a session is not running through the proxy.
  • Neutralized the broken pre-compact.ts and its false savings accounting.
  • New sd_retrieve MCP tool.

Notes

  • No new dependencies — built-in node:http + global fetch only. The only package.json change is a new dietcode-proxy bin entry.
  • All 121 tests pass; new suites cover src/proxy/{transform,store,server} (progressive behavior, cache-stability, auth passthrough, fail-open).

Deferred to a later release

Classification-driven tool routing in the proxy (the current UserPromptSubmit intent hint is dead code — {prompt} isn't a valid hook output). Plan captured; not in this PR.

🤖 Generated with Claude Code

Hooks can only append to the context window, so they cannot reduce tokens or
replace Claude's compaction (PreCompact can't override the summary —
anthropics/claude-code#24965). The old pre-compact hook was built on a
non-existent contract and never actually ran.

Add a local proxy (`dietcode claude` / `dietcode proxy`) that sits in the
request path and rewrites the outgoing /v1/messages payload via progressive
compaction: a per-session running ScaleDown summary stands in for older turns,
extended only at occasional compaction steps (most turns make zero ScaleDown
calls). This genuinely shrinks context and keeps Claude's native auto-compaction
from firing. Response is streamed back untouched; every path is fail-open.

- src/proxy/{transform,server,store}.ts — progressive compaction, HTTP
  forwarder, session state + reversibility cache (cache-safe, fail-open)
- bin/proxy.ts — `dietcode proxy` (foreground) + `dietcode claude` wrapper
- src/tools/retrieve.ts — sd_retrieve MCP tool to pull back compacted originals
- setup: opt-in `claude`→`dietcode claude` alias (recommended), removed on uninstall
- status line: shows "DietCode compaction off" when not running via the proxy
- neutralize the broken pre-compact hook and its false savings accounting
- proxy compaction threshold defaults to 50k tokens
- Claude Code only; Codex/Cursor installs unaffected
- no new dependencies (built-in node:http + global fetch)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: DietCode <ai@scaledown.ai>
@varchanaiyer
varchanaiyer merged commit 31c933f into main Jun 28, 2026
3 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants