fix(swa): keep the prompt head's windowed KV so fan-out can reuse a shared prefix

[huhoo]

Fixes #200.

Summary

The finish-time soft pin in _cache_req_swa kept windowed (SWA) KV only for

[prompt_len - sliding_window - _SWA_RETAIN_GAP, prompt_len)

freeing everything below that eagerly. The comment states the intent: a follow-up turn diverges at the prompt end when the client drops reasoning, so only the trailing window needs to stay live. That holds for append. It does not hold for fan-out — many requests sharing a system prompt (and tool schemas), each carrying its own message, diverge a whole user message before the previous prompt's end. Reuse needs windowed KV live at the divergence point, so every such request re-prefilled the shared prefix.

This removes the eager free. The head stays unlocked and un-tombstoned, so ensure_swa_slots → evict_swa still reclaims it under real pressure — the pool reclaims it lazily instead of eagerly. trim_head_swa itself is untouched; only this one caller goes away, so its own unit tests and the radix driver/adapters keep exercising it.

Measured on hardware

8x RTX 4090, DSV4-Flash, one shared 8192-token system prompt, temperature: 0.

pattern	cached	2nd-request wall
append (history + reply + new turn)	8832 / 8916 = 99%	2.68 s
fan-out (same system prompt, new question)	0 / 8274 = 0%	4.58 s

After the change, same test:

pattern	cached	wall
fan-out	8192 / 8274 = 99%	0.77 s
append	8832 / 8915 = 99%	1.30 s
four sessions, one system prompt	99% / 99% / 99% / 99%	~0.8 s each

Concurrently, 8 sessions sharing one 8192-token prompt: 5 reused 99% and 3 reused 0%, and the engine log shows why — the first batch admits them together, before anything is in the tree:

23:35:45  seq=2  new=9170  cached=0        <- admitted together, nothing cached yet
23:35:48  seq=6  cached=40960              <- six requests reuse five copies of the prefix

That same-batch race is expected and left alone: it is structural (match_req only sees what is already inserted, and a request enters the tree at commit). Quantified against this deployment's logs it accounts for ~1% of recomputed tokens as an upper bound — not worth a change, and noted here so it is not mistaken for a regression.

Risk: does keeping the head break the pool?

That is the one thing this change can get wrong, so it is what the tests target. Keeping the head live must not turn "reclaimed under pressure" into "never reclaimed" (issue #202's unhandled RuntimeError).

• test_swa_fanout_stress.py: multi-round fan-out at a roomy pool; fan-out under a pool too small to retain anything (must evict, not raise); mixed fan-out + append; page_size > 1 alignment; bounded chain.
• In the too-small-pool case both the old and new code behave identically (reuse 0, no raise) — i.e. no leak is introduced; the fix only shows up when there is room to retain, which is the normal case.
• On the deployment: window-pool usage peaked at 0.27 after the change (0.15 before), with zero SWA pool exhausted, zero evictions, zero OOM.

Testing

• tests/scheduler/test_swa_fanout_retention.py — real CacheManager on CPU; fails before the fix with reused 0 of 600 tokens and passes after; the append case is asserted alongside so the fix cannot silently trade one for the other.
• Regression, tests/scheduler tests/kvcache tests/engine: 471 passed, 8 skipped; unmodified main at 68a81ff gives 462 passed, 8 skipped. The delta is exactly the 9 new tests. The 2 failures (tests/engine/test_cache_budget.py, RuntimeError: Attention backend 'fi' requires flashinfer) fail identically on unmodified main.
• Both test files pin CUDA_VISIBLE_DEVICES="" with a guard assertion: _maybe_pinned calls pin_memory() whenever torch.cuda.is_available(), so a "CPU-only" test on a serving box allocates real device memory (I hit AcceleratorError: CUDA error: out of memory on a live engine that way).

Note on #200's open question

The issue says it could not fully reconcile the measured cliff (~7 divergent tokens) with _SWA_RETAIN_GAP = 16. The reproduction here is consistent with the mechanism as stated: what matters is whether the divergence point falls inside the retained [prompt_len - window - gap, prompt_len), and with a long tail it falls far below it. Two earlier attempts of mine failed to reproduce for exactly that reason — the first request's tail has to exceed window + gap (144 tokens here) before the fan-out diverges past the retained region; with an 83-token tail the match still landed inside it and reused fine.

Assisted-by: Claude Code