Skip to content

feat(cache): add HiCache damped-Hermite calibrator - #1359

Merged
DefTruth merged 2 commits into
mainfrom
team/hicache/lead
Nov 18, 2026
Merged

DefTruth merged 2 commits into
mainfrom
team/hicache/lead

Conversation

@DefTruth

@DefTruth DefTruth commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Closes #1357
Closes #1358

What this adds

HiCache (arXiv 2508.16984) as a new forecast calibrator. It is mathematically isomorphic to TaylorSeerCalibrator — the finite-difference ladder is identical — and replaces only the extrapolation basis:

$$\hat{Y}(t) = Y(0) + \sum_{k=1}^{N} \frac{1}{k!},\Delta^k Y \cdot \tilde{H}_k(t), \qquad \tilde{H}_k(t) = \sigma^k H_k(\sigma t)$$

with $H_k$ the physicist Hermite polynomial. Each order is damped by $\sigma^2$ (the first-order gain is $2\sigma^2$ instead of Taylor's $1$), so the expansion stays bounded over the long skip intervals where a raw polynomial diverges.

The ladder stays single-copy: taylorseer.py gains a _basis_term(i, elapsed) hook and hicache.py overrides only that. The TaylorSeer path evaluates the same expression as before (bit-identical), and a missing override degrades loudly rather than silently — the tests pin the concrete state type, the wired sigma, and a non-equal forecast against TaylorSeer.

from cache_dit import DBCacheConfig, HiCacheCalibratorConfig

cache_dit.enable_cache(
  pipe,
  cache_config=DBCacheConfig(
    max_warmup_steps=8, max_cached_steps=-1,
    Fn_compute_blocks=8, Bn_compute_blocks=0,
    residual_diff_threshold=0.12,
  ),
  calibrator_config=HiCacheCalibratorConfig(hicache_order=2, hicache_sigma=0.5),
)
python3 -m cache_dit.generate flux --cache --calibrator hicache \
  --calibrator-params order=2,sigma=0.5

Changes

Area Change
caching/cache_contexts/calibrators/hicache.py new: HiCacheState(TaylorSeerState) + HiCacheCalibrator(TaylorSeerCalibrator) + validate_hicache_params (order <= 4, sigma in (0, 1])
calibrators/taylorseer.py extract the _basis_term() hook, approximate() calls it; ladder untouched
calibrators/__init__.py HiCacheCalibratorConfig(hicache_order=2, hicache_sigma=0.5) + __post_init__ validation + strify() (HC2S0.5) + aliases + factory branch
caching/load_configs.py enable_hicache / enable_encoder_hicache / hicache_order / hicache_sigma; mutual exclusion widened 4 → 5 in both the plain and the reset=True branch
cli/utils.py --calibrator hicache --calibrator-params order=,sigma= (no deprecated --hicache flag, same as SVD)
exports / docs / examples three-layer export, CACHE_API.md chapter + comparison table, EXAMPLES.md help snapshot, cache_hicache.yaml
tests tests/api/test_hicache_config.py, tests/api/test_hicache_calibrator.py, tests/api/test_calibrator_reset.py

Performance & fidelity

FLUX.1-dev, 1024×1024, 28 steps, prompt "A cat holding a sign that says hello world", seed 0, residual_diff_threshold=0.12, 12 of 28 steps cached at identical positions in every row (so the schedule is not a confounder); 1× RTX PRO 5000 72GB, torch 2.15.0. Timings are the CLI's post-warmup inference (Warmup Time precedes them in the logs). Single prompt: a wiring probe, not a benchmark; mean == min for one image.

Config Latency Speedup PSNR (dB) SSIM
base (no cache) 16.91s 1.00× inf 1.0000
DBCache reuse 9.88s 1.71× 31.72 0.9706
+ TaylorSeer o2 9.88s 1.71× 30.91 0.9733
+ HiCache o2 σ=0.4 9.89s 1.71× 33.10 0.9765
+ HiCache o2 σ=0.5 9.88s 1.71× 33.33 0.9781
+ HiCache o1 σ=0.5 9.91s 1.71× 33.36 0.9779
+ HiCache o2 σ=1/√2 9.88s 1.71× 32.44 0.9770
+ HiCache o2 σ=1.0 9.89s 1.71× 24.99 0.9280
+ HiCache o2 σ=0.5 + compile 8.66–8.69s 1.95× 27.53 / 28.24 0.9639 / 0.9668
+ TaylorSeer o2 (mask-pinned) 11.67s 1.45× 27.77 0.9623
+ HiCache o2 σ=0.5 (mask-pinned) 11.68s 1.45× 29.36 0.9738

latency | quality

visual comparison

Top row: base / DBCache reuse / TaylorSeer o2. Bottom row: HiCache o2 σ=0.5 / σ=1.0 / σ=0.5 + compile.

Three readings:

  1. The damping is what carries the gain. sigma=1 is the undamped physicist-Hermite basis (not a degradation of Taylor's) and collapses to 24.99 dB — the same direction as the paper's σ ablation, and the cheapest proof that the basis is wired rather than silently falling back to TaylorSeer. sigma=1/√2 makes the first-order term identical to Taylor's and sits in between, as expected.
  2. At this ~1.7× operating point HiCache is the best of the three (+1.61 dB over reuse, +2.42 dB over TaylorSeer-o2) at the same latency, and order 1 is already enough (33.36 vs 33.33).
  3. A mask-pinned pair (--steps-mask --mask-policy medium, 9 cached steps, schedule independent of the forecast) keeps the ordering (29.36 vs 27.77), so the gap comes from the basis and not from a shifted hit sequence.

Repeat runs: sigma=0.5, sigma=0.4 and sigma=1.0 each reproduce bit-identical PNGs (sha256); the + compile rows differ run to run (27.53 / 28.24 dB, 8.66–8.69 s), as with every calibrator under torch.compile, so that row is reported as a range.

Second commit: refresh_context reset path (#1358)

calibrators/__init__.py: the base CalibratorConfig.to_kwargs() and the TaylorSeer/DMD/SVD overrides used self.calibrator_kwargs.copy(). cache_dit.refresh_context(transformer, **shorthand) is the only reset=True caller and rebuilds the config through empty(), which nulls calibrator_kwargs, so refreshing a cache context into one of those three calibrators raised AttributeError: 'NoneType' object has no attribute 'copy'. All four now use dict(self.calibrator_kwargs or {}) (still a fresh shallow copy), and tests/api/test_calibrator_reset.py covers the five reset=True factory paths plus the public refresh_context call starting from a calibrator-free cache. Reverting any of the four lines fails 4 of those tests (verified by stashing the fix).

Validation

  • pytest tests/api -q subsets: 196 passed (new HiCache suites + TaylorSeer/SVD/DMD/FoCa config + load_configs + refresh_context + forward_pattern).
  • Unit coverage for the math: hand-written closed-form Hermite table (k = 0..4, three sigmas, three points each — comparing against the recurrence would pass even if both were wrong), sigma=1 undamped control, sigma → 0 anchor collapse, sigma=1/√2 first-order equivalence with Taylor, order=0 anchor reuse, max_cached_steps=0 bit-exact run, and a monkeypatched approximate() counter proving the pipeline really queries the forecast.
  • pre-commit run --all-files: green (yapf, docformatter, flake8 incl. F401, pydocstyle, end-of-file).
  • Scope: only the 14 files in the diff (+1068/−27); csrc/, setup.py, .github/workflows/ and every sibling repo untouched.

Compliance

Item Status
base = main (ed926eee), 0 commits behind, git merge-tree no conflict ✅
linked issues ✅ #1357 (feature), #1358 (bug)
performance table + dual-panel chart + visual sheet in the body ✅
timings after warmup, fixed seed, mean/min collapsed (single image) ✅
cached_steps evidence ✅ 12/28 at [8,9,11,12,14,15,16,18,19,21,23,25], identical across rows
CI workflow untouched ✅
temp assets kept out of the tree (pr-assets branch only) ✅

HiCache (arXiv 2508.16984) keeps TaylorSeer's finite-difference ladder and
replaces only the extrapolation basis with H~_k(x) = sigma^k * H_k(sigma * x),
where H is the physicist Hermite polynomial. The ladder stays single-copy:
TaylorSeerState gains a _basis_term hook and hicache.py overrides only that,
so a missing override degrades loudly (tests pin the concrete state type, the
wired sigma and a non-equal forecast against TaylorSeer).

- new HiCacheCalibratorConfig (hicache_order <= 4, hicache_sigma in (0, 1]),
  factory alias, five-way mutual exclusion in load_configs, and
  --calibrator hicache --calibrator-params order=,sigma= on the CLI
- docs: CACHE_API chapter plus comparison table, EXAMPLES help snapshot
- tests: hand-computed Hermite table (k=0..4), sigma=1 undamped control,
  sigma -> 0 anchor collapse, bit-exact max_cached_steps=0 run, pipeline wiring
- FLUX.1-dev 1024^2/28 steps/rdt 0.12: 12 cached steps at identical positions
  in every row; HiCache o2 sigma=0.5 reaches 33.33 dB/0.9781 vs reuse
  31.72/0.9706 and TaylorSeer o2 30.91/0.9733 at the same 9.88 s
cache_dit.refresh_context(transformer, **shorthand) is the only reset=True caller: it rebuilds
the config through CalibratorConfig.empty(), which nulls every non-constant field including
calibrator_kwargs, and the rebuilt context then calls Calibrator(config) -> to_kwargs().
TaylorSeer/DMD/SVD still did self.calibrator_kwargs.copy() there, so refreshing a cache context
into one of those three calibrators raised AttributeError: 'NoneType' object has no attribute
'copy' (FoCa already returned a literal {} and HiCache was written with dict(... or {})).

- base CalibratorConfig.to_kwargs() and the TaylorSeer/DMD/SVD overrides now use
  dict(self.calibrator_kwargs or {}); the result is still a fresh shallow copy, so the in-place
  field writes in the overrides stay safe
- tests/api/test_calibrator_reset.py: five reset=True factory cases, the base config, explicit
  params surviving the reset, and the public refresh_context() path starting from a
  calibrator-free cache (the branch that replaces calibrator_config wholesale); reverting any of
  the four lines fails 4 of those tests
@DefTruth
DefTruth merged commit de3044b into main Nov 18, 2026
4 checks passed
@DefTruth
DefTruth deleted the team/hicache/lead branch November 18, 2026 11:39
DefTruth added a commit that referenced this pull request Nov 18, 2026
)

Judge the case as implemented rather than "worth integrating": HiCache ships as
calibrators/hicache.py (#1359), the landing point moved from "a basis switch on
Taylor" to the _basis_term hook plus a separate calibrator, and the same-harness
numbers now exist (FLUX 1.7x: 33.33 dB vs reuse 31.72 and TaylorSeer-o2 30.91,
sigma=1 collapses to 24.99 which reproduces the paper's sigma ablation).

Also record two things the next integration should not rediscover:

- a new calibrator's to_kwargs() must tolerate a null calibrator_kwargs, because
  refresh_context() rebuilds through empty(); TaylorSeer/DMD/SVD crashed there
  until #1359;
- RandPipeline feeds the same hidden_states to every step, so a predictor's
  forecast equals the anchor and "the output differs" cannot prove the wiring.

landscape.md gains the measured table (4.5): hit positions are identical across
rows, and a mask-pinned pair keeps the ordering, so the gap is the basis and not
the schedule.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: calibrator to_kwargs() crashes on the refresh_context(reset=True) path Support HiCache (damped-Hermite forecast calibrator)

1 participant