Repository navigation
feat(cache): add HiCache damped-Hermite calibrator - #1359
Merged
Merged
Conversation
HiCache (arXiv 2508.16984) keeps TaylorSeer's finite-difference ladder and replaces only the extrapolation basis with H~_k(x) = sigma^k * H_k(sigma * x), where H is the physicist Hermite polynomial. The ladder stays single-copy: TaylorSeerState gains a _basis_term hook and hicache.py overrides only that, so a missing override degrades loudly (tests pin the concrete state type, the wired sigma and a non-equal forecast against TaylorSeer). - new HiCacheCalibratorConfig (hicache_order <= 4, hicache_sigma in (0, 1]), factory alias, five-way mutual exclusion in load_configs, and --calibrator hicache --calibrator-params order=,sigma= on the CLI - docs: CACHE_API chapter plus comparison table, EXAMPLES help snapshot - tests: hand-computed Hermite table (k=0..4), sigma=1 undamped control, sigma -> 0 anchor collapse, bit-exact max_cached_steps=0 run, pipeline wiring - FLUX.1-dev 1024^2/28 steps/rdt 0.12: 12 cached steps at identical positions in every row; HiCache o2 sigma=0.5 reaches 33.33 dB/0.9781 vs reuse 31.72/0.9706 and TaylorSeer o2 30.91/0.9733 at the same 9.88 s
cache_dit.refresh_context(transformer, **shorthand) is the only reset=True caller: it rebuilds
the config through CalibratorConfig.empty(), which nulls every non-constant field including
calibrator_kwargs, and the rebuilt context then calls Calibrator(config) -> to_kwargs().
TaylorSeer/DMD/SVD still did self.calibrator_kwargs.copy() there, so refreshing a cache context
into one of those three calibrators raised AttributeError: 'NoneType' object has no attribute
'copy' (FoCa already returned a literal {} and HiCache was written with dict(... or {})).
- base CalibratorConfig.to_kwargs() and the TaylorSeer/DMD/SVD overrides now use
dict(self.calibrator_kwargs or {}); the result is still a fresh shallow copy, so the in-place
field writes in the overrides stay safe
- tests/api/test_calibrator_reset.py: five reset=True factory cases, the base config, explicit
params surviving the reset, and the public refresh_context() path starting from a
calibrator-free cache (the branch that replaces calibrator_config wholesale); reverting any of
the four lines fails 4 of those tests
This was referenced Nov 18, 2026
DefTruth
added a commit
that referenced
this pull request
Nov 18, 2026
) Judge the case as implemented rather than "worth integrating": HiCache ships as calibrators/hicache.py (#1359), the landing point moved from "a basis switch on Taylor" to the _basis_term hook plus a separate calibrator, and the same-harness numbers now exist (FLUX 1.7x: 33.33 dB vs reuse 31.72 and TaylorSeer-o2 30.91, sigma=1 collapses to 24.99 which reproduces the paper's sigma ablation). Also record two things the next integration should not rediscover: - a new calibrator's to_kwargs() must tolerate a null calibrator_kwargs, because refresh_context() rebuilds through empty(); TaylorSeer/DMD/SVD crashed there until #1359; - RandPipeline feeds the same hidden_states to every step, so a predictor's forecast equals the anchor and "the output differs" cannot prove the wiring. landscape.md gains the measured table (4.5): hit positions are identical across rows, and a mask-pinned pair keeps the ordering, so the gap is the basis and not the schedule.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1357
Closes #1358
What this adds
HiCache (arXiv 2508.16984) as a new forecast calibrator. It is mathematically isomorphic to
TaylorSeerCalibrator— the finite-difference ladder is identical — and replaces only the extrapolation basis:with$H_k$ the physicist Hermite polynomial. Each order is damped by $\sigma^2$ (the first-order gain is $2\sigma^2$ instead of Taylor's $1$ ), so the expansion stays bounded over the long skip intervals where a raw polynomial diverges.
The ladder stays single-copy:
taylorseer.pygains a_basis_term(i, elapsed)hook andhicache.pyoverrides only that. The TaylorSeer path evaluates the same expression as before (bit-identical), and a missing override degrades loudly rather than silently — the tests pin the concrete state type, the wiredsigma, and a non-equal forecast against TaylorSeer.Changes
caching/cache_contexts/calibrators/hicache.pyHiCacheState(TaylorSeerState)+HiCacheCalibrator(TaylorSeerCalibrator)+validate_hicache_params(order <= 4,sigma in (0, 1])calibrators/taylorseer.py_basis_term()hook,approximate()calls it; ladder untouchedcalibrators/__init__.pyHiCacheCalibratorConfig(hicache_order=2, hicache_sigma=0.5)+__post_init__validation +strify()(HC2S0.5) + aliases + factory branchcaching/load_configs.pyenable_hicache/enable_encoder_hicache/hicache_order/hicache_sigma; mutual exclusion widened 4 → 5 in both the plain and thereset=Truebranchcli/utils.py--calibrator hicache --calibrator-params order=,sigma=(no deprecated--hicacheflag, same as SVD)CACHE_API.mdchapter + comparison table,EXAMPLES.mdhelp snapshot,cache_hicache.yamltests/api/test_hicache_config.py,tests/api/test_hicache_calibrator.py,tests/api/test_calibrator_reset.pyPerformance & fidelity
FLUX.1-dev, 1024×1024, 28 steps, prompt "A cat holding a sign that says hello world", seed 0,
residual_diff_threshold=0.12, 12 of 28 steps cached at identical positions in every row (so the schedule is not a confounder); 1× RTX PRO 5000 72GB, torch 2.15.0. Timings are the CLI's post-warmup inference (Warmup Timeprecedes them in the logs). Single prompt: a wiring probe, not a benchmark; mean == min for one image.Top row: base / DBCache reuse / TaylorSeer o2. Bottom row: HiCache o2 σ=0.5 / σ=1.0 / σ=0.5 + compile.
Three readings:
sigma=1is the undamped physicist-Hermite basis (not a degradation of Taylor's) and collapses to 24.99 dB — the same direction as the paper's σ ablation, and the cheapest proof that the basis is wired rather than silently falling back to TaylorSeer.sigma=1/√2makes the first-order term identical to Taylor's and sits in between, as expected.--steps-mask --mask-policy medium, 9 cached steps, schedule independent of the forecast) keeps the ordering (29.36 vs 27.77), so the gap comes from the basis and not from a shifted hit sequence.Repeat runs:
sigma=0.5,sigma=0.4andsigma=1.0each reproduce bit-identical PNGs (sha256); the+ compilerows differ run to run (27.53 / 28.24 dB, 8.66–8.69 s), as with every calibrator undertorch.compile, so that row is reported as a range.Second commit:
refresh_contextreset path (#1358)calibrators/__init__.py: the baseCalibratorConfig.to_kwargs()and the TaylorSeer/DMD/SVD overrides usedself.calibrator_kwargs.copy().cache_dit.refresh_context(transformer, **shorthand)is the onlyreset=Truecaller and rebuilds the config throughempty(), which nullscalibrator_kwargs, so refreshing a cache context into one of those three calibrators raisedAttributeError: 'NoneType' object has no attribute 'copy'. All four now usedict(self.calibrator_kwargs or {})(still a fresh shallow copy), andtests/api/test_calibrator_reset.pycovers the fivereset=Truefactory paths plus the publicrefresh_contextcall starting from a calibrator-free cache. Reverting any of the four lines fails 4 of those tests (verified by stashing the fix).Validation
pytest tests/api -qsubsets: 196 passed (new HiCache suites + TaylorSeer/SVD/DMD/FoCa config +load_configs+refresh_context+forward_pattern).k = 0..4, three sigmas, three points each — comparing against the recurrence would pass even if both were wrong),sigma=1undamped control,sigma → 0anchor collapse,sigma=1/√2first-order equivalence with Taylor,order=0anchor reuse,max_cached_steps=0bit-exact run, and a monkeypatchedapproximate()counter proving the pipeline really queries the forecast.pre-commit run --all-files: green (yapf, docformatter, flake8 incl. F401, pydocstyle, end-of-file).csrc/,setup.py,.github/workflows/and every sibling repo untouched.Compliance
main(ed926eee), 0 commits behind,git merge-treeno conflictcached_stepsevidence[8,9,11,12,14,15,16,18,19,21,23,25], identical across rowspr-assetsbranch only)