[initial-d]

Factor screening currently pools stock-days into one correlation and applies limit masking after factor computation. This corrects screening to daily pairwise IC/RankIC with mask-first factors, and adds streamed factor consumption plus an opt-in stable rolling backend for memory-sensitive research.

Long trailing mean/std/cov/corr calculations allocate work proportional to the window length. This adds opt-in O(T*N) FP64 block moments and connects them to the actual factor APIs and feature CLI, while preserving the default unfold behavior.

compute_legacy_set(..., rolling_backend="linear"), compute_alpha_set(...), and mlquant features --rolling-backend linear now use the selected implementation through a thread/async-local scope. Config selection also works in demo; features.pt records the backend. Existing imported aliases inside registered factors route correctly, and exceptions restore the caller's selection.

During full-engine testing, the original global-prefix approach lost small variances after large past outliers. The final algorithm combines disjoint block prefixes/suffixes, each centred at an observation included in that piece. Independent FP64 oracle regressions cover this failure, masks, causality, gradients and block boundaries. The inaccurate pre-fix runs remain archived.

Correct daily factor screening

daily_factor_ic(values, targets, factor_mask, label_mask) computes [T] Pearson IC and average-tie Spearman RankIC across assets. Nonfinite pairs are excluded before ranks; insufficient/constant cross-sections are missing dates instead of zero IC. Signal availability stays separate from retrospective label eligibility.

scripts/eval_factor_ic.py now streams all 213 factors, with limit masking before factor construction, adjacent finite positive raw-close labels and per-date series/pair counts. Summaries weight eligible dates equally. A regression demonstrates daily IC -1 becoming pooled IC >+0.99 when levels change across dates; independent SciPy tests cover ties, masks and nonfinite inputs.

The complete 500 x 200 synthetic run completed at source 410f6a6, with 431–498 eligible dates per factor. This is full-sample diagnostic screening; no factor selection/holdout, costs, temporal-dependence correction or alpha claim. Complete series, command, environment, input hashes and checks.

Stream factors without materializing the feature cube

iter_legacy_factors(panel, names=None, neutralize=True, rolling_backend="unfold") yields (name, values, mask) in the same order as compute_legacy_set. Backend scopes end before yielding and the previous result is released on resumption. Freeze the panel/registry, avoid mutating masks, and discard outputs/autograd graphs to obtain the memory benefit.

Apple M5 CPU, code 7b90abf, synthetic 750 x 1000, all 213 neutralized factors followed by per-factor mean/RMS reductions, 1/1 threads, 1 warmup / 3 repeats, fresh subprocesses:

• Stack: 5.314 s / 2649.0 MB peak RSS; consume/discard: 4.450 s / 1391.2 MB.
• 47.5% lower process RSS and 16.3% shorter elapsed time for this screening workload.
• All 213 factor byte hashes and the joint mask agree. Retaining every output removes the streaming benefit; a trainer requiring all feature dimensions still needs materialization.
• Raw report, exact command, environment and local checks. This is synthetic engineering evidence, not model-training throughput or alpha.

Opt-in rolling backend: measured behavior and limits

Apple M5 CPU, 1/1 threads, 2 warmups / 5 repeats, seeded synthetic input, code 2620ac1. Fresh process per backend/case; peak RSS includes the interpreter, output tensors and allocator caches.

• 252-day correlation kernel: 7.44x faster, RSS 2284.1 → 416.4 MB. Short 20-day kernels were slower.
• All 213 factors on 750 x 1000: 4.706 → 6.818 seconds, RSS 2648.4 → 2100.4 MB. This is 20.7% lower RSS with 45% longer runtime, not a full-engine acceleration.
• All individual factor masks agreed and outputs were finite. FP64 reductions, finite filtering and correlation clamping change nonlinear ranks: 41 factors had a maximum valid difference above 1e-5; old_071 sampled mean Spearman was 0.9770. Switching backend changes numerical semantics and requires research validation.
• No market-data, predictive-quality or CUDA-throughput claim. MPS lacks FP64 and is rejected; extreme within-window dynamic ranges still need a direct FP64 reference.

Complete raw reports, commands/environment, per-factor diagnostics and preserved failure. The October 24 17.57x global-prefix result is preserved as historical evidence and does not describe the final repaired implementation.

Validation

• python -m pytest: 240 passed, including full-213 streaming equivalence for both backends and neutralization modes, lazy/release/context checks, both complete demo backends and thread/async isolation.
• python -m ruff check src tests scripts/benchmark_factor_backends.py scripts/benchmark_rolling_moments.py scripts/benchmark_factor_streaming.py scripts/eval_factor_ic.py: passed.
• Fresh-process full-engine runs at 500 x 200 and 750 x 1000; separate 20/60/252 kernel sweep with independent sampled FP64 window oracles.

GitHub CI passed Python 3.9 / 3.10 / 3.11 at updated head a20717bb5d8e31c94fd1dbdc0feb05c2a7ad9620: run. Independent maintainer review is required before merge.