Skip to content

fix(generic): return NaN from expanding_mean/std when minp exceeds the length (engine parity) - #873

Merged
polakowo merged 1 commit into
polakowo:masterfrom
Cashubski:fix/expanding-minp-parity
Sep 25, 2026
Merged

polakowo merged 1 commit into
polakowo:masterfrom
Cashubski:fix/expanding-minp-parity

Conversation

@Cashubski

Copy link
Copy Markdown
Contributor

Summary

expanding_mean and expanding_std behave differently across engines when minp exceeds the number of rows:

import numpy as np, vectorbt as vbt
from vectorbt.generic import dispatch
a = np.array([[1.0], [2.0]])
dispatch.expanding_mean(a, 3, engine="rust")   # [[nan], [nan]]  (pandas: expanding(min_periods=3).mean() -> NaN, NaN)
dispatch.expanding_mean(a, 3, engine="numba")  # ValueError: minp must be <= window

The Numba implementations delegate to the rolling functions with window = len(a), so the minp <= window validation that is right for a fixed window rejects an expanding window that has simply not grown to minp yet. pandas returns NaN for those rows, and so does the Rust engine. The discrepancy shows up in practice with short columns (e.g. the first chunks of a walk-forward split) when engine="auto" picks a different engine depending on the input.

Changes

  • expanding_mean_1d_nb, expanding_mean_nb, expanding_std_1d_nb and expanding_std_nb use window = max(len(a), minp). A rolling window of at least the array length is an expanding window, so results are unchanged whenever minp <= len(a), and minp > len(a) now yields NaN rows like pandas and the Rust engine. expanding_min/expanding_max already behaved correctly.
  • Test test_expanding_minp_above_length, parametrized over both engines (Rust skipped when not installed), checks Series and DataFrame inputs against pandas.

Found with a small parity harness that runs every generic and returns dispatch function through both engines on clean, NaN-padded, constant and short inputs; this was the only difference that surfaced.

…e length

expanding_mean_nb and expanding_std_nb delegate to the rolling functions
with window = len(a), so a minp larger than the number of rows raised
'minp must be <= window' in the Numba engine, while the Rust engine and
pandas' expanding(min_periods=minp) return NaN. Use window = max(len(a),
minp) so both engines agree with pandas.
@polakowo
polakowo merged commit 72c21fa into polakowo:master Sep 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants