Skip to content

Fix unseeded shuffle and train/test split entropy (#9097) - #9108

Open
zhengdaodeguL wants to merge 1 commit into
huggingface:mainfrom
zhengdaodeguL:fix/shuffle-seed-legacy-np-random-9097
Open

zhengdaodeguL wants to merge 1 commit into
huggingface:mainfrom
zhengdaodeguL:fix/shuffle-seed-legacy-np-random-9097

Conversation

@zhengdaodeguL

Copy link
Copy Markdown

Fixes #9097

Dataset.shuffle() and Dataset.train_test_split() documented that a None seed pulls "fresh, unpredictable entropy from the OS", but both drew the seed from the legacy global np.random state instead — and consumed one step of it. Anyone who called np.random.seed(42) earlier got a fully deterministic "unseeded" shuffle, and calling shuffle() silently advanced all subsequent np.random.* draws in the user's program.

The fix removes the legacy-state derivation in both call sites and lets np.random.default_rng(seed) handle None directly (NumPy draws OS entropy for None). The same pattern in fingerprint.py (cache fingerprints for randomized transforms) is fixed the same way, so unseeded disk-backed operations don't reuse previous results via identical fingerprints. Seeded behavior is unchanged.

Thanks to PCTEJA for reporting the issue with a precise root-cause analysis.

Verification

  • New tests in tests/test_arrow_dataset.py: unseeded shuffle/split differ across runs even under the same global seed, calling shuffle()/train_test_split() does not advance the global np.random state, and seeded runs stay reproducible (in-memory and on-disk). All 10 new cases pass; the 8 unseeded/global-RNG cases fail on base (verified with a source-only base control).
  • Targeted suites: 42 passed, 9 skipped; ruff lint/format clean.
  • Note: the old test asserting that two unseeded shuffles under temp_seed(42) produce the same order encoded the buggy behavior; it is replaced by the corrected expectation.

Let default_rng(None) draw fresh entropy instead of deriving and consuming
NumPy's legacy global state. Use fresh entropy for the corresponding
cache fingerprints so unseeded disk-backed operations do not reuse
previous results.

Cover shuffle and train_test_split in memory and on disk, including
global RNG preservation and seeded reproducibility.

Reported by PCTEJA: huggingface#9097
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dataset.shuffle() / train_test_split() with seed=None are deterministic given the global NumPy seed and mutate global RNG state, contradicting the docs

1 participant