ENH: reproducible Monte Carlo via per-simulation-index seeding#1054
Draft
thc1006 wants to merge 1 commit into
Draft
ENH: reproducible Monte Carlo via per-simulation-index seeding#1054thc1006 wants to merge 1 commit into
thc1006 wants to merge 1 commit into
Conversation
MonteCarlo seeded the stochastic models per worker in parallel mode (from a fresh, unseeded SeedSequence) and once at construction in serial mode, so the sampled inputs depended on the execution mode and the worker count, and parallel runs were not reproducible run to run. Add a keyword-only random_seed to simulate() (SPEC 7 style: accepts an int, a SeedSequence, or a Generator; None keeps the previous fresh-entropy behavior). Spawn one child seed per simulation index from that root and reseed the stochastic models from child_seeds[i] before simulation i. SeedSequence.spawn is prefix-stable, so index i maps to the same seed regardless of which worker runs it, making the inputs identical across serial, parallel(2) and parallel(N). Each index seed is split three ways so the environment, rocket and flight draw from independent streams rather than sharing one. The serial index field now counts from 0 to match the parallel path. Both changes alter the numbers a fixed seed produces, so stored baselines regenerate. Adds tests/unit/simulation/test_monte_carlo_determinism.py: serial reproducibility, worker invariance (serial == parallel(2) == parallel(4)), and the None-seed path. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull request type
Draft / RFC. I'm opening this to make the design in #1053 concrete and get your call on the open questions below before polishing it.
Current behavior
MonteCarlo.simulate()seeds the stochastic models per worker in parallel mode (from a fresh, unseedednp.random.SeedSequence().spawn(n_workers)) and once at construction in serial mode. So the sampled inputs depend on the execution mode and the number of workers, and parallel runs aren't reproducible run to run. This is #1053.New behavior
Adds a keyword-only
random_seedtosimulate(). From that root I spawn one child seed per simulation index and reseed the models fromchild_seeds[i]before simulationi.SeedSequence.spawnis prefix-stable, so indeximaps to the same seed regardless of which worker runs it. The sampled inputs come out identical across serial, parallel(2) and parallel(N), and reproducible from the seed.The design follows Scientific-Python SPEC 7 (a per-call keyword-only seed, normalized to accept an int, a
SeedSequence, or aGenerator, withNone= fresh entropy so existing behavior is unchanged unless you pass a seed) and NumPy's documented parallel idiom (SeedSequence(root).spawn(n)thendefault_rng(child)).tests/unit/simulation/test_monte_carlo_determinism.pycovers it: serial reproducibility, serial == parallel(2) == parallel(4), and the None-seed path.Flightis stubbed, so it's aslowinput-sampling test.Open questions (why this is a draft)
random_seedfor continuity with theStochastic*seed. SPEC 7's direction isrng(SciPy 1.15 renamedseedtorng; sklearn is weighing it in #29315). I can switch torngif you'd rather be forward-looking.simulate()(per run) versus also a stored default onMonteCarlo.__init__(hybrid). I went withsimulate()only, the minimal SPEC-7 form.None= fresh entropy (backward-compatible, not reproducible). A competition or regression context might want a reproducible default instead.random.choice(an unseeded global), so a model with a multi-elementthrust_sourceisn't fully seed-controlled yet. The test uses a single-thrust_sourcerocket to sidestep it; a full fix would seed the stdlibrandomper index too.Breaking change
The exact numbers a seed produces change (per-index seeding, the env/rocket/flight decorrelation, and the serial index now counting from 0 to match parallel), so stored Monte Carlo baselines such as
test_monte_carlo_simulateregenerate. There's no API break for users who don't pin exact samples, andrandom_seedis opt-in.One more heads-up: Python 3.14 flipped the Linux multiprocessing default from fork to forkserver, so workers no longer inherit memory and the seed objects crossing the process boundary must be picklable.
SeedSequenceis, and the test passes under fork.Closes #1053