Internal ensembling in Aurora models - #199
Open
Agnieszka Słowik (Slowika) wants to merge 4 commits into
Open
Conversation
Agnieszka Słowik (Slowika)
marked this pull request as ready for review
August 13, 2026 12:11
…for backward compatibility.
Agnieszka Słowik (Slowika)
force-pushed
the
agaslowik/ensemble-members-internal-to-aurora
branch
from
August 13, 2026 14:48
118564d to
f0fa368
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addressed Issue #192.
Developed with the aid of AI, in line with the Code of Conduct.
Jonathan Weyn (@jweyn) Wessel (@wesselb)
Problem
Running an
N-member ensemble currently requires a loop that callsAurora.forward()/rollout()once per member, and then manually combining the results. This under-utilises the GPU (Nseparate launches) when the GPU is capable of storing all of the ensemble state.Change
Add a
num_ensemble_membersconstructor argument toAurora(default1, fully backwards compatible). When set toN > 1,forward()/rollout()run allNmembers as a single, fused batched computation internally, rather thanNseparate calls. This is useful when combined withstochastic=True: since the backbone's existing per-batch-element noise injection means every instance receives independent noise.This is purely an additional option: looping over
forward()/rollout()to implement ensembling remains fully supported and unaffected.Design notes
Batch's public shape contract is untouched: no new dimension, no new methods. The batch-dimension tiling used to fuse the computation is a private implementation detail (_tile_batch/_split_batchinaurora/batch.py), never exposed onBatchitself.forward()'s return type is nowBatch | list[Batch]: a plainBatchwhennum_ensemble_members == 1(no change from today), or alist[Batch]ofNstandard-shaped batches:pred[m]is memberm's ordinary, individually inspectableBatch.rollout()follows the same contract per yielded step, keeping the tiled representation internal across autoregressive steps for efficiency, and temporarily forcingmodel.num_ensemble_members = 1during its loop (restored viatry/finally, even on early generator closure) so nestedforward()calls don't re-tile.num_ensemble_members > 1is requested on a non-stochasticmodel, since all members would then be identical.Tests
Added
tests/v1p5/test_ensemble.pycovering:_tile_batch/_split_batchround-trippingforward()'s single-Batchvs.list[Batch]return contractstochastic=Truevs. identity understochastic=Falserollout()'s per-step output shape plusnum_ensemble_membersrestoration (including on early.close()).