Run Rayon jobs through its single-threaded fallback wake hook - #3
Open
guybedford wants to merge 2 commits into
Open
guybedford wants to merge 2 commits into
guybedford wants to merge 2 commits into
Conversation
guybedford
force-pushed
the
rayon-fallback-hook
branch
from
September 26, 2026 01:17
fc38cad to
ffcc482
Compare
The tag moved to 7227f2d7: hosted event loops are driven as a microtask from host callbacks, wakes are coalesced only within a drive, and emscripten_epoll_remove_listener receives its userdata.
rayon-rs/rayon#1323 lets a host without threads learn when a job has been queued on the global fallback pool. The object installs the hook with a driver that runs one `rayon::yield_now()` per turn of the event loop until the queue is idle (a wake fires once per idle-to-pending transition, so one yield per wake would leave jobs queued behind it), so every `rayon::spawn` in Pumpkin runs as its own turn, and the Pumpkin patch drops the `rayon_spawn` shim and its call-site rewrites. Chunk generation keeps its Tokio task on single-threaded builds, as upstream runs it on a private pool.
guybedford
force-pushed
the
rayon-fallback-hook
branch
from
September 26, 2026 02:03
ffcc482 to
bcb30a8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Integrates the latest Rayon PR feedback on rayon-rs/rayon#1323 implementing a new
set_fallback_wake_hookso Pumpkin's stockrayon::spawnworks on the single-threaded target.Without threads Rayon's global pool is the current-thread fallback, and a spawned job only runs when the thread next yields to Rayon; the Pumpkin patch worked around that by rewriting nine
rayon::spawncall sites to apumpkin_util::rayon_spawnshim. With the hook the object drives the queue from the event loop, so each job takes a turn of its own, and those rewrites and the shim go away (patch -114 lines).rayon/rayon-corepinned toguybedford/rayonfallback-spawnin[patch.crates-io]until #1323 releasesMinecraftWorld::newinstalls the hook with a driver: onerayon::yield_now()perspawn_localturn, rescheduling itself while a job ran. A wake fires once per idle-to-pending transition, so a single yield per wake leaves jobs queued behind it stranded until the next transition (chunk encoding stalled that way); the driver runs untilIdlepumpkin-emscripten.patch:rayon_spawnremoved, call sites back torayon::spawn; chunk generation keeps its Tokio task on single-threaded builds since upstream runs it on a privateThreadPoolBuilderpool the fallback does not covernpm testpasses (PUMPKIN-DO-SQLITE-RESTART-OK); native scheduler and chunk-queue tests pass.