Deforkify event loop - #2
Draft
guybedford wants to merge 5 commits into
Draft
guybedford wants to merge 5 commits into
guybedford wants to merge 5 commits into
Conversation
Drop the wasm-streams checkout and cdylib patch in favor of guybedford/wasm-streams (MattiasBuelens/wasm-streams PR pending). The socket2 git rev was exactly the v0.6.5 release.
One perpetual `#[wasm_bindgen(jspi)]` export builds a current-thread runtime and `block_on`s the whole server lifetime; every park suspends the Wasm stack on `epoll_wait`, so the hosted runtime adapter is gone. Pumpkin binds its stock `TcpListener` on 25565 inside the Durable Object's port table, and the object routes each inbound socket to it with `handleAsNodeConnection`, replacing the injected-stream entry point, wasm-streams, and the workers-rs dependency. `stop` cancels the server and the run promise settling is the checkpoint signal. `-sREENTRANT_JSPI` gives each activation its own shadow stack, so other entries into the module while the server is suspended cannot clobber its frames. Toolchain: Rust beta; emscripten main plus the JSPI hooks, reentrant JSPI and epoll listener PRs, with the paired emscripten-releases LLVM and the jspi-hooks Binaryen branch built by setup; wasm-bindgen 0.2.128 CLI via `-sWASM_BINDGEN`; exnref exception handling throughout; tokio `emscripten-epoll`, mio tokio-rs/mio#1969, libc `libc-0.2`. rustc needs a larger compile-thread stack for pumpkin-data. The wasm-bindgen and workers-rs patches and the CLI lockfile are gone; the Pumpkin patch drops the injected-stream entry point. Requires a workerd with per-Durable-Object port tables and `net.Server` inbound routing (`MINIFLARE_WORKERD_PATH`). CI moves to Linux.
The whole Worker is now Rust linked directly by cargo and emcc, with no JSPI and no JS-side driver. The Durable Object owns an EventLoopRuntime (tokio-rs/tokio#8479): Tokio's current-thread scheduler and drivers wait on the host event loop through Emscripten's epoll listeners (emscripten-core/emscripten#27547). Exports return promises; the server lifetime is a scheduled root that completes by callback after the final save and checkpoint. Connections arriving while the server starts or stops wait for the next phase change and retry. worker/index.mjs re-exports the generated module and derives the object class from DurableObject for RPC. The worker and worker-build dependencies, the TypeScript worker, and the filesystem fixtures are removed. The Pumpkin patch makes the stop token replaceable behind a StopSignal and adds reset_stop so a server can start again in the same instance. Under NODERAWFS the host enforces permissions; src/workerd.js forces FS.ignorePermissions since workerd reports directories without execute bits, and keeps stdio on the console callbacks. Documentation describes the event-loop model and the workerd node:fs requirements (positional buffer I/O, O_TRUNC, O_CREAT, rename over an existing path).
The Worker now uses workers-rs (cloudflare/workers-rs#1061): the `worker` crate for the fetch handler, Env and Durable Object plumbing, and `worker-build --emscripten --tokio` to link the bin through emcc, wrap the exports into the entrypoint and DurableObject-derived class, and emit the module Wrangler serves. The JS shim, the factory link arguments in .cargo/config.toml and the wasm-bindgen CLI plumbing in the scripts go away; build.rs keeps only the application's own settings. The Durable Object no longer owns a runtime. `connect` is a `#[wasm_bindgen(tokio)]` export (wasm-bindgen/wasm-bindgen#5334) whose future runs on the thread's Tokio `LocalEventLoop` (tokio-rs/tokio#8484), the host event loop driving the scheduler. The server lifetime is spawned as a task so a panic arrives as a JoinError and fails the object. Pins move to the event-loop stack: tokio `emscripten-event-loop-host`, mio `emscripten`, workers-rs `gbedford/worker-build-emscripten` with its wasm-bindgen submodule supplying every wasm-bindgen crate, and emscripten `cf-final` with the emsdk release its main is paired with. Binaryen comes from that emsdk, so the separate Binaryen checkout and build are dropped.
The world is again a live SQLite-backed filesystem instead of a tree copied to and from storage at start and stop: the object constructor mounts durable-object-fs at /data through worker-fs-mount, and src/workerd.js rebinds Emscripten's NODERAWFS to that node:fs implementation, so every Pumpkin write is durable as it happens. Relative paths are resolved against Emscripten's own working directory, which is the object's rather than the isolate's. The restore/checkpoint code and the files table go away; status reports the last save time. The stdio override is removed: workerd's node:fs supports the process's stdout and stderr descriptors, and Wrangler relays them as plain stdout and stderr lines rather than console errors. Wrangler's build command runs scripts/build.sh so it resolves the provisioned worker-build regardless of PATH.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Same as #1, but updates the patchset to the event loop runtime.