diff --git a/docs/superpowers/plans/2026-09-22-32-bit-phase-4b-async.md b/docs/superpowers/plans/2026-09-22-32-bit-phase-4b-async.md new file mode 100644 index 000000000..fb4ef4c21 --- /dev/null +++ b/docs/superpowers/plans/2026-09-22-32-bit-phase-4b-async.md @@ -0,0 +1,150 @@ +# 32-bit Phase 4b: Async at i686 Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** async/await programs, including `for await`, compile and run correctly as 32-bit executables. + +**Architecture:** Both i686 async failures come from the upstream `ConvertAsyncToLLVMPass` (`3rdParty/llvm-project/mlir/lib/Conversion/AsyncToLLVM/AsyncToLLVM.cpp`), which assumes 64 bits in two places. The pass is part of the prebuilt LLVM, so it can't be patched here. Instead, one small tslang MLIR pass repairs its output for targets whose pointer width is below 64: + +1. **The coroutine frame allocator.** `CoroBeginOpConversion` calls `lookupOrCreateAlignedAllocFn(…, rewriter.getI64Type())` and computes the size with `llvm.coro.size.i64`. That declares `aligned_alloc(i64, i64)`. `GCPass` renames it to `GC_memalign`; without GC, it resolves to the runtime's `aligned_alloc` shim. Both are `(size_t, size_t)`, which is 32-bit at i686. The callee therefore reads `alignment = 4, size = 0`, and the 16-byte frame overflows the tiny block it gets back. + - Evidence: a crash dump of `await g()` at i686. The worker thread faults reading `0x1c` in `mlirAsyncRuntimeEmplaceToken`, which `async_execute_fn` calls with a null token. +2. **Index values crossing into `i64` runtime parameters.** `async.runtime.create_group(%size : index)` and similar operations are lowered with a converter whose index is 64-bit. The runtime really does take `int64_t` there (`lib/AsyncRuntimeCommon.inc`), so `i64` is correct. Our i686 `LLVMTypeConverter` makes `index` an `i32`, which leaves `unrealized_conversion_cast %x : i32 to index` followed by `… : index to i64`. LLVM translation rejects that. This is `00for_await`'s "LLVM Translation failed for operation: builtin.unrealized_conversion_cast". + +**Tech Stack:** C++17, the MLIR LLVM dialect, tslang's MLIR pass pipeline (`tslang/tslang/transform.cpp`), and bash test scripts. + +**Spec:** `docs/superpowers/specs/2026-09-20-32-bit-compilation-design.md`, Phase 4 ("4b — async at i686") and "Open issues". + +## Global Constraints + +- **x64 output is unchanged.** The new pass is added only when `compileOptions.sizeBits() < 64`. Prove it with `tslang/test/compare-ir.sh` against a baseline from this plan's base commit. The only allowed differences are the noise files listed in `tslang/test/mask-ir.sh`. +- **wasm32 has pointers below 64 bits too.** The pass is target-generic (pointer width), not i686-specific. wasm32 async output may change; report it. `check-datalayout.sh` must stay green. +- Hard errors, not silent fallbacks. If the pass finds an `aligned_alloc` whose shape it doesn't expect (other than two integer parameters and a `ptr` result), or a cast chain it can't lower, it emits an error. +- **Suite:** `ctest -j 16 -C Release` stays all-green. **Corpus:** `tslang/test/probe-corpus.sh --compare` against a baseline from the plan's base commit shows no row that passed before and fails after. Two rows are known to flip between runs: `00array8_tuple_spread` and `arrayLiterals2ES5` at x86 under rc. +- **x86 lib flags:** as in `tslang/test/check-x86-run.sh`: `--gc-lib-path=/3rdParty/gc/x64/release/lib`, `--tslang-lib-path=/__build/tslang-runtime/release`, and unset the `*_LIB_PATH` env vars. +- **The async runtime needs Boehm's thread API**, so async programs link only under `-mm=gc`, at x64 as well: under rc and none the link fails with unresolved `GC_allow_register_threads`. Changing that is out of scope. The gate is `-mm=gc`. +- Commits are GPG-signed; never bypass signing. The trailer is `Co-Authored-By: Claude Opus 5 `. + +## Measured starting point (main 9a521c31) + +At i686 under `-mm=gc`: +- `00async_await`, `00async_gc_threading`, `00async_result_types` and `00owned_async` crash with exit 139. +- `00for_await` and `00for_await_yield` fail to compile. + +At x64 all six pass. A minimal repro, `async function g() { print("in g"); } function main() { print("start"); await g(); print("after await"); }`, crashes at i686 and passes at x64. + +## File structure + +| File | Change | Task | +| --- | --- | --- | +| `tslang/test/check-x86-run.sh` | Add the async corpus cases (gc) and an IR check on the frame allocator's width. | 1 | +| `tslang/include/TypeScript/Pass/AsyncTargetWidthPass.h`, `tslang/lib/TypeScript/AsyncTargetWidthPass.cpp` | New MLIR pass that fixes both defects. | 2, 3 | +| `tslang/lib/TypeScript/CMakeLists.txt` | Add the source. | 2 | +| `tslang/tslang/transform.cpp` (~163-178) | Schedule the pass. | 2, 3 | + +--- + +### Task 1: Failing checks + +**Files:** +- Modify: `tslang/test/check-x86-run.sh` + +**Interfaces:** +- Produces: async cases in `check-x86-run.sh`, which Tasks 2 and 3 turn green. + +- [ ] **Step 1: Capture the baselines before any compiler change.** + - Copy the current Release `tslang.exe` to the SDD workspace as `baseline/tslang-base.exe`. It is built from main's sources. + - Run `tslang/test/compare-ir.sh` for `x86_64-pc-windows-msvc` into `baseline/x64`. + - Run `probe-corpus.sh` into `baseline/probe`. + +- [ ] **Step 2: Add the cases.** In `check-x86-run.sh`, add a loop over the corpus files `00async_await`, `00async_gc_threading`, `00async_result_types`, `00owned_async`, `00for_await` and `00for_await_yield` (from `tslang/test/tester/tests`). Build each at `-mtriple=i686-pc-windows-msvc --opt --no-default-lib --entry-point -mm=gc`, check that the PE machine is I386, and run it with a 30 s timeout; it must exit 0. + - Use `--entry-point`: the suite builds with it, and these files define `main` without `export`. Check how the existing cases pass flags and follow them. + - Add the minimal repro above as `tslang/test/x86/await_order.ts`, with its exact expected output. First find out what x64 prints: an `await` inside a non-async `main` may print "after await" before "in g", and that order is the current semantics. Compare against x64 rather than assuming. + +- [ ] **Step 3: IR check.** Emit `--emit=llvm` for the repro at i686, gc. Assert that the frame allocator is declared with pointer-width parameters: `declare ptr @GC_memalign(i32, i32)`, and not `(i64, i64)`. Also emit at `-mm=none` and assert `@aligned_alloc(i32, i32)`. + +- [ ] **Step 4: Run it.** Every new case FAILs, and every existing case still passes. + +- [ ] **Step 5: Commit.** The suite is unaffected, because `check-x86-run.sh` is hand-run. + +--- + +### Task 2: Pointer-width frame allocator + +**Files:** +- Create: `tslang/include/TypeScript/Pass/AsyncTargetWidthPass.h`, `tslang/lib/TypeScript/AsyncTargetWidthPass.cpp` +- Modify: `tslang/lib/TypeScript/CMakeLists.txt`, `tslang/tslang/transform.cpp` + +**Interfaces:** +- Produces: `std::unique_ptr mlir::typescript::createAsyncTargetWidthPass(unsigned pointerBits)`, a module pass. Task 3 extends the same pass. + +- [ ] **Step 1: The pass.** Follow the existing tslang MLIR passes for the declaration and registration style: grep `createGCPass` and its header. Its job: + +```cpp +// Upstream ConvertAsyncToLLVM assumes 64-bit sizes: it allocates coroutine frames with +// aligned_alloc(i64, i64) whatever the target. aligned_alloc takes size_t, which is pointer +// width, so on a 32-bit target the callee reads (alignment, 0) and the frame overflows its +// block. Retype the declaration to the pointer width and truncate the arguments at each call. +``` + + - Find `LLVM::LLVMFuncOp` named `aligned_alloc`. If none exists, do nothing. + - If its parameters are already `i`, do nothing. + - If they are `(i64, i64) -> ptr`, change the function type to `(iP, iP) -> ptr`. For every `LLVM::CallOp` whose callee is `aligned_alloc`, insert `LLVM::TruncOp` on both operands. The values are coroutine frame sizes and alignments, which fit in 32 bits. + - Any other shape is a hard error (`emitError` plus `signalPassFailure`). + +- [ ] **Step 2: Schedule it** in `transform.cpp`, directly after `createConvertAsyncToLLVMPass()`, under the same `#ifdef ENABLE_ASYNC`, and only when `compileOptions.sizeBits() < 64`. That is before `GCPass`, so the rename to `GC_memalign` keeps the corrected signature. + +- [ ] **Step 3: Verify.** + - The Task 1 IR check passes for gc and none. + - `00async_await`, `00async_gc_threading`, `00async_result_types`, `00owned_async` and the repro now run at i686. If one still fails, diagnose it before moving on: compare with x64 and find the crash site. Report what you find; don't widen the pass to cover it without understanding it. + - `00for_await` still fails to compile; that is Task 3. + - `compare-ir.sh` x64: noise only. + - `ctest -j 16 -C Release`: all green. + +- [ ] **Step 4: Commit.** + +--- + +### Task 3: Index values into `i64` runtime parameters + +**Files:** +- Modify: `tslang/lib/TypeScript/AsyncTargetWidthPass.cpp`, `tslang/tslang/transform.cpp` + +**Interfaces:** +- Consumes: Task 2's pass. +- Produces: the same pass gains a second phase. Alternatively, it becomes a second pass factory, `createAsyncIndexCastPass(unsigned pointerBits)`, if it has to run at a different point in the pipeline; decide from where the casts first exist. + +- [ ] **Step 1: Locate the casts.** Emit `--emit=mlir-llvm` for `00for_await` at i686. Today the output holds `unrealized_conversion_cast %x : i32 to index` followed by `… : index to i64`, feeding `llvm.call @mlirAsyncRuntimeCreateGroup(%7) : (i64)`. + - Find which pass leaves this chain. The async pass converts `index` to `i64`, and LowerToLLVM later converts the `index` producer to `i32`. The fix has to run after both, so after `createLowerToLLVMPass` and before the LLVM IR translation. `GCPass` runs there too. + - Also look for the reverse direction, `i64 → index → i32`. For example, `mlirAsyncRuntimGetNumWorkerThreads` returns `index` upstream, and `async.runtime.add_to_group` returns an index. + +- [ ] **Step 2: Fold the chains.** Walk `UnrealizedConversionCastOp`s whose single operand is itself the result of an `UnrealizedConversionCastOp`, where the chain is `iA → index → iB` and both A and B are integer widths. + - Replace the outer cast with `LLVM::SExtOp` when A < B, `LLVM::TruncOp` when A > B, or the original value when A == B. + - Erase the inner cast if it has no uses left. + - Use sign extension because `index` is signed in MLIR's arith semantics. The values involved (group sizes, counts) are non-negative, so sign and zero extension agree. + - Any other `unrealized_conversion_cast` is left alone. LLVM translation will still report it, and it is not in scope. + - Put a comment at the fold explaining why the chain exists: two type converters with different index widths meet here. + +- [ ] **Step 3: Verify.** + - `00for_await` and `00for_await_yield` compile and run at i686 under gc. + - The full `check-x86-run.sh` passes, including Task 1's cases. + - `check-x86-eh.sh` and `check-x86-libs.sh` still pass. + - `compare-ir.sh` x64: noise only. + - `ctest -j 16 -C Release`: all green. + - `probe-corpus.sh`, then `--compare` against `baseline/probe`: no regressions. At x86 gc, the six async files are newly passing. Also report wasm32: run `--emit=llvm` for the six async files with `-mtriple=wasm32-unknown-unknown`, before and after, and report the diff and any compile errors (wasm async may not be supported at all; just record it). + +- [ ] **Step 4: Commit.** + +--- + +## Phase 4b gate + +- [ ] The six async corpus files and `await_order.ts` run correctly as 32-bit exes under gc. +- [ ] The frame allocator is declared with pointer-width parameters at i686, under gc and none. +- [ ] x64 IR is unchanged apart from known noise, and ctest is all green. +- [ ] `probe-corpus.sh --compare`: no regressions at either arch. + +## Rulings made while writing this plan + +- **A repair pass, not a vendored copy of `AsyncToLLVM.cpp`.** The frame allocator width is hard-coded to `i64` upstream, whatever `LowerToLLVMOptions` says. So a copy would need the same edit, and it would also mean carrying about 1000 lines of upstream code. The repair pass is small and states each assumption it relies on. +- **The runtime ABI stays `int64_t`.** `mlirAsyncRuntimeCreateGroup(int64_t)` and similar are correct at every arch; phase 1's spec says so. Only the `index` bridge into them is fixed. +- **Async under rc and none stays out of scope.** It fails to link at x64 as well. diff --git a/docs/superpowers/specs/2026-09-20-32-bit-compilation-design.md b/docs/superpowers/specs/2026-09-20-32-bit-compilation-design.md index 0e07ab109..1c18886d4 100644 --- a/docs/superpowers/specs/2026-09-20-32-bit-compilation-design.md +++ b/docs/superpowers/specs/2026-09-20-32-bit-compilation-design.md @@ -3,7 +3,7 @@ Design for compiling TypeScript to 32-bit targets, with `i686-pc-windows-msvc` as the proof target and width-correctness that generalizes to any 32-bit triple. -Status: approved design. Phases 1-3 and 4a implemented; 4b and 4c not. +Status: approved design. Phases 1-3, 4a and 4b implemented; 4c not. ## Why now @@ -39,7 +39,10 @@ obvious reading of the emitted IR suggests the opposite. `mlirAsyncRuntimeAddRef(ptr, i64)`, `mlirAsyncRuntimeCreateValue(i64)` and similar regardless of triple, which looks like a 64-bit-shaped ABI. It is not: `lib/AsyncRuntimeCommon.inc` declares these parameters as `int64_t`, not -`size_t`. `i64` is correct on both arches. Nothing to change. +`size_t`. `i64` is correct on both arches. Nothing to change in the runtime API. +(Phase 4b found the one real width bug on this path elsewhere: the coroutine +frame allocator, `aligned_alloc`, *is* `size_t`, and upstream declares it +`(i64, i64)`.) **The Win32 EH struct layouts are already correct for x86.** `ThrowInfo` is emitted as `{i32 x 4}` and `CatchableType` as `{i32 x 7}` @@ -254,8 +257,6 @@ They depend on phases 2 and 3 and cannot run before both are in place. Nothing in the probing suggests they are separately broken; if they are, that is a finding of this phase. -Already found: at i686, `00for_await` and `00for_await_yield` fail at -`--emit=llvm` with an `unrealized_conversion_cast` left in the IR; x64 passes. **Gate.** The same suite that runs at 64 bits runs at 32 bits, green. Genuine failures are triaged and recorded the way the Debug-suite failures are, not @@ -275,12 +276,23 @@ only, so phase 4 is three PRs: i686), and a module-level union initialized with a constant now gets a global constructor instead of failing to compile. 29 probe rows newly pass, none regress; x64 `.text` is unchanged within 0.03%. -- **4b — async at i686.** +- **4b — async at i686 (done).** Upstream `ConvertAsyncToLLVM` assumes 64 bits + twice. It declares the coroutine frame allocator `aligned_alloc(i64, i64)` + whatever the target; at i686 the callee reads `size = 0` and the frame + overflows its block (the crash: `mlirAsyncRuntimeEmplaceToken(null)` on a + worker thread). And its 64-bit-index converter meets tslang's 32-bit one, + leaving `i32 -> index -> i64` casts into `mlirAsyncRuntimeCreateGroup` that + LLVM translation rejects (`00for_await`). The upstream pass is prebuilt, so + two tslang passes repair its output when `sizeBits() < 64`: + `AsyncTargetWidthPass` (after `ConvertAsyncToLLVM`) retypes `aligned_alloc` + to pointer width, and `AsyncIndexCastPass` (after `LowerToLLVM`) folds the + cast chains into `sext`/`trunc`. x64 never runs them. All six async corpus + files run at i686 under gc (async needs Boehm's thread API, so under rc and + none it does not link at x64 either); wasm32 had the same allocator bug. - **4c — the suite at i686:** test-runner arch flag, x86 default library, reading `__decls` from a DLL file, and the remaining failures. -Remaining i686-only probe failures after 4a: the six async files (4b); -`internals` (an `inline_asm` with an `=r` constraint, which no single +Remaining i686-only probe failures after 4b: `internals` (an `inline_asm` with an `=r` constraint, which no single i686 register can satisfy); `02funcs_vararg` (unresolved `_printf`; it is commented out of the suite). @@ -312,14 +324,15 @@ confusion: `MLIRGenImpl::mlirGenImportSharedLib` calls `getPermanentLibrary`, which loads the x86 DLL into the x64 compiler process to read its `__decls`. The fix is to read `__decls` from the file rather than by loading it. -- **`00for_await` at i686 (phase 4).** A likely cause of the leftover - `unrealized_conversion_cast`: upstream `ConvertAsyncToLLVMPass` - (`AsyncToLLVM.cpp:1026`) builds `LowerToLLVMOptions(ctx)` with LLVM's default - layout and a 64-bit index, so its types disagree with the i686 type converter's. - -- **Async programs crash at i686 (phase 4).** Found in phase 3: every async - program, including `00async_await` and one with no `try`, exits 139 as a - 32-bit exe. Not EH-related. +- **`mlirAsyncRuntimGetNumWorkerThreads` is declared `() -> i32` at 32 bits** + (upstream declares it returning `index`) while the runtime returns `int64_t`. + Harmless at i686 cdecl (the low half is in EAX); at wasm32 it would be a + signature mismatch. Unreachable from TypeScript today: only upstream's + async-parallel-for creates the op. +- **wasm32 `-mm=none` async crashes the compiler (pre-existing).** Compiling an + async program for wasm32 without GC usually fails with 0xC0000005, before and + after phase 4b. Suspected: `MemAllocFixPass` erasing the `free` declaration + that the coroutine frame's `free` calls still use. - **x86 EH type names use the 64-bit mangling (phase 4, interop only).** The hardcoded names in `LLVMRTTIHelperVCWin32Const.h` (`??_R0PEAD@8`, `_CT??_R0PEAD@88`) carry the `__ptr64` qualifier and 64-bit sizes. tslang's diff --git a/tslang/include/TypeScript/Passes.h b/tslang/include/TypeScript/Passes.h index af003521c..08e197900 100644 --- a/tslang/include/TypeScript/Passes.h +++ b/tslang/include/TypeScript/Passes.h @@ -41,6 +41,16 @@ std::unique_ptr createGCPass(CompileOptions&); /// MemAlloc Pass to replace ts_malloc, ts_realloc, ts_free std::unique_ptr createMemAllocPass(CompileOptions&); +/// Repairs the upstream ConvertAsyncToLLVM output for a target whose pointers are narrower than +/// 64 bits: that pass declares the coroutine frame allocator as aligned_alloc(i64, i64) whatever +/// the target. Schedule only when the pointer width is below 64. +std::unique_ptr createAsyncTargetWidthPass(unsigned pointerBits); + +/// Folds the `iA -> index -> iB` unrealized_conversion_cast chains left where upstream +/// ConvertAsyncToLLVM (64-bit index) meets LowerToLLVM (pointer-width index). Schedule after +/// LowerToLLVM, only when the pointer width is below 64. +std::unique_ptr createAsyncIndexCastPass(); + } // end namespace typescript } // end namespace mlir diff --git a/tslang/lib/TypeScript/AsyncTargetWidthPass.cpp b/tslang/lib/TypeScript/AsyncTargetWidthPass.cpp new file mode 100644 index 000000000..ee1ed1396 --- /dev/null +++ b/tslang/lib/TypeScript/AsyncTargetWidthPass.cpp @@ -0,0 +1,216 @@ +#include "mlir/Pass/Pass.h" + +#include "TypeScript/Passes.h" +#include "TypeScript/Pass/ModulePass.h" + +#include "mlir/Dialect/LLVMIR/LLVMDialect.h" +#include "mlir/IR/Builders.h" + +#include "llvm/Support/Debug.h" + +#define DEBUG_TYPE "pass" + +namespace mlir_ts = mlir::typescript; + +namespace +{ + +// the name upstream's lookupOrCreateAlignedAllocFn declares +constexpr auto ALIGNED_ALLOC_NAME = "aligned_alloc"; + +// Repairs what the upstream ConvertAsyncToLLVM pass leaves behind for a target whose pointers are +// narrower than 64 bits. That pass is part of the prebuilt LLVM, so it is fixed up here rather +// than patched. Scheduled only when the target's pointer width is below 64, so x64 never sees it. +class AsyncTargetWidthPass : public mlir::PassWrapper +{ + public: + MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(AsyncTargetWidthPass) + + unsigned pointerBits; + + AsyncTargetWidthPass(unsigned pointerBits) : pointerBits(pointerBits) + { + } + + void runOnModule() override + { + if (mlir::failed(fixFrameAllocator(getModule()))) + { + signalPassFailure(); + } + } + + // Upstream ConvertAsyncToLLVM assumes 64-bit sizes: it allocates coroutine frames with + // aligned_alloc(i64, i64) whatever the target. aligned_alloc takes size_t, which is pointer + // width. At i686 (cdecl, arguments on the stack) the callee reads (alignment, 0) and the frame + // overflows its block; at wasm32 the call does not match the definition's signature. Retype the declaration to the pointer width and truncate the arguments at each call. + // The values are a coroutine frame's size and alignment, which fit in 32 bits. + // + // Runs directly after ConvertAsyncToLLVM and before GCPass, which renames the declaration + // (and its calls) to GC_memalign and so keeps the corrected signature. Under the other memory + // models the name resolves to the runtime's aligned_alloc shim, which is (size_t, size_t) too. + mlir::LogicalResult fixFrameAllocator(mlir::ModuleOp m) + { + auto funcOp = m.lookupSymbol(ALIGNED_ALLOC_NAME); + if (!funcOp) + { + return mlir::success(); + } + + auto *context = m.getContext(); + auto pointerIntType = mlir::IntegerType::get(context, pointerBits); + auto i64Type = mlir::IntegerType::get(context, 64); + + auto funcType = funcOp.getFunctionType(); + auto params = funcType.getParams(); + auto hasShape = [&](mlir::Type intType) { + return funcOp.isExternal() && !funcType.isVarArg() && params.size() == 2 && params[0] == intType && + params[1] == intType && isa(funcType.getReturnType()); + }; + + if (hasShape(pointerIntType)) + { + return mlir::success(); + } + + if (!hasShape(i64Type)) + { + return funcOp.emitError("async target width: unexpected aligned_alloc declaration ") + << funcType << ", expected an external (i64, i64) -> ptr"; + } + + llvm::SmallVector calls; + auto walkResult = m.walk([&](LLVM::CallOp callOp) { + auto callee = callOp.getCallee(); + if (!callee.has_value() || callee.value() != ALIGNED_ALLOC_NAME) + { + return mlir::WalkResult::advance(); + } + + auto operands = callOp.getArgOperands(); + if (operands.size() != 2 || operands[0].getType() != i64Type || operands[1].getType() != i64Type) + { + callOp.emitError("async target width: unexpected aligned_alloc call, expected two i64 arguments"); + return mlir::WalkResult::interrupt(); + } + + calls.push_back(callOp); + return mlir::WalkResult::advance(); + }); + + if (walkResult.wasInterrupted()) + { + return mlir::failure(); + } + + funcOp.setFunctionType( + LLVM::LLVMFunctionType::get(funcType.getReturnType(), {pointerIntType, pointerIntType})); + + mlir::OpBuilder builder(context); + for (auto callOp : calls) + { + builder.setInsertionPoint(callOp); + auto operands = callOp.getArgOperands(); + auto alignment = builder.create(callOp.getLoc(), pointerIntType, operands[0]); + auto size = builder.create(callOp.getLoc(), pointerIntType, operands[1]); + callOp.getArgOperandsMutable().assign(mlir::ValueRange{alignment, size}); + } + + LLVM_DEBUG(llvm::dbgs() << "\n!! AsyncTargetWidthPass: retyped aligned_alloc to i" << pointerBits + << ", calls: " << calls.size() << "\n";); + + return mlir::success(); + } +}; + +// Folds `iA -> index -> iB` chains of unrealized_conversion_cast into one integer cast. +// +// Two type converters with different index widths meet here. Upstream ConvertAsyncToLLVM lowers +// the async runtime calls with its own converter, whose index is 64 bits, and the runtime really +// does take int64_t there (e.g. mlirAsyncRuntimeCreateGroup(int64_t), lib/AsyncRuntimeCommon.inc), +// so it bridges an index operand with `index -> i64`. LowerToLLVM later converts the operand's +// producer with our converter, whose index is pointer width, and bridges back with `i32 -> index`. +// Neither cast is an identity, so reconciliation cannot remove the pair and LLVM translation +// rejects it. Scheduled after LowerToLLVM, the first point at which both halves exist. +class AsyncIndexCastPass : public mlir::PassWrapper +{ + public: + MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(AsyncIndexCastPass) + + void runOnModule() override + { + llvm::SmallVector outerCasts; + getModule().walk([&](mlir::UnrealizedConversionCastOp castOp) { + if (isIndexBridge(castOp)) + { + outerCasts.push_back(castOp); + } + }); + + mlir::OpBuilder builder(&getContext()); + for (auto outerCast : outerCasts) + { + auto innerCast = outerCast.getInputs().front().getDefiningOp(); + auto source = innerCast.getInputs().front(); + auto sourceType = cast(source.getType()); + auto resultType = cast(outerCast.getResult(0).getType()); + + mlir::Value replacement = source; + builder.setInsertionPoint(outerCast); + // index is signed in MLIR's arith semantics, so widen with sign extension. Today the + // only widened value is async.runtime.create_group's size (for-await's constant 0); + // the narrowing direction would be add_to_group's rank, which tslang never uses. Both + // are non-negative, where sign and zero extension agree. The fold is not limited to + // async: any iA -> index -> iB chain is lowered, but only ConvertAsyncToLLVM makes + // them, and before this pass such a chain failed LLVM translation anyway. A chain of + // any other shape is left for translation to reject. + if (sourceType.getWidth() < resultType.getWidth()) + { + replacement = builder.create(outerCast.getLoc(), resultType, source); + } + else if (sourceType.getWidth() > resultType.getWidth()) + { + replacement = builder.create(outerCast.getLoc(), resultType, source); + } + + outerCast.getResult(0).replaceAllUsesWith(replacement); + outerCast.erase(); + if (innerCast->use_empty()) + { + innerCast.erase(); + } + } + + LLVM_DEBUG(llvm::dbgs() << "\n!! AsyncIndexCastPass: folded " << outerCasts.size() << " index casts\n";); + } + + // The `index -> iB` cast of an `iA -> index -> iB` chain, each cast having one operand and one + // result. Every other unrealized_conversion_cast is left alone. + static bool isIndexBridge(mlir::UnrealizedConversionCastOp outerCast) + { + if (outerCast.getInputs().size() != 1 || outerCast->getNumResults() != 1 || + !isa(outerCast.getResult(0).getType()) || + !isa(outerCast.getInputs().front().getType())) + { + return false; + } + + auto innerCast = outerCast.getInputs().front().getDefiningOp(); + return innerCast && innerCast.getInputs().size() == 1 && innerCast->getNumResults() == 1 && + isa(innerCast.getInputs().front().getType()); + } +}; +} // end anonymous namespace + +#undef DEBUG_TYPE + +/// Create pass. +std::unique_ptr mlir_ts::createAsyncTargetWidthPass(unsigned pointerBits) +{ + return std::make_unique(pointerBits); +} + +std::unique_ptr mlir_ts::createAsyncIndexCastPass() +{ + return std::make_unique(); +} diff --git a/tslang/lib/TypeScript/CMakeLists.txt b/tslang/lib/TypeScript/CMakeLists.txt index 293019953..c17d3260c 100644 --- a/tslang/lib/TypeScript/CMakeLists.txt +++ b/tslang/lib/TypeScript/CMakeLists.txt @@ -34,6 +34,7 @@ add_mlir_dialect_library(MLIRTypeScript OwnershipVerifierPass.cpp OwnedReturnConsumptionPass.cpp GCPass.cpp + AsyncTargetWidthPass.cpp ObjDumper.cpp DeclarationPrinter.cpp diff --git a/tslang/test/check-x86-run.sh b/tslang/test/check-x86-run.sh index 53af47629..a6f47d540 100755 --- a/tslang/test/check-x86-run.sh +++ b/tslang/test/check-x86-run.sh @@ -10,10 +10,17 @@ # gc_stress.ts keeps a 1000-node list alive while it allocates about a million garbage nodes plus # strings and arrays; under -mm=gc that forces many collections, and the list's sum is wrong (or # the program crashes) if a collection freed any of it. +# +# Phase 4b gate: async/await programs, including `for await`, at i686. The async runtime needs +# Boehm's thread API, so these link only under -mm=gc (rc and none fail to link at x64 too; out of +# scope). Each corpus file asserts internally, so exiting 0 is the check, as in check-x86-eh.sh's +# exception corpus loop. await_order.ts is the minimal repro from the phase 4b plan and gets an +# exact-output check because its whole point is the print ORDER around the await. set -u TSLANG="${1:?usage: check-x86-run.sh }" REPO="$(cd "$(dirname "$0")/../.." && pwd)" PROGRAMS="$REPO/tslang/test/x86" +TESTS="$REPO/tslang/test/tester/tests" READOBJ="$REPO/3rdParty/llvm/x64/release/bin/llvm-readobj.exe" # Hermetic: the compiler falls back to these when a flag is missing, and they may name stale builds. @@ -26,12 +33,26 @@ GC_X86="$GC_DIR/x86/gc.lib" RT_DIR="$REPO/__build/tslang-runtime/release" RT_X86="$RT_DIR/x86/TypeScriptAsyncRuntime.lib" -for f in "$TSLANG" "$READOBJ" "$GC_X86" "$RT_X86" "$PROGRAMS/hello.ts" "$PROGRAMS/gc_stress.ts"; do +# The async corpus cases (gc only), from test/tester/tests. They are built with --entry-point to +# mirror the suite; for --emit=exe it changes nothing, it matters only to the --emit=llvm IR checks. +ASYNC_CORPUS=(00async_await 00async_gc_threading 00async_result_types 00owned_async + 00for_await 00for_await_yield) + +for f in "$TSLANG" "$READOBJ" "$GC_X86" "$RT_X86" "$PROGRAMS/hello.ts" "$PROGRAMS/gc_stress.ts" \ + "$PROGRAMS/await_order.ts"; do if [ ! -f "$f" ]; then echo "FAIL missing prerequisite: $f" exit 1 fi done +for name in "${ASYNC_CORPUS[@]}"; do + if [ ! -f "$TESTS/$name.ts" ]; then + echo "FAIL missing prerequisite: $TESTS/$name.ts" + exit 1 + fi +done +# Without timeout a hang looks identical to a run in progress rather than a FAIL. +command -v timeout >/dev/null 2>&1 || { echo "FAIL missing prerequisite: timeout"; exit 1; } work="$(mktemp -d)" trap 'rm -rf "$work"' EXIT @@ -46,7 +67,10 @@ expected() { hello) printf 'hello from 32-bit\n' ;; # sum(0..999) = 499500; sum of lengths of "round of 200" for r in 0..199 # = 200 * 13 + (10 * 1 + 90 * 2 + 100 * 3) = 3090 - gc_stress) printf '499500\n3090\n' ;; + gc_stress) printf '499500\n3090\n' ;; + # Confirmed at x64 (docs/superpowers/plans/2026-09-22-32-bit-phase-4b-async.md): a + # non-async main resumes past `await g()` before g()'s body runs on the async runtime. + await_order) printf 'start\nafter await\nin g\n' ;; esac } @@ -56,7 +80,7 @@ run_case() { local prog="$1" mm="$2" shift 2 local name="$prog -mm=$mm" exe="$work/$prog-$mm.exe" err out status machine - err="$("$TSLANG" --emit=exe --opt -mm="$mm" --no-default-lib -mtriple=i686-pc-windows-msvc \ + err="$("$TSLANG" --emit=exe --opt -mm="$mm" --no-default-lib --entry-point -mtriple=i686-pc-windows-msvc \ "$@" "$PROGRAMS/$prog.ts" -o "$exe" 2>&1 >/dev/null)" status=$? if [ "$status" -ne 0 ] || [ ! -f "$exe" ]; then @@ -70,8 +94,9 @@ run_case() { fail=1 return fi - # The Windows CRT writes stdout in text mode, so lines end in CR LF. - out="$("$exe" 2>"$work/stderr.txt" | tr -d '\r'; echo "exit=${PIPESTATUS[0]}")" + # The Windows CRT writes stdout in text mode, so lines end in CR LF. A 30s timeout catches a + # hang (distinct from the crashes these cases are otherwise looking for) as a FAIL, not a wait. + out="$(timeout 30 "$exe" 2>"$work/stderr.txt" | tr -d '\r'; echo "exit=${PIPESTATUS[0]}")" if [ "$out" != "$(expected "$prog"; echo "exit=0")" ]; then echo "FAIL $name: expected output" expected "$prog" | sed 's/^/ /' @@ -91,4 +116,93 @@ for prog in hello gc_stress; do run_case "$prog" none "$RT_FLAG" done +# await_order.ts links only under -mm=gc: async needs Boehm's thread API, and rc/none fail to +# link even at x64 (out of scope; see the plan's Global Constraints). +run_case await_order gc "$GC_FLAG" "$RT_FLAG" + +# --- Phase 4b: async corpus files run at i686 under gc ------------------------------------- +# run_async_corpus -> builds test/tester/tests/.ts at i686, -mm=gc, --entry-point; +# checks the I386 machine and that a 30s-timeout run exits 0. The file's own assert()s are the +# correctness check; only the exit code is verified here, as check-x86-eh.sh does for its +# exception corpus. +run_async_corpus() { + local name="$1" + local label="$name (async, -mm=gc)" exe="$work/async-$name.exe" err status machine run_status + err="$("$TSLANG" --emit=exe --opt -mm=gc --no-default-lib --entry-point -mtriple=i686-pc-windows-msvc \ + "$GC_FLAG" "$RT_FLAG" "$TESTS/$name.ts" -o "$exe" 2>&1 >/dev/null)" + status=$? + if [ "$status" -ne 0 ] || [ ! -f "$exe" ]; then + echo "FAIL $label: compile failed (exit $status): $err" + fail=1 + return + fi + machine="$("$READOBJ" --file-headers "$exe" | grep -m1 'Machine:')" + if ! printf '%s' "$machine" | grep -q 'IMAGE_FILE_MACHINE_I386 (0x14C)'; then + echo "FAIL $label: expected an I386 (0x14C) executable, got: $machine" + fail=1 + return + fi + timeout 30 "$exe" >"$work/stdout.txt" 2>"$work/stderr.txt" + run_status=$? + if [ "$run_status" -ne 0 ]; then + echo "FAIL $label: expected exit 0, got $run_status" + sed 's/^/ stdout: /' "$work/stdout.txt" + sed 's/^/ stderr: /' "$work/stderr.txt" + fail=1 + return + fi + echo "ok $label" +} + +for name in "${ASYNC_CORPUS[@]}"; do + run_async_corpus "$name" +done + +# --- Phase 4b: frame allocator width (i686) ------------------------------------------------- +# Upstream ConvertAsyncToLLVM hardcodes the coroutine frame allocator's declaration to +# aligned_alloc(i64, i64), whatever the target. aligned_alloc takes size_t, which is pointer +# width, so at i686 the callee reads (alignment, size=0) and the frame overflows its block. +# The repair pass (task 2/3) must retype the declaration to the pointer width: GC_memalign is +# GCPass's rename of aligned_alloc when a collector is linked in. +emit_await_order_ir() { + local mm="$1" + local out="$work/await_order.$mm.ll" err status + err="$("$TSLANG" --emit=llvm --opt -mm="$mm" --no-default-lib --entry-point -mtriple=i686-pc-windows-msvc \ + "$PROGRAMS/await_order.ts" -o "$out" 2>&1 >/dev/null)" + status=$? + if [ "$status" -ne 0 ] || [ ! -f "$out" ]; then + echo "FAIL x86 IR -mm=$mm: await_order compiles (exit $status): $err" + fail=1 + return 1 + fi + printf '%s' "$out" +} + +# Only the parameter TYPES are checked: once aligned_alloc has the target's size_t signature, +# LLVM recognizes it as the C library function and adds attributes to the declaration, e.g. +# `declare noalias noundef ptr @aligned_alloc(i32 allocalign noundef, i32 noundef)`. +declared_with_i32_params() { + grep -Eq "^declare .*ptr @$1\(i32( [^,]*)?, i32( [^)]*)?\)" "$2" +} + +if out="$(emit_await_order_ir gc)"; then + if declared_with_i32_params GC_memalign "$out"; then + echo "ok x86 IR -mm=gc: frame allocator is GC_memalign(i32, i32)" + else + echo "FAIL x86 IR -mm=gc: frame allocator is GC_memalign(i32, i32)" + grep -m1 -E '@GC_memalign\(|@aligned_alloc\(' "$out" | sed 's/^/ got: /' + fail=1 + fi +fi + +if out="$(emit_await_order_ir none)"; then + if declared_with_i32_params aligned_alloc "$out"; then + echo "ok x86 IR -mm=none: frame allocator is aligned_alloc(i32, i32)" + else + echo "FAIL x86 IR -mm=none: frame allocator is aligned_alloc(i32, i32)" + grep -m1 -E '@aligned_alloc\(' "$out" | sed 's/^/ got: /' + fail=1 + fi +fi + exit "$fail" diff --git a/tslang/test/x86/await_order.ts b/tslang/test/x86/await_order.ts new file mode 100644 index 000000000..8c3141ccf --- /dev/null +++ b/tslang/test/x86/await_order.ts @@ -0,0 +1,8 @@ +// Minimal repro for 32-bit async: a non-async main awaits an async function. Confirmed at x64 +// (docs/superpowers/plans/2026-09-22-32-bit-phase-4b-async.md): "after await" prints before +// "in g" because g() keeps running on the async runtime after main resumes past the await. +// check-x86-run.sh builds this for 32-bit Windows and compares its output exactly. That order is +// not normal TypeScript await semantics; a future fix to await in a non-async function is expected +// to change it, and this expected output with it. +async function g() { print("in g"); } +function main() { print("start"); await g(); print("after await"); } diff --git a/tslang/tslang/transform.cpp b/tslang/tslang/transform.cpp index 0c39a4a5d..b4df4c4b5 100644 --- a/tslang/tslang/transform.cpp +++ b/tslang/tslang/transform.cpp @@ -165,8 +165,24 @@ int runMLIRPasses(mlir::MLIRContext &context, llvm::SourceMgr &sourceMgr, mlir:: { #ifdef ENABLE_ASYNC pm.addPass(mlir::createConvertAsyncToLLVMPass()); + // Before GCPass, which renames aligned_alloc to GC_memalign and so keeps the corrected + // signature; before LowerToLLVM, whose memref alloc lowering (AlignedAlloc) looks up + // aligned_alloc with a pointer-width index and would reject the (i64, i64) declaration as a + // redefinition. + if (compileOptions.sizeBits() < 64) + { + pm.addPass(mlir::typescript::createAsyncTargetWidthPass(compileOptions.sizeBits())); + } #endif pm.addPass(mlir::typescript::createLowerToLLVMPass(compileOptions)); +#ifdef ENABLE_ASYNC + // After LowerToLLVM: the index casts it folds are half made by ConvertAsyncToLLVM and half + // by LowerToLLVM, so this is the first point at which the whole chain exists. + if (compileOptions.sizeBits() < 64) + { + pm.addPass(mlir::typescript::createAsyncIndexCastPass()); + } +#endif if (compileOptions.generateDebugInfo) { pm.addPass(mlir::LLVM::createDIScopeForLLVMFuncOpPass());