diff --git a/docs/memory-models.md b/docs/memory-models.md index 6f3c029ec..d049cbdf4 100644 --- a/docs/memory-models.md +++ b/docs/memory-models.md @@ -128,7 +128,8 @@ includes an object that is: - returned, thrown, stored in a field, array, global or captured variable, or converted to `any`, an interface or a union; -- passed to a function in another module, including the default library; +- passed to a function in another module, unless that module's own analysis showed the function + only reads the argument (see below); - made in a function that is `async`, a generator, or recursive, or - under `-mm=rc` and `-mm=own` - one that has a `try`; - still in use when the next iteration of its loop makes a new one. A use on the way out of the @@ -137,6 +138,20 @@ includes an object that is: Under `-mm=rc` and `-mm=own`, an object that owns memory stays on the heap in a module that uses `Shared`, because `Shared.count` would see its contents given back later. +### Calls into other modules + +Every module the compiler builds ahead of time records, inside its object file, which parameters of its functions +are only read and never kept (the `__tsborrow_` symbol). A program reads these facts from the +libraries it links statically - the default library's static library and any `--lib` - and, under +`--emit=jit`, from the default library's DLL it loads. An object passed to such a parameter may then +go on the stack: `Math.max(a, b, c)`, for instance, builds its argument array on the stack. + +The facts are read only where the file read is the file that will run. A program linked against a +DLL, a `-shared` import and `--emit=obj` read none, and every call into them stays on the heap, as +does a call through a library class's virtual method. A library built before this feature publishes +nothing, which is not an error. `tslang --print-borrow-facts=` prints what an object file, +static library or DLL publishes. + Two limits keep frames small: one object may take at most 256 bytes (`--stack-promotion-max-object=`), and one function at most 4096 bytes in all (`--stack-promotion-max-frame=`). These are estimates, made before the exact layout is known. diff --git a/tslang/docs/superpowers/plans/2026-10-18-borrow-facts-across-modules.md b/tslang/docs/superpowers/plans/2026-10-18-borrow-facts-across-modules.md new file mode 100644 index 000000000..909e8e18b --- /dev/null +++ b/tslang/docs/superpowers/plans/2026-10-18-borrow-facts-across-modules.md @@ -0,0 +1,259 @@ +# Borrow Facts Across Modules Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A library records which parameters of its non-private functions are only borrowed; a +program linked statically against it (or running under the JIT against the default library) +reads those facts, and stack promotion stops treating such calls as escapes. + +**Architecture:** A new pass writes the facts of every module into one exported `[N x i8]` +constant, `__tsborrow__`, computed by `StackEscapeFacts` in a conservative mode. The +driver reads every `__tsborrow_` symbol from the archives an `--emit=exe` link uses (the default +library's static archive and `--lib` archives) or, under the JIT, from the default library DLL it +loads, and hands the merged text to the passes through `CompileOptions`. `StackEscapeFacts` +consults it for calls to bodiless functions. + +**Tech Stack:** C++17, MLIR (TypeScript dialect, LLVM dialect), `llvm::object` (COFF, ELF, +archives), CMake/CTest scripts. + +**Spec:** `tslang/docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md` + +## Global Constraints + +- Facts are generated, never hand-written. No attribute, no flag that asserts a borrow. +- A fact may only claim a parameter is borrowed when the library's own analysis proves it. When + unsure: escape. +- Facts are read only for static links (`--emit=exe` with the default library's archive, `--lib` + archives) and the JIT (the default library DLL it loads). Never for DLL links, never for + `-shared` imports, never for `--emit=obj|llvm|bc|asm|mlir*` or `--emit=dll`. +- The file read must be the file linked or loaded. +- Missing, old or unreadable facts are not an error and print no warning: every call is an escape. +- Facts are emitted whether or not `--no-stack-promotion` is given, and whatever `--export=`, + `--embed-declarations=` say. +- Producer: every class counts as exported (no vtable is trusted as local). +- Consumer: a virtual call on a library class stays an escape. +- Platforms: Windows, Linux, and the JIT. +- Version 1 may be almost pure plumbing; no default-library code changes. +- Full suite passes with promotion on and with `-DTSLANG_TEST_NO_STACK_PROMOTION=ON`. + +## Review Focus + +1. **A library function that keeps its argument in a way the analysis misses** (stored through + an alias, captured by a closure that is returned, handed to a callee in another object file + of the same library). Expected: not listed. Task 1 tests keep/return/callback/other-module + cases. +2. **A `--lib` value the linker resolves to a different file than the one read.** Expected: the + driver reads only a file it then passes to the linker by full path (Task 3 ruling). +3. **Symbol name mismatch between the library's definition and the program's declaration** + (methods, namespaces). Expected: no match means no fact, never a wrong fact. Task 3 tests a + namespaced function. +4. **A stale JIT cache after the default library is rebuilt with different facts.** Expected: a + cache miss. Task 4 test. +5. **Archives with members that are not objects of the target format** (import members in a + Windows import library, LLVM bitcode members). Expected: skipped silently. Task 2 test with an + import library. + +--- + +### Task 1: Producer — `__tsborrow_` symbol + +**Files:** +- Create: `lib/TypeScript/BorrowFactsExportPass.cpp` +- Modify: `lib/TypeScript/StackEscapeFacts.h` (conservative mode, public query) +- Modify: `include/TypeScript/Passes.h`, `include/TypeScript/Defines.h`, + `lib/TypeScript/CMakeLists.txt`, `tslang/transform.cpp` +- Test: `test/tester/stack-promotion/facts_lib.ts`, `test/tester/stack-promotion/facts_lib_other.ts`, + `test/tester/CMakeLists.txt` (stack-promotion section) + +**Interfaces:** +- Produces: `#define SHARED_LIB_BORROW_FACTS "__tsborrow_"` in Defines.h. +- Produces: `StackEscapeFacts(mlir::ModuleOp module, bool trustLocalVTables = true, + const BorrowFactsMap *imported = nullptr)` (the third parameter is added in Task 3; add it now + defaulted to null, unused) and `bool paramBorrowed(mlir_ts::FuncOp funcOp, unsigned index) const` + (true only for a function with a body whose parameter never escapes). +- Produces: `std::unique_ptr mlir_ts::createBorrowFactsExportPass(CompileOptions &)`. +- Produces: hidden option `--no-borrow-facts-export` (tslang.cpp, read in transform.cpp): skip + the pass. Test-only stand-in for a library built before this change. +- Produces: facts text format, one line per function with at least one borrowed parameter: + ` [...]\n`, symbol names sorted, indexes ascending, body-argument + indexes (`this` is 0 for a method). The global holds the text plus a terminating `\0`. + +- [ ] **Step 1: Write the failing test.** `facts_lib.ts` exports: + `sumOf(a: number[]): number` (reads only → borrowed), + `keep(a: number[])` (stores into a module global → not listed), + `same(a: number[]): number[]` (returns it → not listed), + `viaCallback(a: number[], f: (x: number[]) => void)` (calls `f(a)` → `a` not listed), + `mixed(a: number[], b: number[]): number` (keeps `b`, reads `a` → `mixed 0`), + `useOther(a: number[])` passing `a` to `otherKeep` declared from `facts_lib_other.ts` via + `declare function` (no body here → not listed), + `namespace Geo { export function len(a: number[]): number }` (borrowed, namespaced symbol), + a private (non-exported, `--export` default) helper that borrows (must not appear), and a class + `Holder` with a method `peek(a: number[])` that only reads `a` (listed as ` 1`) + plus a virtual method called through `this` inside the library to confirm vtables are not + trusted. Register decision tests per model (gc rc own none): + `tslang --emit=llvm --no-default-lib -mm= facts_lib.ts` with `PASS_REGULAR_EXPRESSION` + `@__tsborrow_facts_lib_[0-9]+ = (dllexport )?constant \[[0-9]+ x i8\] c"[^"]*sumOf 0\\0A` + and `FAIL_REGULAR_EXPRESSION` matching `keep |same |viaCallback |useOther |helper ` inside the + constant. Add one more with `--no-stack-promotion --export=none` expecting the same symbol. +- [ ] **Step 2: Run** `ctest -C Release -R stack-facts-lib` — Expected: FAIL (no `__tsborrow_`). +- [ ] **Step 3: Implement.** In `StackEscapeFacts`, `trustLocalVTables=false` makes + `collectVTables` treat every vtable as non-local. Add `paramBorrowed`. The pass builds + `StackEscapeFacts(module, /*trustLocalVTables=*/false)`, walks non-declaration `FuncOp`s whose + symbol visibility is not private, writes the text, and when it is non-empty creates an + `LLVM::GlobalOp` (constant, external linkage, type `!llvm.array`, `value` the text with + `\0`, section `DLL_EXPORT` so `ExportFixPass` makes it `dllexport` on Windows and clears the + section). Name: `SHARED_LIB_BORROW_FACTS + stem + "_" + to_string(xxh3_64bits(canonical + main-file path))`, the same suffix `MLIRGenImpl::moduleSymbolSuffix` builds (move that helper + to a header both can use, or duplicate it beside a comment naming the original). Register the + pass in `transform.cpp` right before the stack promotion block, unconditionally except + `compileOptions.isJit`. +- [ ] **Step 4: Run** the step 2 command — Expected: PASS, all models. +- [ ] **Step 5: Run the stack-promotion section** `ctest -C Release -R stack -j8` — Expected: all + pass (the new pass changes no promotion decision). +- [ ] **Step 6: Commit** `Borrow facts: libraries record borrowed parameters in __tsborrow_`. + +### Task 2: Reader — `Dump::readCStringSymbols` and `--print-borrow-facts` + +**Files:** +- Modify: `include/TypeScript/ObjDumper.h`, `lib/TypeScript/ObjDumper.cpp` +- Modify: `tslang/tslang.cpp` (hidden flag) +- Create: `test/tester/borrow-facts.cmake` (grows in Task 3) +- Test: `test/tester/CMakeLists.txt` + +**Interfaces:** +- Consumes: the facts format and symbol from Task 1. +- Produces: `std::vector> Dump::readCStringSymbols( + llvm::StringRef path, llvm::StringRef prefix)` returning (symbol, text) for every defined + symbol with that prefix in an object file (COFF or ELF) or in every object member of an + archive (`.lib`, `.a`). Members that are not COFF/ELF objects (import members, bitcode) and a + file that cannot be opened yield nothing. The text is read from the symbol's section contents + at the symbol's offset up to the first `\0` (bounded by the section end). +- Produces: hidden option `--print-borrow-facts=`: prints each `symbol\ntext` found in + `` (object or archive) and exits 0, printing nothing for none. + +- [ ] **Step 1: Write the failing test.** `borrow-facts.cmake`, invoked with `TSLANG`, + `SOURCE_DIR`, `WORK_DIR`, `AR` (`${CMAKE_AR}`), `MM`: compiles `facts_lib.ts` with + `--emit=obj --no-default-lib -mm=${MM}`, archives it (`lib /OUT:facts_lib.lib facts_lib.obj` + on Windows, `ar rcs libfacts_lib.a facts_lib.o` elsewhere), then runs + `--print-borrow-facts=` on the object and on the archive; both must match `sumOf 0` and + `mixed 0`. On Windows also run it on `gc.lib`'s import library or any import `.lib` passed as + `IMPORT_LIB` (Review Focus 5): exit 0, no output. Register `test-borrow-facts-` for the + four models, WIN32 and UNIX. +- [ ] **Step 2: Run** `ctest -C Release -R test-borrow-facts` — Expected: FAIL (unknown option). +- [ ] **Step 3: Implement** `readCStringSymbols` with `llvm::object::createBinary`: an `Archive` + iterates `children()` and recurses into each child that is an `ObjectFile`; an `ObjectFile` + walks `symbols()`, keeps defined ones whose name starts with `prefix`, and reads + `section->getContents()` at `symbol.getValue()` (COFF: the value is the section offset; ELF + relocatable: likewise). Then the flag. +- [ ] **Step 4: Run** the step 2 command — Expected: PASS. +- [ ] **Step 5: Commit** `Borrow facts: read __tsborrow_ from objects and archives`. + +### Task 3: Consumer — static links (`--lib` and the default library archive) + +**Files:** +- Modify: `include/TypeScript/DataStructs.h` (`CompileOptions`) +- Modify: `lib/TypeScript/StackEscapeFacts.h`, `lib/TypeScript/StackPromotionPass.cpp` +- Create: `tslang/borrowfacts.cpp` +- Modify: `tslang/defaultlib.cpp`, `tslang/exe.cpp`, `tslang/tslang.cpp`, `tslang/CMakeLists.txt` +- Test: `test/tester/stack-promotion/facts_use.ts`, `test/tester/borrow-facts.cmake`, + `test/tester/CMakeLists.txt` + +**Interfaces:** +- Consumes: `Dump::readCStringSymbols` (Task 2); facts format (Task 1). +- Produces: `CompileOptions::importedBorrowFacts` (`std::string`, merged text, empty = none). +- Produces: `using BorrowFactsMap = llvm::StringMap>;` and + `BorrowFactsMap parseBorrowFacts(llvm::StringRef text)` in `StackEscapeFacts.h` (malformed + lines skipped). +- Produces: `std::string getDefaultLibBinary(bool shared, const CompileOptions &, + const llvm::Triple &)` in `defaultlib.cpp`: the full path of the default library file a link + or the JIT uses (`/TypeScriptDefaultLib.lib`, `.dll`, `libTypeScriptDefaultLib.a`, `.so`), + composed with `getDefaultLibSubDir`; `exe.cpp` and `jit.cpp` derive their directory from it. +- Produces: `std::string resolveLinkLibrary(llvm::StringRef value)` in `borrowfacts.cpp`: the + value itself when it names an existing file, else `.lib` (Windows) or `lib.a` + (elsewhere) in the current directory when that exists, else empty. +- Produces: `void readStaticBorrowFacts(CompileOptions &)` in `borrowfacts.cpp`, called in + `main` after `prepareDefaultLib` when `emitAction == BuildExe`. + +- [ ] **Step 1: Write the failing test.** `facts_use.ts` declares the library's functions with + `declare function` (and `declare namespace Geo`, and the `Holder` class with a factory + `declare function makeHolder(): Holder`) and calls each with a fresh array literal of numbers + that is otherwise only read. Extend `borrow-facts.cmake`: build the archive as in Task 2, then + `tslang --emit=exe --no-default-lib -mm=${MM} --stack-promotion-report --lib=facts_lib + facts_use.ts` in the work directory, and check the remarks: + `sumOf`, `mixed`'s first argument, `Geo.len` → `stack`; + `keep`, `same`, `viaCallback`, `mixed`'s second argument, `useOther` → `heap: unknown-callee`; + `holder.peek(...)` (virtual) → `heap`. + Run the exe; its printed result must equal the same program built with `--no-stack-promotion`. + Also check: the same compile with `--emit=obj` reports every call `heap` (no facts read). And + the missing-facts case: an archive of `facts_lib.ts` built with the hidden Task 1 flag + `--no-borrow-facts-export` (it only suppresses the symbol, the safe direction) links cleanly, + prints no warning, and reports every call `heap: unknown-callee`. + Add the `-shared` negative: `test-runner -shared` style DLL of `facts_lib.ts` imported by + `facts_import.ts` (an `import` version of `facts_use.ts`) with `--stack-promotion-report`: + every call `heap`. +- [ ] **Step 2: Run** `ctest -C Release -R test-borrow-facts` — Expected: FAIL (`sumOf` is + `heap: unknown-callee`). +- [ ] **Step 3: Implement.** `readStaticBorrowFacts` reads `getDefaultLibBinary(false, ...)` + unless `noDefaultLib`, and every `--lib` value through `resolveLinkLibrary`, concatenating the + texts into `importedBorrowFacts`. `exe.cpp`: a `--lib` value that `resolveLinkLibrary` + resolves is passed to the linker as that full path instead of `-l` (so the file read + is the file linked); others keep `-l`. `StackPromotionPass` parses the text once and passes + the map to `StackEscapeFacts`. In `argumentEscapes`, a candidate `FuncOp` with an empty body + whose name is in the map and lists `argIndex` returns `""`; otherwise unchanged. Virtual + candidates are never declarations of library methods in a way that reaches this path because + `virtualCandidates` already fails on non-local vtables — keep it that way. +- [ ] **Step 4: Run** the step 2 command — Expected: PASS, all models. +- [ ] **Step 5: Teeth.** Temporarily make the producer list every parameter; rerun; Expected: + the `keep` program's output differs or it crashes (the kept array is read after the frame is + gone — print it from the global after the calling function returns). Revert. +- [ ] **Step 6: Run the whole suite** twice (promotion on; `-DTSLANG_TEST_NO_STACK_PROMOTION=ON`), + `ctest -C Release -j16 --timeout 300 > /suite.txt` — Expected: no new failures. +- [ ] **Step 7: Commit** `Borrow facts: static links read facts from --lib and the default library`. + +### Task 4: JIT — the default library DLL, and the cache key + +**Files:** +- Modify: `tslang/borrowfacts.cpp`, `tslang/jit.cpp`, `tslang/jitcache.cpp`, `tslang/tslang.cpp` +- Test: `test/tester/borrow-facts-defaultlib.cmake`, `test/tester/CMakeLists.txt` + +**Interfaces:** +- Consumes: `getDefaultLibBinary(true, ...)` (Task 3), `Dump::getSymbols`. +- Produces: `void readJitBorrowFacts(CompileOptions &)`: when `isJit` and not `noDefaultLib`, + loads the DLL with `llvm::sys::DynamicLibrary::getPermanentLibrary` (the same file + `prepareJitProcess` loads), enumerates its `__tsborrow_` symbols with `Dump::getSymbols` + (on ELF include the dynamic symbol table), and reads each as `(const char *)address`. + Called before both `runJitCached` and the non-cached compile. +- Produces: the JIT cache manifest lists the default library DLL as a dependency hashed with + `fileHash`, as `sharedLibraryImports` already are, so a rebuilt library (new facts) misses the + cache. (Ruling: a file-hash dependency instead of a facts hash in `optionsKey`: it covers the + facts and anything else in the DLL, and needs no extra key text.) + +- [ ] **Step 1: Write the failing test.** `borrow-facts-defaultlib.cmake` (SKIPPED, like + `defaultlib-collector.cmake`, when no default library is built for the model): a program + using `s.replace(new RegExp("a", "g"), "b")` and a function from the default library that + the producer lists; pick one from `--print-borrow-facts` on the built archive whose parameter + is an object or array. If the library lists none, the test asserts only that facts were read: + the archive's facts text is non-empty and the AOT and JIT compiles succeed with correct + output. Check `--emit=exe` and `--emit=jit` remarks agree. Cache: run `--emit=jit` with the + JIT cache on into a fresh cache folder, then check the manifest written there names the + default library DLL among its dependencies. +- [ ] **Step 2: Run** `ctest -C Release -R borrow-facts-defaultlib` — Expected: FAIL or SKIPPED; + if SKIPPED, build the default library for one model first (DefaultLib `build.bat`) so the + test runs. +- [ ] **Step 3: Implement** as in Interfaces. +- [ ] **Step 4: Run** the step 2 command — Expected: PASS. +- [ ] **Step 5: Commit** `Borrow facts: the JIT reads the default library's facts; cache key`. + +### Task 5: Documentation + +**Files:** +- Modify: `docs/memory-models.md` (Stack promotion section) +- Modify: `tslang/docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md` + (status line: implemented, PR number) + +- [ ] **Step 1:** Replace "passed to a function in another module, including the default + library;" with: passed to a function in another module, unless that module is linked + statically (or is the default library under the JIT) and its own analysis showed the function + only reads the argument; never through a DLL or a `-shared` import. Mention + `--print-borrow-facts=`. +- [ ] **Step 2: Commit** `Docs: borrow facts across modules`. diff --git a/tslang/docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md b/tslang/docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md new file mode 100644 index 000000000..dfeefb8f2 --- /dev/null +++ b/tslang/docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md @@ -0,0 +1,199 @@ +# Borrow facts across modules + +Status: implemented 2026-10-18 (plan docs/superpowers/plans/2026-10-18-borrow-facts-across-modules.md). Follows +`2026-10-18-stack-promotion-design.md` (sections 8.1 and 8.2) and its v1 and v1.1 implementations +(PRs #605, #606). + +## 1. Intent + +Stack promotion treats any call into another module as an escape: the analysis has no body to +look at, so it cannot know the callee does not keep its argument. This adds the plumbing for one +module to tell another which parameters of its exported functions are only borrowed. The default +library is the first library that publishes these facts and the first one programs read them from. + +The user's stated purpose is the plumbing itself. The gain from the default library alone is +expected to be modest: its non-generic API mostly takes numbers and strings (§4.2). The same +mechanism is what caller-provided result slots (project "a", next) and `-shared` modules (later) +will build on. + +## 2. Decisions (the user's) + +- **Scope of version 1: calls from a program into the default library.** +- **Facts are generated, never hand-written.** When a library is built, the compiler records, for + every exported function, which parameters its body only borrows. It derives them from the same + escape analysis stack promotion runs. There is no `@borrows` attribute and no hand-written claim: + a wrong claim is a dangling stack pointer. +- **Facts are stored inside the library binary** as exported symbols, beside the existing + `__tsmm_` and `__tsown_` markers, so they can never disagree with the code they describe. +- **The compiler reads the default library binary on every compile**, AOT and JIT alike. +- **Version 1 may be almost pure plumbing.** A small gain from the default library alone (§4.2) + is acceptable; changing the library or the analysis so more of its calls benefit is not part of + version 1. +- **Platforms: Windows, Linux and the JIT.** Linux ahead-of-time builds read the facts from the + static archive (`.a`) they link, so `Dump` needs ELF and `ar` support beside its PE support. +- **Facts are read only for static links and the JIT, never for DLL links.** A program linked + against the default library's DLL (or `.so`) loads whatever copy sits beside it at run time, + which may not be the copy read at compile time. Such programs get no facts: every call into the + library stays an escape, as today. Under the JIT the library read is the library loaded. + +## 3. Goals, non-goals, anti-goals + +Goals +- A program compiled against a default library built with this change promotes objects passed to + default-library functions whose parameter is only borrowed. +- The facts survive any rebuild of the library: they are regenerated with it. + +Non-goals +- Hand-written facts, for C functions or anything else. +- Facts about results (a result that borrows an argument, or a fresh result): that is project (a). +- Changing what the default library's API looks like. + +Anti-goals (failure even if it works) +- Any fact that claims a parameter is borrowed when the body may keep it. +- Facts read from a binary other than the one the program is linked against. +- A compile that fails, or warns, because the facts are missing or unreadable. Missing facts mean + "escape", exactly as today. + +## 4. Feasibility notes + +### 4.1 What exists + +- `StackEscapeFacts` (`tslang/lib/TypeScript/StackEscapeFacts.h`) already computes, for every + function in a module, whether each parameter may escape (`paramEscapes`, a module fixpoint). An + unknown callee is an escape; a virtual call is trusted only when every vtable holding that slot + is local to the module. +- A library already exports text facts for importers: own's `__tsown__` symbol (a C + string, one name per line), emitted by `OwnershipSignaturePass` (`SHARED_LIB_OWN_FACTS`, + `Defines.h`). It is read by `MLIRGenModule.cpp` when importing a shared library: the symbol list + comes from `Dump::getSymbols`, the text from `Dump::readExportedCString` (or from the loaded + library under the JIT), and `addImportedOwnNoDrops` merges it into a module attribute. +- The default library reaches a program as hand-written declaration files (`lib.d.ts` and the files + it references, in TypeScriptCompilerDefaultLib/src), and as a per-model binary under + `{lib,dll}//////` (`tslang --print-default-lib-dir`). Generic + code (`lib.generics.ts`) is compiled into the program with its bodies, so it needs no facts. + +### 4.2 What the default library's API can actually gain + +Most non-generic default-library functions take numbers and strings, and strings are not +promotion candidates. The arguments that could move to the stack are `RegExp` objects +(`s.replace(new RegExp(...), x)`), rest-parameter arrays built at the call site +(`concat(...other: string[])`; promotable under gc/none only, since the elements are strings), and +small option objects (`toLocaleString(locale, { timeZone })`). + +## 5. Design + +### 5.1 Producing the facts (library side) + +When a module is built as a library (`--emit=dll`, `--emit=lib`, and the object files the default +library build links), the compiler emits one exported symbol per module, in the style of +`__tsown_`. Suggested name: `__tsborrow__`, using the same suffix the other markers use. +Its value is a C string with one line per exported function that has at least one borrowed +parameter: the function's symbol name, then the indexes of its borrowed parameters (`this` is +parameter 0 of a method). The exact text format is the builder's call. + +- The facts come from `StackEscapeFacts` run over the library module, at the same point stack + promotion runs. They are emitted whether or not `--no-stack-promotion` is given: the flag decides + what this module promotes, not what it promises importers. +- **Not tied to the `__tsmm_` marker (my call).** The default library's two builds emit no marker + today: the DLL uses `--embed-declarations=false`, and the archive's object uses `--export=none` + (DefaultLib `scripts/build_core.bat`). So the facts symbol is emitted on its own, for every + function that is not private, whatever the export settings. +- **In a library build every class counts as exported (my call).** An importer can subclass any of + them, even under `--export=none`, so the library's own analysis never trusts a virtual call + through one of its classes' vtables. Otherwise a program subclass passed into the library could + break a fact. +- Only functions that are not private get facts. A parameter is listed only when `paramEscapes` says it does + not escape. Everything unlisted is an escape to the reader. +- A method of an exported class is listed under its own symbol. Whether a virtual call to it may be + trusted is the reader's question (§5.3). + +### 5.2 Reading the facts (program side) + +When a program is compiled with the default library, the compiler opens the binary it will link: +the DLL under the JIT, and the static library it links ahead of time. An ahead-of-time link +against the DLL reads nothing (§2). It reads every +`__tsborrow_` symbol and records the facts on the module, as `addImportedOwnNoDrops` does for own, +for `StackEscapeFacts` to read. + +- **Same binary.** The facts must come from the exact file the link uses. If the AOT link takes a + static archive, the archive is read: `Dump` gains whatever archive support that needs. Reading + the DLL beside it instead is not allowed, since the two could come from different builds. +- **Missing or unreadable facts are not an error.** No symbol, an old library, a file that cannot + be read: no facts, every call an escape, as today. No warning. +- **JIT cache.** The facts read become part of what a JIT cache entry depends on, like the default + library path already is (`jitcache.cpp`). Otherwise a rebuilt library would leave stale + promotions in cached objects. + +### 5.3 Using the facts + +In `StackEscapeFacts`: +- **A direct call** (`ts.SymbolCallInternal`) to a declared function (no body) whose symbol has an + imported fact: the argument at a listed index does not escape through the call. Today + `argumentEscapes` returns `unknown-callee` there. +- **A call through a `ts.SymbolRef`** of such a function: the same. +- **A virtual call on a library class stays an escape in version 1 (my call).** A static + library or an object file given to the link can subclass a library class, and MLIRGen cannot see + those inputs. Trusting such calls needs a way to detect them, which is left for later. +- Everything else is unchanged: an unlisted parameter, an unknown function, a call through a value. + +### 5.3a When no facts are read (my call) + +`--emit=obj`, `--emit=llvm` and `--emit=bc` do not link, so the compiler cannot know which library +the object will be linked with (the CMake template and the default library's own build work this +way). They read no facts. Facts are read only when tslang itself links or loads the library. The +choice of file - lib or dll, debug or release, Android - is made today in `exe.cpp` and `jit.cpp`, +after MLIRGen. The builder moves that choice to a function MLIRGen can call, so both read the same +answer. + +### 5.4 Static libraries given with `--lib`, never `-shared` imports (my call for `--lib`) + +A `-shared` import is a DLL link and reads no facts (§2, the user's rule). + +The reader also runs on every static library given to an `--emit=exe` link with `--lib`: the file +read is the file linked, which is the same guarantee the default library's archive gives. This is +how the tests get a library whose facts they control. The runner builds every test with +`--no-default-lib`, so a default-library-only reader could not be tested in the ordinary suite. A +test builds a small library with `--emit=lib`, then compiles a program against it with +`--emit=exe --lib=`, declaring the library's functions with `declare`. + +## 6. Soundness + +- A fact is a property of the library's body, computed by the analysis that already promotes + objects inside that library. It needs no closed world. If any caller anywhere passes a value the + body keeps, the parameter is already an escape. +- Inside the library, calls through function values, unknown callees and exported classes' virtual + methods are already escapes, so a library function that hands its argument to a user callback + never claims to borrow it. +- The only new trust is that the binary read is the binary linked (§5.2). + +## 7. Testing + +- **Producer:** a library module with functions that borrow, keep, return and hand to a callback + their parameters. Check its emitted symbol text, by reading the built binary or by a dump flag of + the builder's choosing. Every case gets a positive and a negative. +- **Consumer:** a program linking that library with `--emit=exe --lib=`, with a decision test per + case (the remark is printed during the `--emit=exe` compile): the + `--stack-promotion-report` remark says `stack` for a borrowed argument and `heap: argument` or + `heap: unknown-callee` otherwise. Run tests under gc, rc, own and none. +- **Stale or missing facts:** a program against a library built without facts (an old build, or + the symbol stripped) still compiles and promotes nothing through those calls. +- **No facts from DLLs:** the same library built with `--emit=dll` and imported as `-shared`: every + call stays `heap`. Likewise `--emit=obj` of the program reads nothing. +- **Virtual-call rule:** a library class with a virtual method; one program linking only that + library, and one also linking a second library that subclasses it. Expected: stack in the + first, heap in the second. +- **Default library end to end:** a program using `new RegExp` with `replace`, compiled with the + default library (outside the runner's `--no-default-lib` suite, as the default-library tests + already are). Check the remark and the output. +- **Teeth:** make the producer list every parameter, and confirm the keeper test then reads a + dangling pointer. + +## 8. Left to the builder + +- The symbol's name and text format. +- How `Dump` reads a static archive. +- Where in the pipeline the facts are emitted (the stack promotion pass, or a pass of its own + sharing `StackEscapeFacts`). +- The JIT cache key detail. +- The default-library build changes, if its build does not already export the new symbol. +- Phasing into PRs, including the DefaultLib repository. diff --git a/tslang/include/TypeScript/DataStructs.h b/tslang/include/TypeScript/DataStructs.h index 80c698d7c..54a4e7a86 100644 --- a/tslang/include/TypeScript/DataStructs.h +++ b/tslang/include/TypeScript/DataStructs.h @@ -67,6 +67,12 @@ struct CompileOptions // shared libraries imported, whose declarations the module was generated against. std::vector sourceImports; std::vector sharedLibraryImports; + // The borrow facts (__tsborrow_ texts, concatenated) of the libraries this program is linked + // statically against or, under the JIT, loads: see + // docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md. Empty: none. + std::string importedBorrowFacts; + // the files they were read from: a JIT cache entry depends on them + std::vector importedBorrowFactsFiles; // an import led back to a file still being generated: the modules in the cycle cannot be // compiled one by one (the JIT cache then compiles the program as one module) bool importCycle = false; diff --git a/tslang/include/TypeScript/Defines.h b/tslang/include/TypeScript/Defines.h index 9ba2f9668..77b177c48 100644 --- a/tslang/include/TypeScript/Defines.h +++ b/tslang/include/TypeScript/Defines.h @@ -212,6 +212,13 @@ // under own report more, while the facts a callee's body relies on cannot be seen by an importer // that re-parses source, and would free twice there. #define SHARED_LIB_OWN_FACTS "__tsown_" +// Which parameters of a module's functions their bodies only borrow, for stack promotion in a +// program linked statically against it: `__tsborrow__`, a NUL-terminated char array +// (not a pointer, so it can be read from an object file without relocations), one line per +// function: ` ...`. See docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md. +#define SHARED_LIB_BORROW_FACTS "__tsborrow_" +// [symbol name, text]: set by BorrowFactsExportPass, made into the symbol by the LLVM lowering +#define SHARED_LIB_BORROW_FACTS_ATTR_NAME "ts.borrow_facts" // a module attribute, so it has to carry the dialect's prefix #define SHARED_LIB_OWN_NO_DROPS_ATTR_NAME "ts.own_imported_no_drops" diff --git a/tslang/include/TypeScript/MLIRGen.h b/tslang/include/TypeScript/MLIRGen.h index e724ae552..3a08a93f7 100644 --- a/tslang/include/TypeScript/MLIRGen.h +++ b/tslang/include/TypeScript/MLIRGen.h @@ -24,6 +24,10 @@ class StringRef; namespace typescript { +// `_`: the suffix of a module's exported marker symbols +// (`__tsmm_`, `__tsown_`, `__tsborrow_`), unique per source file. +::std::string moduleSymbolSuffix(llvm::StringRef filePath); + ::std::string dumpFromSource(const llvm::StringRef &, const llvm::StringRef &); mlir::OwningOpRef mlirGenFromMainSource( diff --git a/tslang/include/TypeScript/ObjDumper.h b/tslang/include/TypeScript/ObjDumper.h index 5442e5325..0f8915f94 100644 --- a/tslang/include/TypeScript/ObjDumper.h +++ b/tslang/include/TypeScript/ObjDumper.h @@ -94,6 +94,18 @@ namespace Dump // architecture, which this process cannot load. std::nullopt when the file is not a PE image, // does not export `symbol`, or the pointer or string lies outside the file's data. std::optional readExportedCString(llvm::StringRef path, llvm::StringRef symbol); + + // Every defined symbol starting with `prefix` in an object file (COFF or ELF), in the object + // members of a static library, among a DLL's exports or an ELF shared object's symbols, with the + // NUL-terminated char array it names: (symbol, text). Read from the file, nothing loaded. + // Members that are not objects (import members, bitcode) and an unreadable file give nothing. + std::vector> readCStringSymbols(llvm::StringRef path, + llvm::StringRef prefix); + + // The global symbols a link input defines: a static library's symbol table, an object file's + // defined globals, a DLL's exports, a shared object's defined dynamic symbols. Empty for a file + // that cannot be read. + std::vector definedSymbols(llvm::StringRef path); } std::unique_ptr createCOFFDumper(const COFFObjectFile &); diff --git a/tslang/include/TypeScript/Passes.h b/tslang/include/TypeScript/Passes.h index 56708e661..c41180cb5 100644 --- a/tslang/include/TypeScript/Passes.h +++ b/tslang/include/TypeScript/Passes.h @@ -61,6 +61,11 @@ struct StackPromotionLimits /// model. Runs on the whole module at the affine level, after own's inference. std::unique_ptr createStackPromotionPass(CompileOptions &, StackPromotionLimits); +/// Writes the module's borrow facts - which parameters of its functions their bodies only borrow - +/// into the exported char array `symbolName` (`__tsborrow__`), for programs linked +/// statically against it. See docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md. +std::unique_ptr createBorrowFactsExportPass(std::string symbolName); + /// GC Pass to replace malloc, realloc, free with GC_malloc, GC_realloc, GC_free std::unique_ptr createGCPass(CompileOptions&); std::unique_ptr createProcessHeapPass(); diff --git a/tslang/lib/TypeScript/BorrowFactsExportPass.cpp b/tslang/lib/TypeScript/BorrowFactsExportPass.cpp new file mode 100644 index 000000000..808720870 --- /dev/null +++ b/tslang/lib/TypeScript/BorrowFactsExportPass.cpp @@ -0,0 +1,92 @@ +#include "mlir/Pass/Pass.h" +#include "mlir/IR/BuiltinOps.h" +#include "mlir/Dialect/LLVMIR/LLVMDialect.h" + +#include "TypeScript/TypeScriptDialect.h" +#include "TypeScript/TypeScriptOps.h" +#include "TypeScript/Passes.h" +#include "TypeScript/Defines.h" +#include "TypeScript/MLIRLogic/MLIRTypeHelper.h" + +#include "StackEscapeFacts.h" + +#include "llvm/ADT/SmallVector.h" + +#define DEBUG_TYPE "borrow-facts-export" + +namespace mlir_ts = mlir::typescript; + +namespace +{ + +// Borrow facts (docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md): what this +// module promises a program linked statically against it - for each function another module can +// call, the parameters its body only borrows. Computed by the analysis stack promotion runs, but +// trusting no vtable: an importer may subclass any class and override anything. Written whether +// or not this module promotes anything itself, and whatever it exports. +class BorrowFactsExportPass : public mlir::PassWrapper> +{ + std::string symbolName; + + public: + MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(BorrowFactsExportPass) + + explicit BorrowFactsExportPass(std::string symbolName) : symbolName(std::move(symbolName)) + { + } + + void runOnOperation() override + { + auto module = getOperation(); + stack_escape::StackEscapeFacts facts(module, /*trustLocalVTables=*/false); + + llvm::SmallVector> lines; + module.walk([&](mlir_ts::FuncOp funcOp) { + // renamed by @dllname / @linkname: it defines another symbol than its name, which an + // importer could not tell apart + if (funcOp.isDeclaration() || funcOp.isPrivate() || funcOp->hasAttr(DLL_NAME)) + { + return; + } + + std::string indexes; + for (auto index : llvm::seq(0, funcOp.getBody().getNumArguments())) + { + if (facts.paramBorrowed(funcOp, index)) + { + indexes += " " + std::to_string(index); + } + } + + if (!indexes.empty()) + { + lines.push_back({funcOp.getSymName(), indexes}); + } + }); + + if (lines.empty()) + { + return; + } + + llvm::sort(lines); + std::string text; + for (auto &[name, indexes] : lines) + { + text += name.str() + indexes + "\n"; + } + + // The LLVM lowering turns this into the symbol (emitBorrowFacts): an LLVM global in the + // module before then upsets its lookups of the globals it converts. + mlir::Builder builder(module.getContext()); + module->setAttr(SHARED_LIB_BORROW_FACTS_ATTR_NAME, + builder.getArrayAttr({builder.getStringAttr(symbolName), builder.getStringAttr(text)})); + } +}; + +} // namespace + +std::unique_ptr mlir_ts::createBorrowFactsExportPass(std::string symbolName) +{ + return std::make_unique(std::move(symbolName)); +} diff --git a/tslang/lib/TypeScript/CMakeLists.txt b/tslang/lib/TypeScript/CMakeLists.txt index 2009f5700..dbfc61897 100644 --- a/tslang/lib/TypeScript/CMakeLists.txt +++ b/tslang/lib/TypeScript/CMakeLists.txt @@ -36,6 +36,7 @@ add_mlir_dialect_library(MLIRTypeScript OwnershipSignaturePass.cpp OwnedReturnConsumptionPass.cpp StackPromotionPass.cpp + BorrowFactsExportPass.cpp GCPass.cpp ProcessHeapPass.cpp InlinedLocationPass.cpp diff --git a/tslang/lib/TypeScript/LowerToLLVM.cpp b/tslang/lib/TypeScript/LowerToLLVM.cpp index 80c09d15c..010a63eed 100644 --- a/tslang/lib/TypeScript/LowerToLLVM.cpp +++ b/tslang/lib/TypeScript/LowerToLLVM.cpp @@ -8373,6 +8373,30 @@ static LogicalResult cleanupUnrealizedConversionCast(mlir::ModuleOp &module) // linker itself, which is how this went unnoticed once already. A JIT run has no C runtime reading // a return register, and a DLL's `main` is not an entry point and may be something an importer // resolves, so those two are the exclusions. +// The module's borrow facts (BorrowFactsExportPass) as the exported char array importers read: a +// char array, NUL included, so its bytes sit in the object file where the symbol points. +// ExportFixPass makes it dllexport on Windows, so the JIT finds it in the default library's DLL. +static void emitBorrowFacts(mlir::ModuleOp m) +{ + auto facts = m->getAttrOfType(SHARED_LIB_BORROW_FACTS_ATTR_NAME); + if (!facts) + { + return; + } + + m->removeAttr(SHARED_LIB_BORROW_FACTS_ATTR_NAME); + auto name = mlir::cast(facts[0]).getValue(); + auto text = mlir::cast(facts[1]).getValue().str(); + text.push_back(char(0)); + + mlir::OpBuilder builder(m.getBodyRegion()); + builder.setInsertionPointToEnd(m.getBody()); + auto arrayType = LLVM::LLVMArrayType::get(builder.getI8Type(), text.size()); + auto globalOp = builder.create(m.getLoc(), arrayType, /*isConstant=*/true, LLVM::Linkage::External, + name, builder.getStringAttr(text)); + globalOp.setSection(llvm::StringRef(DLL_EXPORT)); +} + static void giveEntryPointAnExitCode(mlir::ModuleOp m, CompileOptions &compileOptions) { if (compileOptions.isJit || compileOptions.isDLL) @@ -8716,6 +8740,8 @@ void TypeScriptToLLVMLoweringPass::runOnOperation() giveEntryPointAnExitCode(m, tsContext.compileOptions); + emitBorrowFacts(m); + LLVM_DEBUG(llvm::dbgs() << "\n!! AFTER DUMP: \n" << m << "\n";); LLVM_DEBUG(verifyModule(m);); diff --git a/tslang/lib/TypeScript/MLIRGen.cpp b/tslang/lib/TypeScript/MLIRGen.cpp index d38c30293..0c6189d7a 100644 --- a/tslang/lib/TypeScript/MLIRGen.cpp +++ b/tslang/lib/TypeScript/MLIRGen.cpp @@ -25,6 +25,11 @@ void SourceMgrDiagnosticHandlerEx::emit(mlir::Diagnostic &diag) namespace typescript { +::std::string moduleSymbolSuffix(llvm::StringRef filePath) +{ + return mlirgen::MLIRGenImpl::moduleSymbolSuffix(filePath); +} + ::std::string dumpFromSource(const llvm::StringRef &fileName, const llvm::StringRef &source) { auto showLineCharPos = false; diff --git a/tslang/lib/TypeScript/ObjDumper.cpp b/tslang/lib/TypeScript/ObjDumper.cpp index 171795464..00adffe88 100644 --- a/tslang/lib/TypeScript/ObjDumper.cpp +++ b/tslang/lib/TypeScript/ObjDumper.cpp @@ -100,9 +100,262 @@ static std::optional> imageBytesAt(const COFFObjectFile &coffO return std::nullopt; } +// readCStringSymbols for one object file or ELF shared object: each defined symbol starting with +// `prefix`, its text read from its section's contents at its offset, up to the first NUL or the +// section's end. +template +static void readCStringSymbolsOf(const ObjectFile &objFile, Symbols symbols, StringRef prefix, + std::vector> &result) +{ + for (auto symbol : symbols) + { + auto name = symbol.getName(); + if (!name) + { + consumeError(name.takeError()); + continue; + } + + if (!name->starts_with(prefix)) + { + continue; + } + + auto section = symbol.getSection(); + if (!section) + { + consumeError(section.takeError()); + continue; + } + + if (*section == objFile.section_end()) + { + // undefined, or absolute + continue; + } + + auto contents = (*section)->getContents(); + if (!contents) + { + consumeError(contents.takeError()); + continue; + } + + // in a relocatable object a symbol's value is its offset in its section; in a shared + // object, its address + auto value = symbol.getValue(); + if (!value) + { + consumeError(value.takeError()); + continue; + } + + auto offset = *value; + if (!objFile.isRelocatableObject()) + { + auto sectionAddress = (*section)->getAddress(); + if (offset < sectionAddress) + { + continue; + } + + offset -= sectionAddress; + } + + if (offset >= contents->size()) + { + continue; + } + + auto text = contents->drop_front(offset); + result.push_back({name->str(), text.substr(0, text.find(char(0))).str()}); + } +} + namespace Dump { +template +static void definedSymbolsOf(Symbols symbols, std::vector &result) +{ + for (auto symbol : symbols) + { + auto flags = symbol.getFlags(); + if (!flags) + { + consumeError(flags.takeError()); + continue; + } + + if (!(*flags & SymbolRef::SF_Global) || (*flags & SymbolRef::SF_Undefined)) + { + continue; + } + + auto name = symbol.getName(); + if (!name) + { + consumeError(name.takeError()); + continue; + } + + result.push_back(name->str()); + } +} + +std::vector definedSymbols(StringRef path) +{ + std::vector result; + auto expectedOwningBinary = createBinary(path); + if (!expectedOwningBinary) + { + consumeError(expectedOwningBinary.takeError()); + return result; + } + + auto *binary = expectedOwningBinary.get().getBinary(); + if (auto *archive = dyn_cast(binary)) + { + for (auto &symbol : archive->symbols()) + { + result.push_back(symbol.getName().str()); + } + + return result; + } + + if (auto *coffObj = dyn_cast(binary); coffObj && (coffObj->getPE32Header() || coffObj->getPE32PlusHeader())) + { + for (const ExportDirectoryEntryRef &entry : coffObj->export_directories()) + { + StringRef name; + if (Error err = entry.getSymbolName(name)) + { + consumeError(std::move(err)); + continue; + } + + result.push_back(name.str()); + } + + return result; + } + + if (auto *elfObj = dyn_cast(binary)) + { + definedSymbolsOf(elfObj->symbols(), result); + if (!elfObj->isRelocatableObject()) + { + definedSymbolsOf(elfObj->getDynamicSymbolIterators(), result); + } + + return result; + } + + if (auto *objFile = dyn_cast(binary)) + { + definedSymbolsOf(objFile->symbols(), result); + } + + return result; +} + +std::vector> readCStringSymbols(StringRef path, StringRef prefix) +{ + std::vector> result; + auto expectedOwningBinary = createBinary(path); + if (!expectedOwningBinary) + { + consumeError(expectedOwningBinary.takeError()); + return result; + } + + auto *binary = expectedOwningBinary.get().getBinary(); + if (auto *coffObj = dyn_cast(binary); coffObj && (coffObj->getPE32Header() || coffObj->getPE32PlusHeader())) + { + // a PE image (a DLL): no symbol table, only its exports + for (const ExportDirectoryEntryRef &entry : coffObj->export_directories()) + { + StringRef name; + if (Error err = entry.getSymbolName(name)) + { + consumeError(std::move(err)); + continue; + } + + bool forwarder = false; + if (Error err = entry.isForwarder(forwarder)) + { + consumeError(std::move(err)); + continue; + } + + uint32_t rva = 0; + if (!name.starts_with(prefix) || forwarder) + { + continue; + } + + if (Error err = entry.getExportRVA(rva)) + { + consumeError(std::move(err)); + continue; + } + + if (auto bytes = imageBytesAt(*coffObj, rva)) + { + StringRef text(reinterpret_cast(bytes->data()), bytes->size()); + result.push_back({name.str(), text.substr(0, text.find(char(0))).str()}); + } + } + + return result; + } + + if (auto *elfObj = dyn_cast(binary)) + { + // a shared object exports through its dynamic symbols; it may have no other table + readCStringSymbolsOf(*elfObj, elfObj->symbols(), prefix, result); + if (!elfObj->isRelocatableObject()) + { + readCStringSymbolsOf(*elfObj, elfObj->getDynamicSymbolIterators(), prefix, result); + } + + return result; + } + + if (auto *objFile = dyn_cast(binary)) + { + readCStringSymbolsOf(*objFile, objFile->symbols(), prefix, result); + return result; + } + + auto *archive = dyn_cast(binary); + if (!archive) + { + return result; + } + + Error err = Error::success(); + for (auto &child : archive->children(err)) + { + auto childBinary = child.getAsBinary(); + if (!childBinary) + { + consumeError(childBinary.takeError()); + continue; + } + + if (auto *objFile = dyn_cast(childBinary->get())) + { + readCStringSymbolsOf(*objFile, objFile->symbols(), prefix, result); + } + } + + consumeError(std::move(err)); + return result; +} + void getSymbols(StringRef filePath, SmallVector &symbols, llvm::BumpPtrAllocator &stringAllocator) { auto expectedOwningBinary = createBinary(filePath); diff --git a/tslang/lib/TypeScript/StackEscapeFacts.h b/tslang/lib/TypeScript/StackEscapeFacts.h index 5be3b92cd..5a7e05320 100644 --- a/tslang/lib/TypeScript/StackEscapeFacts.h +++ b/tslang/lib/TypeScript/StackEscapeFacts.h @@ -2,6 +2,7 @@ #define MLIR_TYPESCRIPT_STACKESCAPEFACTS_H #include "TypeScript/TypeScriptOps.h" +#include "TypeScript/Defines.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Interfaces/CallInterfaces.h" @@ -30,10 +31,54 @@ namespace stack_escape namespace mlir_ts = mlir::typescript; +// Facts another module published (its __tsborrow_ text): function symbol -> the indexes of the +// parameters its body only borrows. +using BorrowFactsMap = llvm::StringMap>; + +// ` ...` per line, as BorrowFactsExportPass writes it. A line that does not parse +// says nothing: every parameter of a function no line names may escape. +inline BorrowFactsMap parseBorrowFacts(llvm::StringRef text) +{ + BorrowFactsMap facts; + llvm::SmallVector lines; + text.split(lines, '\n', -1, /*KeepEmpty=*/false); + for (auto line : lines) + { + llvm::SmallVector words; + line.trim().split(words, ' ', -1, /*KeepEmpty=*/false); + if (words.size() < 2) + { + continue; + } + + llvm::SmallVector indexes; + for (auto word : llvm::drop_begin(words)) + { + unsigned index; + if (word.getAsInteger(10, index)) + { + indexes.clear(); + break; + } + + indexes.push_back(index); + } + + auto &listed = facts[words[0]]; + listed.append(indexes.begin(), indexes.end()); + } + + return facts; +} + class StackEscapeFacts { public: - explicit StackEscapeFacts(mlir::ModuleOp module) : module(module) + // trustLocalVTables: whether a vtable this module keeps to itself lists every method a virtual + // call may reach. Not when the facts are for importers, who may subclass any class. + explicit StackEscapeFacts(mlir::ModuleOp module, bool trustLocalVTables = true, + const BorrowFactsMap *imported = nullptr) + : module(module), trustLocalVTables(trustLocalVTables), imported(imported) { collectVTables(); @@ -112,6 +157,13 @@ class StackEscapeFacts return recursive->contains(funcOp); } + // A parameter of a function with a body that nothing done with it lets outlive the call. + bool paramBorrowed(mlir_ts::FuncOp funcOp, unsigned index) const + { + auto it = paramEscapes.find({funcOp, index}); + return it != paramEscapes.end() && !it->second; + } + // Whether argument `argIndex` of a call through `candidates` may outlive the call: empty when // every candidate's parameter stays inside it. llvm::StringRef argumentEscapes(llvm::ArrayRef candidates, unsigned argIndex) @@ -119,6 +171,14 @@ class StackEscapeFacts for (auto candidate : candidates) { auto funcOp = module.lookupSymbol(candidate); + // declared here, defined in a library linked statically that published its facts - by + // the name it binds: one renamed by @dllname / @linkname binds another symbol + if (funcOp && funcOp.getBody().empty() && !funcOp->hasAttr(DLL_NAME) && + importedBorrows(candidate.getValue(), argIndex)) + { + continue; + } + if (!funcOp || funcOp.getBody().empty() || argIndex >= funcOp.getBody().getNumArguments()) { return "unknown-callee"; @@ -134,7 +194,20 @@ class StackEscapeFacts } private: + bool importedBorrows(llvm::StringRef symbol, unsigned argIndex) const + { + if (!imported) + { + return false; + } + + auto it = imported->find(symbol); + return it != imported->end() && llvm::is_contained(it->second, argIndex); + } + mlir::ModuleOp module; + bool trustLocalVTables; + const BorrowFactsMap *imported; std::optional> recursive; // Every function on a cycle of the call graph, in one pass (Tarjan's strongly connected @@ -322,7 +395,7 @@ class StackEscapeFacts auto &vtable = vtables[name.drop_back(strlen("..vtbl"))]; auto linkage = globalOp->getAttrOfType("linkage"); - vtable.local = linkage && (linkage.getLinkage() == mlir::LLVM::Linkage::Private || + vtable.local = trustLocalVTables && linkage && (linkage.getLinkage() == mlir::LLVM::Linkage::Private || linkage.getLinkage() == mlir::LLVM::Linkage::Internal); globalOp.walk([&](mlir_ts::InsertPropertyOp insertOp) { auto symbolRef = insertOp.getValue().getDefiningOp(); diff --git a/tslang/lib/TypeScript/StackPromotionPass.cpp b/tslang/lib/TypeScript/StackPromotionPass.cpp index bacca5d45..037debd65 100644 --- a/tslang/lib/TypeScript/StackPromotionPass.cpp +++ b/tslang/lib/TypeScript/StackPromotionPass.cpp @@ -9,6 +9,8 @@ #include "TypeScript/Defines.h" #include "TypeScript/MLIRLogic/MLIRTypeHelper.h" +#include "TypeScript/ObjDumper.h" + #include "StackEscapeFacts.h" #include "llvm/ADT/MapVector.h" @@ -39,6 +41,8 @@ class StackPromotionPass : public mlir::PassWrapper> loopInfos; llvm::DenseMap tryRegions; @@ -49,8 +53,25 @@ class StackPromotionPass : public mlir::PassWrapper --emit=llvm --no-default-lib -mm=${stack_mm} -o - + "${PROJECT_SOURCE_DIR}/test/tester/stack-promotion/facts_lib.ts") + set_tests_properties(test-stack-facts-lib-${stack_mm} + PROPERTIES PASS_REGULAR_EXPRESSION "${stack_facts_listed}" + FAIL_REGULAR_EXPRESSION "${stack_facts_unlisted}") +endforeach() +# -mm=own refuses a function that keeps or returns its parameter, so its library has the rest +add_test(NAME test-stack-facts-lib-own + COMMAND $ --emit=llvm --no-default-lib -mm=own -o - + "${PROJECT_SOURCE_DIR}/test/tester/stack-promotion/facts_lib_own.ts") +set_tests_properties(test-stack-facts-lib-own + PROPERTIES PASS_REGULAR_EXPRESSION "@__tsborrow_facts_lib_own_[0-9]+ = (dllexport )?constant [[][0-9]+ x i8[]] c\"[^\"]*Geo[.]len 0.0A[^\"]*sumOf 0.0A" + FAIL_REGULAR_EXPRESSION "${stack_facts_unlisted}") +# what the module promises importers, whatever it promotes itself or exports +add_test(NAME test-stack-facts-lib-no-promotion + COMMAND $ --emit=llvm --no-default-lib --no-stack-promotion --export=none -o - + "${PROJECT_SOURCE_DIR}/test/tester/stack-promotion/facts_lib.ts") +set_tests_properties(test-stack-facts-lib-no-promotion + PROPERTIES PASS_REGULAR_EXPRESSION "${stack_facts_listed}" + FAIL_REGULAR_EXPRESSION "${stack_facts_unlisted}") +# a library built before facts existed: no symbol at all +add_test(NAME test-stack-facts-lib-suppressed + COMMAND $ --emit=llvm --no-default-lib --no-borrow-facts-export -o - + "${PROJECT_SOURCE_DIR}/test/tester/stack-promotion/facts_lib.ts") +set_tests_properties(test-stack-facts-lib-suppressed + PROPERTIES PASS_REGULAR_EXPRESSION "define [a-z ]*double @sumOf" + FAIL_REGULAR_EXPRESSION "__tsborrow_|Stack dump|Assertion failed") +# ...read back from the object, a static library holding it, and not from an import library; and +# read by a program linked against the static library +foreach(stack_mm gc rc own none) + add_test(NAME test-borrow-facts-${stack_mm} + COMMAND ${CMAKE_COMMAND} + "-DTSLANG=$" + "-DSOURCE_DIR=${PROJECT_SOURCE_DIR}/test/tester/stack-promotion" + "-DWORK_DIR=${CMAKE_CURRENT_BINARY_DIR}/borrow-facts" + "-DAR=${CMAKE_AR}" + "-DMM=${stack_mm}" + "-DGC_LIB=${TEST_GC_LIBDIR}" + "-DLLVM_LIB=${LLVM_LIBRARY_DIR}" + "-DTSLANG_LIB=${CMAKE_BINARY_DIR}/lib" + -P "${CMAKE_CURRENT_SOURCE_DIR}/borrow-facts.cmake") +endforeach() +# ...and from the real default library, when one built with facts is there +add_test(NAME test-borrow-facts-defaultlib + COMMAND ${CMAKE_COMMAND} + "-DTSLANG=$" + "-DSOURCE_DIR=${PROJECT_SOURCE_DIR}/test/tester/stack-promotion" + "-DWORK_DIR=${CMAKE_CURRENT_BINARY_DIR}/borrow-facts-defaultlib" + "-DAR=${CMAKE_AR}" + "-DGC_LIB=${TEST_GC_LIBDIR}" + "-DLLVM_LIB=${LLVM_LIBRARY_DIR}" + "-DTSLANG_LIB=${CMAKE_BINARY_DIR}/lib" + "-DDEFAULT_LIB_CANDIDATES=${PROJECT_SOURCE_DIR}/../TypeScriptCompilerDefaultLib/__build;${PROJECT_SOURCE_DIR}/../../TypeScriptCompilerDefaultLib/__build" + -P "${CMAKE_CURRENT_SOURCE_DIR}/borrow-facts-defaultlib.cmake") +set_tests_properties(test-borrow-facts-defaultlib PROPERTIES SKIP_REGULAR_EXPRESSION "SKIPPED:") # The same site is a stack site where nothing counts, and a heap site where a Shared count would see it. foreach(stack_case "gc|stack" "none|stack" "rc|heap: shared-handle" "own|heap: shared-handle") string(FIND "${stack_case}" "|" stack_sep) diff --git a/tslang/test/tester/borrow-facts-defaultlib.cmake b/tslang/test/tester/borrow-facts-defaultlib.cmake new file mode 100644 index 000000000..7f3e47d7c --- /dev/null +++ b/tslang/test/tester/borrow-facts-defaultlib.cmake @@ -0,0 +1,135 @@ +# Borrow facts from the real default library (docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md): +# `Math.max(a, b, c)` builds its rest array at the call, and the default library lists that parameter +# as only read, so the array goes on the stack - read from the static library an exe links, and from +# the DLL the JIT loads. A JIT cache entry depends on that DLL. +# +# The default library is a separate build (TypeScriptCompilerDefaultLib), checked per model: one +# built before borrow facts existed publishes none and is passed over. With none to check the test +# prints SKIPPED. + +cmake_minimum_required(VERSION 3.17.3) + +foreach(var TSLANG SOURCE_DIR WORK_DIR AR GC_LIB LLVM_LIB TSLANG_LIB DEFAULT_LIB_CANDIDATES) + if(NOT DEFINED ${var}) + message(FATAL_ERROR "${var} is required") + endif() +endforeach() + +set(ENV{GC_LIB_PATH} "") +set(ENV{TSLANG_LIB_PATH} "") + +set(default_lib "") +foreach(candidate "$ENV{DEFAULT_LIB_PATH}" ${DEFAULT_LIB_CANDIDATES}) + if(NOT candidate STREQUAL "" AND IS_DIRECTORY "${candidate}/defaultlib") + set(default_lib "${candidate}") + break() + endif() +endforeach() +set(ENV{DEFAULT_LIB_PATH} "") + +if(default_lib STREQUAL "") + message("SKIPPED: no default library built (set DEFAULT_LIB_PATH to the folder holding 'defaultlib')") + return() +endif() + +if(WIN32) + set(static_name "TypeScriptDefaultLib.lib") + set(shared_name "TypeScriptDefaultLib.dll") + set(run_env "PATH=${GC_LIB}/../bin;$ENV{PATH}") +else() + set(static_name "libTypeScriptDefaultLib.a") + set(shared_name "libTypeScriptDefaultLib.so") + set(run_env "LD_LIBRARY_PATH=$ENV{LD_LIBRARY_PATH}") +endif() + +# run( ) - fails unless it exits 0 +function(run dir what out_var) + execute_process(COMMAND ${CMAKE_COMMAND} -E env "${run_env}" ${ARGN} + WORKING_DIRECTORY "${dir}" + OUTPUT_VARIABLE out + ERROR_VARIABLE err + RESULT_VARIABLE status) + if(NOT status EQUAL 0 OR "${out}${err}" MATCHES "Stack dump|Assertion failed") + message(FATAL_ERROR "${what}: exit ${status}\n${out}\n${err}") + endif() + set(${out_var} "${out}${err}" PARENT_SCOPE) +endfunction() + +set(source "${SOURCE_DIR}/facts_defaultlib.ts") +set(promoted "facts_defaultlib[.]ts:5:21: remark: stack-promotion: stack") +set(checked 0) + +foreach(mm gc rc own none) + execute_process(COMMAND "${TSLANG}" --print-default-lib-dir=lib -mm=${mm} + OUTPUT_VARIABLE lib_subdir OUTPUT_STRIP_TRAILING_WHITESPACE) + execute_process(COMMAND "${TSLANG}" --print-default-lib-dir=dll -mm=${mm} + OUTPUT_VARIABLE dll_subdir OUTPUT_STRIP_TRAILING_WHITESPACE) + set(static_lib "${default_lib}/${lib_subdir}/${static_name}") + set(shared_lib "${default_lib}/${dll_subdir}/${shared_name}") + if(NOT EXISTS "${static_lib}" OR NOT EXISTS "${shared_lib}") + continue() + endif() + + # The default library is linked by the file its facts were read from, never searched for: a + # stray library of that name in the build folder is not linked. (This one defines none of it.) + set(stray_dir "${WORK_DIR}/${mm}-stray") + file(REMOVE_RECURSE "${stray_dir}") + file(MAKE_DIRECTORY "${stray_dir}") + file(WRITE "${stray_dir}/stray.ts" "export function strayOnly(): number { return 1; }\n") + run("${stray_dir}" "-mm=${mm} stray object" out + "${TSLANG}" --emit=obj --no-default-lib -mm=${mm} stray.ts -o stray.obj) + if(WIN32) + run("${stray_dir}" "-mm=${mm} stray library" out "${AR}" /NOLOGO /OUT:${static_name} stray.obj) + else() + run("${stray_dir}" "-mm=${mm} stray library" out "${AR}" rcs ${static_name} stray.obj) + endif() + run("${stray_dir}" "-mm=${mm} --emit=exe beside a stray ${static_name}" out + "${TSLANG}" --emit=exe -mm=${mm} "--default-lib-path=${default_lib}" + "--gc-lib-path=${GC_LIB}" "--llvm-lib-path=${LLVM_LIB}" "--tslang-lib-path=${TSLANG_LIB}" + "${source}" -o use.exe) + + execute_process(COMMAND "${TSLANG}" --print-borrow-facts=${static_lib} OUTPUT_VARIABLE static_facts) + if(NOT static_facts MATCHES "\nMath[.]max 0\n") + message(STATUS "-mm=${mm}: the default library publishes no facts for Math.max (built before borrow facts)") + continue() + endif() + + set(dir "${WORK_DIR}/${mm}") + file(REMOVE_RECURSE "${dir}") + file(MAKE_DIRECTORY "${dir}") + set(common -mm=${mm} "--default-lib-path=${default_lib}" --stack-promotion-report) + + run("${dir}" "-mm=${mm} --emit=exe" decided + "${TSLANG}" --emit=exe ${common} "--gc-lib-path=${GC_LIB}" "--llvm-lib-path=${LLVM_LIB}" "--tslang-lib-path=${TSLANG_LIB}" + "${source}" -o use.exe) + if(NOT decided MATCHES "${promoted}") + message(FATAL_ERROR "-mm=${mm} --emit=exe: expected '${promoted}'\n${decided}") + endif() + run("${dir}" "-mm=${mm} run" out "${dir}/use.exe") + if(NOT out MATCHES "^6\n") + message(FATAL_ERROR "-mm=${mm} run: expected 6\n${out}") + endif() + + # the JIT reads the DLL it loads; its cache entry depends on it + run("${dir}" "-mm=${mm} --emit=jit" decided + "${TSLANG}" --emit=jit ${common} "--jit-cache-dir=${dir}/cache" "${source}") + if(NOT decided MATCHES "${promoted}" OR NOT decided MATCHES "(^|\n)6\n") + message(FATAL_ERROR "-mm=${mm} --emit=jit: expected '${promoted}' and 6\n${decided}") + endif() + file(GLOB manifests "${dir}/cache/*.deps") + if(manifests STREQUAL "") + message(FATAL_ERROR "-mm=${mm} --emit=jit: no cache manifest in ${dir}/cache") + endif() + file(READ "${manifests}" manifest) + file(TO_NATIVE_PATH "${shared_lib}" shared_lib_native) + string(FIND "${manifest}" "${shared_name}" at) + if(at EQUAL -1) + message(FATAL_ERROR "-mm=${mm} --emit=jit: the cache manifest does not name ${shared_lib}\n${manifest}") + endif() + + math(EXPR checked "${checked} + 1") +endforeach() + +if(checked EQUAL 0) + message("SKIPPED: no default library under ${default_lib} publishes borrow facts (rebuild it)") +endif() diff --git a/tslang/test/tester/borrow-facts.cmake b/tslang/test/tester/borrow-facts.cmake new file mode 100644 index 000000000..a0a3a5452 --- /dev/null +++ b/tslang/test/tester/borrow-facts.cmake @@ -0,0 +1,257 @@ +# Borrow facts across modules (docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md): +# a library's __tsborrow_ facts read back from its object file and from a static library holding it, +# and nothing from an import library; then a program linked against that static library. + +cmake_minimum_required(VERSION 3.17.3) + +foreach(var TSLANG SOURCE_DIR WORK_DIR AR MM GC_LIB LLVM_LIB TSLANG_LIB) + if(NOT DEFINED ${var}) + message(FATAL_ERROR "${var} is required") + endif() +endforeach() + +set(ENV{GC_LIB_PATH} "") +set(ENV{TSLANG_LIB_PATH} "") + +# run( ) - is "ok" or a regex the output must match +function(run dir what expect out_var) + execute_process(COMMAND ${ARGN} + WORKING_DIRECTORY "${dir}" + OUTPUT_VARIABLE out + ERROR_VARIABLE err + RESULT_VARIABLE status) + if("${out}${err}" MATCHES "Stack dump|Assertion failed") + message(FATAL_ERROR "${what}: crashed\n${out}\n${err}") + endif() + if(NOT status EQUAL 0) + message(FATAL_ERROR "${what}: exit ${status}\n${out}\n${err}") + endif() + if(NOT expect STREQUAL "ok" AND NOT "${out}${err}" MATCHES "${expect}") + message(FATAL_ERROR "${what}: expected '${expect}'\n${out}\n${err}") + endif() + set(${out_var} "${out}${err}" PARENT_SCOPE) +endfunction() + +if(MM STREQUAL "own") + set(lib_source facts_lib_own) + set(listed "Geo[.]len 0\n.*sumOf 0\n") +else() + set(lib_source facts_lib) + set(listed "Geo[.]len 0\n.*mixed 0\n.*sumOf 0\n") +endif() + +set(dir "${WORK_DIR}/${MM}") +file(REMOVE_RECURSE "${dir}") +file(MAKE_DIRECTORY "${dir}") + +if(WIN32) + set(obj facts_lib.obj) + set(archive facts_lib.lib) +else() + set(obj facts_lib.o) + set(archive libfacts_lib.a) +endif() + +run("${dir}" "--emit=obj ${lib_source}" ok out + "${TSLANG}" --emit=obj --no-default-lib -mm=${MM} "${SOURCE_DIR}/${lib_source}.ts" -o ${obj}) +if(WIN32) + run("${dir}" "archive" ok out "${AR}" /NOLOGO /OUT:${archive} ${obj}) +else() + run("${dir}" "archive" ok out "${AR}" rcs ${archive} ${obj}) +endif() + +run("${dir}" "facts of the object" "__tsborrow_${lib_source}_[0-9]+\n[^\n]*${listed}" out + "${TSLANG}" --print-borrow-facts=${obj}) +run("${dir}" "facts of the static library" "__tsborrow_${lib_source}_[0-9]+\n[^\n]*${listed}" out + "${TSLANG}" --print-borrow-facts=${archive}) + +# A DLL's import library: import members and descriptor objects, never facts. (A gc DLL needs the +# shared collector, which these tests are not given.) +if(WIN32 AND NOT MM STREQUAL "gc") + file(MAKE_DIRECTORY "${dir}/dll") + # self-contained: the test libraries call a function they only declare + file(WRITE "${dir}/dll/facts_dll.ts" "export function sumOf(a: number[]): number { return a.length; }\n") + run("${dir}/dll" "--emit=dll facts_dll" ok out + "${TSLANG}" --emit=dll --no-default-lib -mm=${MM} + "--llvm-lib-path=${LLVM_LIB}" "--tslang-lib-path=${TSLANG_LIB}" + facts_dll.ts -o facts_lib.dll) + run("${dir}/dll" "--emit=obj facts_dll" ok out + "${TSLANG}" --emit=obj --no-default-lib -mm=${MM} facts_dll.ts -o facts_dll.obj) + run("${dir}/dll" "facts of the DLL's object" "sumOf 0" out + "${TSLANG}" --print-borrow-facts=facts_dll.obj) + # what the JIT reads from the default library's DLL: its exports, from the file + run("${dir}/dll" "facts of the DLL" "__tsborrow_facts_dll_[0-9]+\nsumOf 0\n" out + "${TSLANG}" --print-borrow-facts=facts_lib.dll) + run("${dir}/dll" "facts of the import library" ok out + "${TSLANG}" --print-borrow-facts=facts_lib.lib) + if(NOT out STREQUAL "") + message(FATAL_ERROR "facts of the import library: expected none\n${out}") + endif() +endif() + +# no such file: no facts, no error +run("${dir}" "facts of a missing file" ok out "${TSLANG}" --print-borrow-facts=missing.lib) +if(NOT out STREQUAL "") + message(FATAL_ERROR "facts of a missing file: expected none\n${out}") +endif() + +# A program linked statically against the library reads its facts: an array passed where the library +# listed the parameter goes on the stack, every other stays on the heap. -mm=own refuses the keeping +# functions, so it has a program of its own. +if(MM STREQUAL "own") + set(use_source facts_use_own) + set(decisions + "facts_use_own[.]ts:13:15: remark: stack-promotion: stack" + "facts_use_own[.]ts:18:15: remark: stack-promotion: heap: unknown-callee" + "facts_use_own[.]ts:23:15: remark: stack-promotion: stack") +else() + set(use_source facts_use) + set(decisions + "facts_use[.]ts:20:15: remark: stack-promotion: stack" + "facts_use[.]ts:25:15: remark: stack-promotion: heap: unknown-callee" + "facts_use[.]ts:30:15: remark: stack-promotion: heap: unknown-callee" + "facts_use[.]ts:35:15: remark: stack-promotion: heap: unknown-callee" + "facts_use[.]ts:42:15: remark: stack-promotion: stack" + "facts_use[.]ts:43:15: remark: stack-promotion: heap: unknown-callee" + "facts_use[.]ts:48:15: remark: stack-promotion: heap: unknown-callee" + "facts_use[.]ts:53:15: remark: stack-promotion: stack") +endif() + +set(link_libs "--gc-lib-path=${GC_LIB}" "--llvm-lib-path=${LLVM_LIB}" "--tslang-lib-path=${TSLANG_LIB}") + +run("${dir}" "--emit=exe ${use_source} --lib=facts_lib" ok decided + "${TSLANG}" --emit=exe --no-default-lib -mm=${MM} --stack-promotion-report ${link_libs} --lib=facts_lib + "${SOURCE_DIR}/${use_source}.ts" -o use.exe) +foreach(decision ${decisions}) + if(NOT decided MATCHES "${decision}") + message(FATAL_ERROR "--lib=facts_lib: expected '${decision}'\n${decided}") + endif() +endforeach() + +# a declaration bound to another symbol (@linkname) takes no fact by its own name +if(NOT MM STREQUAL "own") + run("${dir}" "--emit=exe facts_linkname --lib=facts_lib" ok decided + "${TSLANG}" --emit=exe --no-default-lib -mm=${MM} --stack-promotion-report ${link_libs} --lib=facts_lib + "${SOURCE_DIR}/facts_linkname.ts" -o linkname.exe) + if(NOT decided MATCHES "facts_linkname[.]ts:12:15: remark: stack-promotion: heap: unknown-callee") + message(FATAL_ERROR "@linkname declaration: expected heap\n${decided}") + endif() +endif() + +# Another input defining a symbol a library published facts for: the linker may take that +# definition, so the fact decides nothing. Here facts_lib_other defines `sumOf` and keeps its argument. +if(NOT MM STREQUAL "own") + file(MAKE_DIRECTORY "${dir}/other") + if(WIN32) + set(other_obj facts_lib_other.obj) + set(other_archive facts_lib_other.lib) + else() + set(other_obj facts_lib_other.o) + set(other_archive libfacts_lib_other.a) + endif() + run("${dir}/other" "--emit=obj facts_lib_other" ok out + "${TSLANG}" --emit=obj --no-default-lib -mm=${MM} "${SOURCE_DIR}/facts_lib_other.ts" -o ${other_obj}) + if(WIN32) + run("${dir}/other" "archive" ok out "${AR}" /NOLOGO /OUT:${other_archive} ${other_obj}) + else() + run("${dir}/other" "archive" ok out "${AR}" rcs ${other_archive} ${other_obj}) + endif() + + # decided( ): the remarks, whether or not the link then succeeds + function(decided what out_var) + execute_process(COMMAND ${ARGN} WORKING_DIRECTORY "${dir}" OUTPUT_VARIABLE out ERROR_VARIABLE err) + if("${out}${err}" MATCHES "Stack dump|Assertion failed") + message(FATAL_ERROR "${what}: crashed +${out} +${err}") + endif() + set(${out_var} "${out}${err}" PARENT_SCOPE) + endfunction() + + set(sum_heap "facts_use[.]ts:20:15: remark: stack-promotion: heap") + decided("--lib=facts_lib --lib=other/facts_lib_other" out + "${TSLANG}" --emit=exe --no-default-lib -mm=${MM} --stack-promotion-report ${link_libs} --lib=facts_lib + "--lib=${dir}/other/${other_archive}" "${SOURCE_DIR}/facts_use.ts" -o collide1.exe) + if(NOT out MATCHES "${sum_heap}" OR NOT out MATCHES "facts_use[.]ts:53:15: remark: stack-promotion: stack") + message(FATAL_ERROR "two libraries defining sumOf: expected sumOf heap, Geo.len stack +${out}") + endif() + decided("--lib=facts_lib --obj=other/facts_lib_other" out + "${TSLANG}" --emit=exe --no-default-lib -mm=${MM} --stack-promotion-report ${link_libs} --lib=facts_lib + "--obj=${dir}/other/${other_obj}" "${SOURCE_DIR}/facts_use.ts" -o collide2.exe) + if(NOT out MATCHES "${sum_heap}") + message(FATAL_ERROR "an object defining sumOf: expected sumOf heap +${out}") + endif() + + # a library the linker searches for itself: what it defines is unknown, so no fact is trusted + if(WIN32) + set(searched kernel32) + else() + set(searched m) + endif() + decided("--lib=facts_lib --lib=${searched}" out + "${TSLANG}" --emit=exe --no-default-lib -mm=${MM} --stack-promotion-report ${link_libs} --lib=facts_lib + --lib=${searched} "${SOURCE_DIR}/facts_use.ts" -o searched.exe) + if(out MATCHES "remark: stack-promotion: stack") + message(FATAL_ERROR "a searched library: expected no fact trusted +${out}") + endif() +endif() + +# A -shared import defines what it exports: here the DLL facts_lib (from facts_dll.ts) defines +# `sumOf`, and the static library's `sumOf 0` must not apply to it. +if(WIN32 AND NOT MM STREQUAL "gc" AND NOT MM STREQUAL "own") + execute_process(COMMAND "${TSLANG}" --emit=exe --no-default-lib -mm=${MM} --stack-promotion-report ${link_libs} + "--lib=${dir}/${archive}" "${SOURCE_DIR}/facts_shared_use.ts" -o shared_use.exe + WORKING_DIRECTORY "${dir}/dll" OUTPUT_VARIABLE out ERROR_VARIABLE err) + if(NOT "${out}${err}" MATCHES "facts_shared_use[.]ts:8:15: remark: stack-promotion: heap") + message(FATAL_ERROR "a -shared import's sumOf: expected heap +${out}${err}") + endif() +endif() + +# --lib naming a DLL: linked dynamically, so whatever copy runs is not the file read - no facts +if(WIN32 AND NOT MM STREQUAL "gc" AND NOT MM STREQUAL "own") + execute_process(COMMAND "${TSLANG}" --emit=exe --no-default-lib -mm=${MM} --stack-promotion-report ${link_libs} + "--lib=${dir}/dll/facts_lib.dll" "${SOURCE_DIR}/facts_use.ts" -o dll_lib.exe + WORKING_DIRECTORY "${dir}" OUTPUT_VARIABLE out ERROR_VARIABLE err) + if("${out}${err}" MATCHES "remark: stack-promotion: stack") + message(FATAL_ERROR "--lib naming a DLL: expected no facts read +${out}${err}") + endif() +endif() + +# the same program, nothing promoted: the same output +run("${dir}" "--emit=exe ${use_source} --no-stack-promotion" ok out + "${TSLANG}" --emit=exe --no-default-lib -mm=${MM} --no-stack-promotion ${link_libs} --lib=facts_lib + "${SOURCE_DIR}/${use_source}.ts" -o use_heap.exe) +run("${dir}" "run use" ok promoted_output "${dir}/use.exe") +run("${dir}" "run use_heap" ok heap_output "${dir}/use_heap.exe") +if(NOT promoted_output STREQUAL heap_output) + message(FATAL_ERROR "promoted program printed\n${promoted_output}\nunpromoted\n${heap_output}") +endif() + +# --emit=obj does not link, so it reads nothing +run("${dir}" "--emit=obj ${use_source}" ok decided + "${TSLANG}" --emit=obj --no-default-lib -mm=${MM} --stack-promotion-report --lib=facts_lib + "${SOURCE_DIR}/${use_source}.ts" -o use_obj.obj) +if(decided MATCHES "remark: stack-promotion: stack") + message(FATAL_ERROR "--emit=obj read facts\n${decided}") +endif() + +# a library built before facts existed: links, no warning, nothing promoted through it +file(MAKE_DIRECTORY "${dir}/old") +run("${dir}/old" "--emit=obj ${lib_source} --no-borrow-facts-export" ok out + "${TSLANG}" --emit=obj --no-default-lib -mm=${MM} --no-borrow-facts-export "${SOURCE_DIR}/${lib_source}.ts" -o ${obj}) +if(WIN32) + run("${dir}/old" "archive" ok out "${AR}" /NOLOGO /OUT:${archive} ${obj}) +else() + run("${dir}/old" "archive" ok out "${AR}" rcs ${archive} ${obj}) +endif() +run("${dir}/old" "--emit=exe ${use_source} against the old library" ok decided + "${TSLANG}" --emit=exe --no-default-lib -mm=${MM} --stack-promotion-report ${link_libs} --lib=facts_lib + "${SOURCE_DIR}/${use_source}.ts" -o use.exe) +if(decided MATCHES "remark: stack-promotion: stack|warning") + message(FATAL_ERROR "against the old library\n${decided}") +endif() diff --git a/tslang/test/tester/stack-promotion/facts_defaultlib.ts b/tslang/test/tester/stack-promotion/facts_defaultlib.ts new file mode 100644 index 000000000..b0fbaa6ea --- /dev/null +++ b/tslang/test/tester/stack-promotion/facts_defaultlib.ts @@ -0,0 +1,10 @@ +// Borrow facts from the default library: Math.max's rest array is built here and only read there. +// See borrow-facts-defaultlib.cmake. + +function biggest(n: number) { + return Math.max(n, n * 2, n - 1); +} + +function main() { + print(biggest(3)); +} diff --git a/tslang/test/tester/stack-promotion/facts_lib.ts b/tslang/test/tester/stack-promotion/facts_lib.ts new file mode 100644 index 000000000..8ad5d92bf --- /dev/null +++ b/tslang/test/tester/stack-promotion/facts_lib.ts @@ -0,0 +1,71 @@ +// Borrow facts (docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md): what +// this library's __tsborrow_ symbol lists - the parameters its own analysis shows are only read. + +declare function otherKeep(a: number[]): void; + +let kept: number[] = []; + +export function sumOf(a: number[]): number { + let s = 0; + for (const v of a) s += v; + return s; +} + +export function keep(a: number[]) { + kept = a; +} + +export function same(a: number[]): number[] { + return a; +} + +export function viaCallback(a: number[], f: (x: number[]) => void) { + f(a); +} + +export function mixed(a: number[], b: number[]): number { + kept = b; + return a[0]; +} + +export function useOther(a: number[]) { + otherKeep(a); +} + +function helper(a: number[]): number { + return a.length; +} + +export function callsHelper(): number { + return helper([1, 2]); +} + +export namespace Geo { + export function len(a: number[]): number { + return a.length; + } +} + +export class Holder { + peek(a: number[]): number { + return a[0] + this.twice(a); + } + + twice(a: number[]): number { + return a[0] * 2; + } +} + +export function makeHolder(): Holder { + return new Holder(); +} + +export function keptLength(): number { + return kept.length; +} + +// defines `renamedLen`, not `lenOf`: no fact may name either +@linkname("renamedLen") +export function lenOf(a: number[]): number { + return a.length; +} diff --git a/tslang/test/tester/stack-promotion/facts_lib_other.ts b/tslang/test/tester/stack-promotion/facts_lib_other.ts new file mode 100644 index 000000000..a5867c703 --- /dev/null +++ b/tslang/test/tester/stack-promotion/facts_lib_other.ts @@ -0,0 +1,10 @@ +// Another library defining `sumOf`, which keeps its argument and publishes no fact for it. Linked +// beside facts_lib, or given as an object, facts_lib's `sumOf 0` must not decide anything: the +// linker may take this definition. + +let held: number[] = []; + +export function sumOf(a: number[]): number { + held = a; + return 0; +} diff --git a/tslang/test/tester/stack-promotion/facts_lib_own.ts b/tslang/test/tester/stack-promotion/facts_lib_own.ts new file mode 100644 index 000000000..5d8405460 --- /dev/null +++ b/tslang/test/tester/stack-promotion/facts_lib_own.ts @@ -0,0 +1,30 @@ +// Borrow facts under -mm=own, which refuses a function that keeps or returns its parameter: what +// is left is a parameter only read, one handed to a function with no body, and a virtual call. + +declare function otherKeep(a: number[]): void; + +export function sumOf(a: number[]): number { + let s = 0; + for (const v of a) s += v; + return s; +} + +export function useOther(a: number[]) { + otherKeep(a); +} + +export namespace Geo { + export function len(a: number[]): number { + return a.length; + } +} + +export class Holder { + peek(a: number[]): number { + return a[0] + this.twice(a); + } + + twice(a: number[]): number { + return a[0] * 2; + } +} diff --git a/tslang/test/tester/stack-promotion/facts_linkname.ts b/tslang/test/tester/stack-promotion/facts_linkname.ts new file mode 100644 index 000000000..ac4e7c9ba --- /dev/null +++ b/tslang/test/tester/stack-promotion/facts_linkname.ts @@ -0,0 +1,19 @@ +// Borrow facts are keyed by the symbol a declaration binds: this `sumOf` binds facts_lib's `keep`, +// so facts_lib's `sumOf 0` must not apply to it. + +@linkname("keep") +declare function sumOf(a: number[]): void; +declare function keptLength(): number; + +function otherKeep(a: number[]) { +} + +function call(n: number) { + const a = [n, n, n]; + sumOf(a); +} + +function main() { + call(1); + print(keptLength()); +} diff --git a/tslang/test/tester/stack-promotion/facts_shared_use.ts b/tslang/test/tester/stack-promotion/facts_shared_use.ts new file mode 100644 index 000000000..19d2d155d --- /dev/null +++ b/tslang/test/tester/stack-promotion/facts_shared_use.ts @@ -0,0 +1,14 @@ +// Borrow facts and a -shared import: this `sumOf` comes from the DLL facts_lib in the working +// directory, so the static library's fact for a function of that name must not apply. + +@static +import { sumOf } from "./facts_lib"; + +function call(n: number) { + const a = [n, n]; + return sumOf(a); +} + +function main() { + print(call(1)); +} diff --git a/tslang/test/tester/stack-promotion/facts_use.ts b/tslang/test/tester/stack-promotion/facts_use.ts new file mode 100644 index 000000000..66db755ef --- /dev/null +++ b/tslang/test/tester/stack-promotion/facts_use.ts @@ -0,0 +1,67 @@ +// Borrow facts: a program linked with --lib against facts_lib. A fresh array goes on the stack +// when the library listed the parameter it is passed to, and stays on the heap otherwise. Arrays +// are built from a parameter: a constant literal is a static global, never allocated. + +declare function sumOf(a: number[]): number; +declare function keep(a: number[]): void; +declare function same(a: number[]): number[]; +declare function viaCallback(a: number[], f: (x: number[]) => void): void; +declare function mixed(a: number[], b: number[]): number; +declare function useOther(a: number[]): void; +declare function keptLength(): number; +declare namespace Geo { + function len(a: number[]): number; +} + +function otherKeep(a: number[]) { +} + +function callSum(n: number) { + const a = [n, n + 1, n + 2]; + return sumOf(a); +} + +function callKeep(n: number) { + const a = [n, n]; + keep(a); +} + +function callSame(n: number) { + const a = [n]; + return same(a).length; +} + +function callViaCallback(n: number) { + const a = [n, n]; + let seen = 0; + viaCallback(a, (x: number[]) => { seen = x.length; }); + return seen; +} + +function callMixed(n: number) { + const a = [n]; + const b = [n, n]; + return mixed(a, b); +} + +function callUseOther(n: number) { + const a = [n]; + useOther(a); +} + +function callLen(n: number) { + const a = [n, n, n, n]; + return Geo.len(a); +} + +function main() { + print(callSum(1)); + callKeep(1); + print(keptLength()); + print(callSame(1)); + print(callViaCallback(1)); + print(callMixed(1)); + print(keptLength()); + callUseOther(1); + print(callLen(1)); +} diff --git a/tslang/test/tester/stack-promotion/facts_use_own.ts b/tslang/test/tester/stack-promotion/facts_use_own.ts new file mode 100644 index 000000000..b7ad7e194 --- /dev/null +++ b/tslang/test/tester/stack-promotion/facts_use_own.ts @@ -0,0 +1,31 @@ +// Borrow facts under -mm=own: a program linked with --lib against facts_lib_own. + +declare function sumOf(a: number[]): number; +declare function useOther(a: number[]): void; +declare namespace Geo { + function len(a: number[]): number; +} + +function otherKeep(a: number[]) { +} + +function callSum(n: number) { + const a = [n, n + 1, n + 2]; + return sumOf(a); +} + +function callUseOther(n: number) { + const a = [n]; + useOther(a); +} + +function callLen(n: number) { + const a = [n, n, n, n]; + return Geo.len(a); +} + +function main() { + print(callSum(1)); + callUseOther(1); + print(callLen(1)); +} diff --git a/tslang/tslang/CMakeLists.txt b/tslang/tslang/CMakeLists.txt index 4cb4420cf..ca7542df1 100644 --- a/tslang/tslang/CMakeLists.txt +++ b/tslang/tslang/CMakeLists.txt @@ -68,7 +68,8 @@ add_llvm_executable(tslang declarationInline.cpp vscode.cpp cmake.cpp - defaultlib.cpp) + defaultlib.cpp + borrowfacts.cpp) llvm_update_compile_flags(tslang) target_link_libraries(tslang PRIVATE ${LIBS}) diff --git a/tslang/tslang/borrowfacts.cpp b/tslang/tslang/borrowfacts.cpp new file mode 100644 index 000000000..592b40f9f --- /dev/null +++ b/tslang/tslang/borrowfacts.cpp @@ -0,0 +1,171 @@ +#include "TypeScript/DataStructs.h" +#include "TypeScript/Defines.h" +#include "TypeScript/TypeScriptCompiler/Defines.h" + +#include "llvm/ADT/STLExtras.h" +#include "llvm/ADT/Sequence.h" +#include "llvm/ADT/StringSet.h" +#include "llvm/BinaryFormat/Magic.h" +#include "llvm/Support/CommandLine.h" +#include "llvm/Support/FileSystem.h" +#include "llvm/Support/Path.h" +#include "llvm/TargetParser/Host.h" +#include "llvm/TargetParser/Triple.h" + +#define DEBUG_TYPE "tslang" + +namespace cl = llvm::cl; + +// Borrow facts across modules (docs/superpowers/specs/2026-10-18-borrow-facts-across-modules-design.md): +// which parameters of a library's functions their bodies only borrow, read from the library itself so +// stack promotion may put an object passed to one on the stack. Read only from a file this compile +// links or loads, because the facts must describe the code that runs: the static libraries of an +// --emit=exe link, and the default library's DLL the JIT loads. Never from a DLL a program links, +// nor a -shared import - those load whatever copy sits beside the program at run time. Missing or +// unreadable facts are not an error: no facts, every call an escape. + +extern cl::opt emitAction; +extern cl::list libs; +extern cl::list objs; + +llvm::Triple getLinkTargetTriple(); +std::string getTargetStaticLibName(llvm::StringRef); +std::string getDefaultLibBinary(bool, const CompileOptions &, const llvm::Triple &); + +// From TypeScript/ObjDumper.h, whose llvm/BinaryFormat/COFF.h collides with macros +namespace Dump +{ + std::vector> readCStringSymbols(llvm::StringRef, llvm::StringRef); + std::vector definedSymbols(llvm::StringRef); +} + +// A static library or an object file: linked into the program, so what it publishes describes the +// code that runs. Not a DLL or shared object, whose copy at run time may be another. +static bool isStaticLinkInput(llvm::StringRef file) +{ + llvm::file_magic magic; + if (llvm::identify_magic(file, magic)) + { + return false; + } + + return magic == llvm::file_magic::archive || magic == llvm::file_magic::coff_object || + magic == llvm::file_magic::elf_relocatable; +} + +// The static library a `--lib` value names, when this compile can tell: the value itself if it is +// one, else the target's static library of that name (`.lib`, `lib.a`) in the current +// directory. Empty otherwise - the linker searches for it, or it is linked dynamically, and then no +// borrow facts are trusted at all (readBorrowFacts). exe.cpp links a resolved library by this path, +// so the file read is the file linked. +std::string resolveLinkLibrary(llvm::StringRef value) +{ + for (auto candidate : {value.str(), getTargetStaticLibName(value)}) + { + if (llvm::sys::fs::is_regular_file(candidate) && isStaticLinkInput(candidate)) + { + llvm::SmallString<256> absolute(candidate); + llvm::sys::fs::make_absolute(absolute); + return absolute.str().str(); + } + } + + return ""; +} + +// The facts of `files`, each line kept only when no other of `files` or `others` defines its symbol: +// the linker may take that other definition, which the fact does not describe. +static void appendBorrowFacts(CompileOptions &compileOptions, llvm::ArrayRef files, + llvm::ArrayRef others = {}) +{ + llvm::SmallVector> defined; + for (auto &file : files) + { + defined.emplace_back(); + for (auto &symbol : Dump::definedSymbols(file)) + { + defined.back().insert(symbol); + } + } + + llvm::StringSet<> definedByOthers; + for (auto &file : others) + { + for (auto &symbol : Dump::definedSymbols(file)) + { + definedByOthers.insert(symbol); + } + } + + for (auto [index, file] : llvm::enumerate(files)) + { + if (llvm::sys::fs::exists(file)) + { + compileOptions.importedBorrowFactsFiles.push_back(file); + } + + for (auto &[symbol, text] : Dump::readCStringSymbols(file, SHARED_LIB_BORROW_FACTS)) + { + llvm::SmallVector lines; + llvm::StringRef(text).split(lines, '\n', -1, /*KeepEmpty=*/false); + for (auto line : lines) + { + auto name = line.split(' ').first; + auto elsewhere = definedByOthers.contains(name); + for (auto other : llvm::seq(0, files.size())) + { + elsewhere |= other != index && defined[other].contains(name); + } + + if (!elsewhere) + { + compileOptions.importedBorrowFacts += line.str() + "\n"; + } + } + } + } +} + +// Called before the module is compiled: stack promotion reads them. +// --emit=exe: the facts of the default library's static library and of every --lib this compile can +// find. --emit=jit: those of the default library's DLL the JIT loads, read from its file. +void readBorrowFacts(CompileOptions &compileOptions) +{ + if (emitAction == Action::RunJIT) + { + if (!compileOptions.noDefaultLib) + { + // the JIT is host-only, and loads this file (jit.cpp, prepareJitProcess) + appendBorrowFacts(compileOptions, + {getDefaultLibBinary(/*shared=*/true, compileOptions, + llvm::Triple(llvm::Triple::normalize(llvm::sys::getDefaultTargetTriple())))}); + } + + return; + } + + if (emitAction != Action::BuildExe) + { + return; + } + + std::vector files; + if (!compileOptions.noDefaultLib) + { + files.push_back(getDefaultLibBinary(/*shared=*/false, compileOptions, getLinkTargetTriple())); + } + + for (auto &lib : libs) + { + auto file = resolveLinkLibrary(lib); + if (file.empty()) + { + // one the linker searches for itself may define anything, under any name + return; + } + + files.push_back(file); + } + + appendBorrowFacts(compileOptions, files, std::vector(objs.begin(), objs.end())); +} diff --git a/tslang/tslang/defaultlib.cpp b/tslang/tslang/defaultlib.cpp index dfdc8bb7e..54ec100ea 100644 --- a/tslang/tslang/defaultlib.cpp +++ b/tslang/tslang/defaultlib.cpp @@ -53,6 +53,8 @@ std::string getLLVMLibPath(); std::string getTslangLibPath(); std::string getDefaultLibPath(); std::string getpath(std::string, const SmallVectorImpl&); +std::string mergeWithDefaultLibPath(std::string, std::string); +std::string getDefaultLibSubDir(bool, bool, const char *, const llvm::Triple &); std::error_code copy_from_to(const SmallVectorImpl&, const SmallVectorImpl&); // The subfolder of --default-lib-path holding the default library's binaries for a target, a build @@ -73,6 +75,19 @@ std::string getDefaultLibSubDir(bool shared, bool debugBuild, const char *memory return subDir + "/" + (debugBuild ? DEFAULT_LIB_BUILD_DIR_DEBUG : DEFAULT_LIB_BUILD_DIR_RELEASE) + "/" + memoryModel; } +// The default library file a link or the JIT uses: the static library (`TypeScriptDefaultLib.lib`, +// `libTypeScriptDefaultLib.a`) or the shared one (`.dll`, `.so`) for the target, build mode and model. +std::string getDefaultLibBinary(bool shared, const CompileOptions &compileOptions, const llvm::Triple &triple) +{ + auto win = triple.getOS() == llvm::Triple::Win32; + std::string fileName = shared ? (win ? DEFAULT_LIB_NAME ".dll" : "lib" DEFAULT_LIB_NAME ".so") + : (win ? DEFAULT_LIB_NAME ".lib" : "lib" DEFAULT_LIB_NAME ".a"); + return mergeWithDefaultLibPath(getDefaultLibPath(), + getDefaultLibSubDir(shared, compileOptions.generateDebugInfo, + memoryModelName(compileOptions.memoryModel), triple) + + "/" + fileName); +} + bool checkFileExistsAtPath(const SmallVectorImpl& path, std::string subPath, std::string fileName) { llvm::SmallVector destPath(0); diff --git a/tslang/tslang/exe.cpp b/tslang/tslang/exe.cpp index 58a37a9ae..762cd094b 100644 --- a/tslang/tslang/exe.cpp +++ b/tslang/tslang/exe.cpp @@ -57,6 +57,8 @@ extern cl::opt verbose; std::string getDefaultOutputFileName(enum Action); std::string mergeWithDefaultLibPath(std::string, std::string); std::string getDefaultLibSubDir(bool, bool, const char *, const llvm::Triple &); +std::string getDefaultLibBinary(bool, const CompileOptions &, const llvm::Triple &); +std::string resolveLinkLibrary(llvm::StringRef); using llvm::StringRef; @@ -134,7 +136,7 @@ bool checkFileExistsAtPath(std::string path, std::string fileName) // Windows x86 takes its libraries from an `x86` subdirectory of each configured lib path, so the // flat path has none of them by design; buildExe checks the subdirectory instead (see // resolveWindowsLibPath). Every other target keeps the flat layout. -static llvm::Triple getLinkTargetTriple() +llvm::Triple getLinkTargetTriple() { return llvm::Triple(TargetTriple.empty() ? llvm::sys::getDefaultTargetTriple() : llvm::Triple::normalize(TargetTriple)); } @@ -146,7 +148,7 @@ static bool isWindowsX86Target() } // A library's file name as the target names it, not the host: gc.lib on Windows, libgc.a elsewhere. -static std::string getTargetStaticLibName(llvm::StringRef name) +std::string getTargetStaticLibName(llvm::StringRef name) { return getLinkTargetTriple().getOS() == llvm::Triple::Win32 ? (name + ".lib").str() : ("lib" + name + ".a").str(); } @@ -675,6 +677,7 @@ int buildExe(int argc, char **argv, std::string objFileName, std::string additio std::string tslangLibPathOpt; std::string emsdkSysRootPathOpt; std::string defaultLibPathOpt; + std::string defaultLibFile; std::string defaultLibFileOpt; // Windows x86 programs: the asInvoker manifest's resource object, removed after the link llvm::SmallString<128> manifestObjFileName; @@ -779,7 +782,10 @@ int buildExe(int argc, char **argv, std::string objFileName, std::string additio llvm::SmallVector libOpts; for (auto lib : libs) { - auto libPathOpt = getLibOpt(lib); + // a library whose borrow facts were read is linked by that file's path, so the file read + // is the file linked (see borrowfacts.cpp) + auto resolved = resolveLinkLibrary(lib); + auto libPathOpt = resolved.empty() ? getLibOpt(lib) : resolved; if (!libPathOpt.empty()) { libOpts.push_back(libPathOpt); @@ -793,8 +799,6 @@ int buildExe(int argc, char **argv, std::string objFileName, std::string additio if (!compileOptions.noDefaultLib) { - // default lib file - args.push_back("-l" DEFAULT_LIB_NAME); // default lib path (per-build subfolder: debug/release must match how // this program is being compiled so the CRT and default-lib binaries agree). @@ -808,9 +812,8 @@ int buildExe(int argc, char **argv, std::string objFileName, std::string additio // self-contained .so. Its shared default library leaves GC_* to the process, and an app's // process (the Java VM) has no collector to give it. auto sharedDefaultLib = shared && !android; - auto defaultLibSubDir = getDefaultLibSubDir(sharedDefaultLib, compileOptions.generateDebugInfo, - memoryModelName(compileOptions.memoryModel), TheTriple); - auto defaultLibDir = mergeWithDefaultLibPath(getDefaultLibPath(), defaultLibSubDir); + auto defaultLibDir = + llvm::sys::path::parent_path(getDefaultLibBinary(sharedDefaultLib, compileOptions, TheTriple)).str(); // Checked here rather than left to the linker. mergeWithDefaultLibPath only joins the // path, so a model that has not been built reaches lld as a `-L` to nowhere and comes @@ -858,12 +861,24 @@ int buildExe(int argc, char **argv, std::string objFileName, std::string additio } } - defaultLibPathOpt = getLibsPathOpt(defaultLibDir); - if (!defaultLibPathOpt.empty()) + // The static library by its path: the file the borrow facts were read from + // (borrowfacts.cpp) is the file linked, never one of that name the linker finds first, + // such as a stray copy in the current folder. The DLL's import library is searched for. + if (!sharedDefaultLib && !defaultLibDir.empty()) { - args.push_back(defaultLibPathOpt.c_str()); + defaultLibFile = getDefaultLibBinary(/*shared=*/false, compileOptions, TheTriple); + args.push_back(defaultLibFile.c_str()); } - } + else + { + args.push_back("-l" DEFAULT_LIB_NAME); + defaultLibPathOpt = getLibsPathOpt(defaultLibDir); + if (!defaultLibPathOpt.empty()) + { + args.push_back(defaultLibPathOpt.c_str()); + } + } + } // Which Boehm. A shared library, and a program that loads one, share a process with other gc // code, so they take the collector from gc.dll: linked statically, each binary brings a diff --git a/tslang/tslang/jit.cpp b/tslang/tslang/jit.cpp index a69df5620..3ca14a093 100644 --- a/tslang/tslang/jit.cpp +++ b/tslang/tslang/jit.cpp @@ -70,6 +70,7 @@ std::string getDefaultLibPath(); std::string getTslangLibPath(); std::string mergeWithDefaultLibPath(std::string, std::string); std::string getDefaultLibSubDir(bool, bool, const char *, const llvm::Triple &); +std::string getDefaultLibBinary(bool, const CompileOptions &, const llvm::Triple &); std::string makeAbsolutePath(std::string); int registerMLIRDialects(mlir::ModuleOp); @@ -595,17 +596,8 @@ static int prepareJitProcess(CompileOptions &compileOptions) // Keyed on --di (generate debug info): with debug info use the debug lib. // ...and per memory model, for the same reason the linker path is: the default lib // allocates the way the model it was built for allocates. See getDefaultLibSubDir. - auto defaultLibSubDir = - getDefaultLibSubDir(/*shared=*/true, compileOptions.generateDebugInfo, - memoryModelName(compileOptions.memoryModel), - llvm::Triple(llvm::Triple::normalize(llvm::sys::getDefaultTargetTriple()))); // JIT is host-only - auto defaultLibFile = mergeWithDefaultLibPath(getDefaultLibPath(), -#ifdef WIN32 - defaultLibSubDir + "/" DEFAULT_LIB_NAME ".dll" -#else - defaultLibSubDir + "/lib" DEFAULT_LIB_NAME ".so" -#endif - ); + auto defaultLibFile = getDefaultLibBinary(/*shared=*/true, compileOptions, + llvm::Triple(llvm::Triple::normalize(llvm::sys::getDefaultTargetTriple()))); // JIT is host-only // Named here rather than left to the loader, for the reason exe.cpp checks the link // directory: a model that has not been built otherwise surfaces as the platform's diff --git a/tslang/tslang/jitcache.cpp b/tslang/tslang/jitcache.cpp index 5ecc51644..849f096d6 100644 --- a/tslang/tslang/jitcache.cpp +++ b/tslang/tslang/jitcache.cpp @@ -574,6 +574,12 @@ class JitUnitBuilder addDep(library, fileHash(library)); } + // the libraries whose borrow facts decided what went on the stack (borrowfacts.cpp) + for (auto &library : options.importedBorrowFactsFiles) + { + addDep(library, fileHash(library)); + } + registerMLIRDialects(*module); llvm::LLVMContext llvmContext; diff --git a/tslang/tslang/transform.cpp b/tslang/tslang/transform.cpp index fbf66ff65..407cde8c1 100644 --- a/tslang/tslang/transform.cpp +++ b/tslang/tslang/transform.cpp @@ -80,6 +80,7 @@ extern cl::opt disableWarnings; extern cl::opt verifyOwnership; extern cl::opt ownSkipInference; extern cl::opt noStackPromotion; +extern cl::opt noBorrowFactsExport; extern cl::opt stackPromotionMaxObject; extern cl::opt stackPromotionMaxFrame; extern cl::opt stackPromotionReport; @@ -154,6 +155,14 @@ int runMLIRPasses(mlir::MLIRContext &context, llvm::SourceMgr &sourceMgr, mlir:: pm.nest().addPass(mlir::typescript::createOwnershipInferencePass()); } + // What this module promises importers about its parameters, whether or not it promotes + // anything itself. Nothing imports a JIT-run program. + if (!compileOptions.isJit && !noBorrowFactsExport) + { + pm.addPass(mlir::typescript::createBorrowFactsExportPass( + SHARED_LIB_BORROW_FACTS + typescript::moduleSymbolSuffix(inputFilename))); + } + // Under every model, after own's inference: what it decides about a stack object's retains // is already settled, and the optimiser has not yet inlined anything if (!noStackPromotion) diff --git a/tslang/tslang/tslang.cpp b/tslang/tslang/tslang.cpp index 257d9b49e..d4e031da8 100644 --- a/tslang/tslang/tslang.cpp +++ b/tslang/tslang/tslang.cpp @@ -51,7 +51,14 @@ namespace cl = llvm::cl; +// From TypeScript/ObjDumper.h, whose llvm/BinaryFormat/COFF.h collides with macros +namespace Dump +{ + std::vector> readCStringSymbols(llvm::StringRef, llvm::StringRef); +} + CompileOptions prepareOptions(); +void readBorrowFacts(CompileOptions &); std::string getDefaultOutputFileName(enum Action); std::string getDefaultExt(enum Action); std::string getDefaultLibPath(); @@ -141,6 +148,10 @@ cl::opt noStackPromotion("no-stack-promotion", cl::desc("Keep every alloca cl::opt stackPromotionMaxObject("stack-promotion-max-object", cl::desc("The most bytes one object may take on the stack"), cl::init(256), cl::cat(TypeScriptCompilerCategory)); cl::opt stackPromotionMaxFrame("stack-promotion-max-frame", cl::desc("The most bytes one function may put on the stack through promotion"), cl::init(4096), cl::cat(TypeScriptCompilerCategory)); cl::opt stackPromotionReport("stack-promotion-report", cl::desc("Report, for every allocation, whether it went on the stack and if not why"), cl::cat(TypeScriptCompilerCategory)); +// Test-only: build a module as one built before borrow facts existed, which publishes none. +cl::opt noBorrowFactsExport("no-borrow-facts-export", cl::desc("Do not write the module's borrow facts (__tsborrow_) for importers"), cl::Hidden, cl::cat(TypeScriptCompilerCategory)); +// Debugging and tests: what a program linked against the file would read. +cl::opt printBorrowFacts("print-borrow-facts", cl::desc("Print the borrow facts (__tsborrow_ symbols) of an object file or static library, then exit"), cl::value_desc("file"), cl::Hidden, cl::cat(TypeScriptCompilerCategory)); cl::opt ownSkipInference("own-skip-inference", cl::Hidden, cl::desc("Under -mm=own, skip ownership inference so the lowering backstop can be tested"), cl::cat(TypeScriptCompilerDebugCategory)); cl::opt generateDebugInfo("di", cl::desc("Generate Debug Infomation"), cl::cat(TypeScriptCompilerCategory)); cl::opt lldbDebugInfo("lldb", cl::desc("Debug Infomation for LLDB"), cl::cat(TypeScriptCompilerCategory)); @@ -441,6 +452,16 @@ int main(int argc, char **argv) enableOpt.setValue(true); } + if (!printBorrowFacts.empty()) + { + for (auto &[symbol, text] : Dump::readCStringSymbols(printBorrowFacts, SHARED_LIB_BORROW_FACTS)) + { + llvm::outs() << symbol << '\n' << text; + } + + return 0; + } + if (printDefaultLibDir != DefaultLibKindNotAsked) { auto triple = llvm::Triple(llvm::Triple::normalize(TargetTriple.empty() ? llvm::sys::getDefaultTargetTriple() : TargetTriple.getValue())); @@ -476,6 +497,8 @@ int main(int argc, char **argv) return 0; } + readBorrowFacts(compileOptions); + // The JIT with its cache compiles each file it needs by itself, and only if it has to. if (emitAction == Action::RunJIT && compileOptions.jitCache) {