-mm=own phase 5a: a closure that does not escape borrows what it captures

tslang/docs/superpowers/plans/2026-10-02-own-phase-5.md

@@ -0,0 +1,184 @@
+# `-mm=own` Phase 5 Implementation Plan: closures
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development
+> (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use
+> checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Under `-mm=own`, a closure that captures a variable compiles when its ownership can be
+proven (spec §2.5, §4.2 step 6). Today every `ts.RetainCell` is the error "closures that capture a
+variable are not supported by -mm=own yet" (25 corpus files stop there first), and a captured
+parameter with a heap value is a lowering error.
+
+After this phase:
+
+- A closure that does not escape **borrows** what it captures. The frame keeps its cells, the box
+  owns nothing but itself, and the closure may not run after anything it borrows is released.
+- A closure that escapes (returned, stored, pushed, kept) **owns** what it captures. Its cells
+  and by-value captures move into the box, and the frame may not use them after that.
+- Everything else is an error, with the rule in the message.
+
+**Architecture:** inference-side only, plus two marks the lowering reads. MLIRGen does not change
+(`test-own-mlirgen-matches-rc` keeps passing).
+
+1. **`OwnershipInferencePass`** classifies each `ts.CreateBoundFunction {__owns_capture}`: does its
+   value escape? Then it decides each capture: a cell (a captured `ts.Variable`, or a cell an
+   enclosing closure captured, read out of its box) or a by-value copy (`ts.Store` of a value into
+   a box field, after rc's `ts.Retain`).
+2. **Two marks for the lowering**, own only:
+   - `__own_borrows_captures` on `ts.CreateBoundFunction`: the box's tag names a routine that
+     frees the box and nothing it holds.
+   - `__own_cell_borrows` on `ts.ReleaseCell`: the cell does not own its value (a captured
+     parameter, `this`), so its release frees the cell only.
+3. **Drops** learn cells: an assignment to a captured variable, in the frame or in a closure body,
+   may destroy what a borrow read out of it. A call given a closure may run it.
+
+**Tech Stack:** C++17, MLIR, CMake/ctest, `test/tester/test-runner.cpp`,
+`test/tester/tools/measure.ps1`.
+
+**Spec:** `docs/superpowers/specs/2026-09-24-own-memory-model-design.md`: §2.5 (closures), §4.2
+step 6, §15.7 (this phase's input). Results go into §16.
+
+## Global Constraints
+
+- Branch `own-phase-5` from `origin/main` (7c543604, with #434 merged).
+- `gc`, `rc` and `none` output does not change: the new marks are set by an own-only pass and read
+  by the own branch of the lowering.
+- Never branch on the memory model in MLIRGen (§4.3).
+- `__tslang_inc_ref` and `__tslang_dec_ref` never appear in own output.
+- Own tests run with `--no-default-lib`, under JIT and AOT, and read through a `churn()` after the
+  point where a wrong release would free. Expected values are checked under `gc` first.
+- Build: `cmake --build __build/tslang/windows-msbuild-2026-release --config Release --target tslang test-runner`
+  from `I:/TypeScriptCompiler`. Test: `ctest -j 16 -C Release` from the build directory. Build the
+  branch in WSL (GCC) before pushing.
+- Source files are CRLF. Commits are GPG-signed, each ending with the session's `Co-Authored-By:`.
+- Check IR shapes with `--emit=mlir-affine --own-skip-inference` (the IR is on stderr) and, since
+  CSE runs first under `--opt`/JIT, with `--emit=jit --mlir-print-ir-after-all --mlir-disable-threading`.
+- Baseline corpus on this tree: 310 of 591 compile under `-mm=own --no-default-lib`.
+
+## Shapes (measured on 7c543604)
+
+- **The closure** is `V = ts.CreateBoundFunction(B, @f) {__owned_result, __owns_capture}` followed by
+  rc's `ts.Retain(V)`: a fresh value, moved or released like any other. `B` is the box,
+  `ts.Variable {captured, __capture_box}`, filled by `ts.Store(x, ts.PropertyRef(B, i))`.
+- **A call of a folded `const f`** is devirtualized to `ts.SymbolCallInternal(B) @f`: the box is
+  used without `V`. A call through a value is `ts.CallInternal(ts.GetMethod(V), ts.GetThis(V))`. A
+  closure passed on is `ts.Cast(V)` to `!ts.hybrid_func`: the same `{func, this, tag}` triple.
+- **A cell** is `ts.Variable(init) {captured}`: `__owned`/`__owned_consumed` when the frame owns the
+  value, nothing for a parameter's (`this` included) or for a value that owns no heap. rc:
+  `ts.RetainCell(cell)` before the box store, `ts.ReleaseCell(cell)` at the frame's scope exit; the
+  box's release routine (`tsrelcb_`) releases every cell field (value, then cell) and every
+  by-value field. Under own, today's `tsrelcb_` destroys all of it.
+- **A cell inherited by a nested closure** is `ts.Load(ts.PropertyRef(%arg0, i))` of type `!ts.ref<T>`
+  in the enclosing closure's body, `ts.RetainCell`ed for the inner box.
+- **A by-value capture** (a folded `const`) is `ts.Retain(x)` then `ts.Store(x, ts.PropertyRef(B, i))`.
+- **An assignment to a captured variable** is `ts.ReleaseSlot(cell)` + `ts.Store(new, cell)` in
+  the frame, and `ts.ReleaseSlot(ts.Load(ts.PropertyRef(%arg0, i)))` + `ts.Store` in a closure body.
+  Neither is in the inference's `drops`, nor in the signature pass's `dropsInBody`.
+- **A loop** re-creates the box each iteration, releases the closure at the iteration's end, and
+  releases the cells after the loop.
+
+## Rulings made while planning
+
+- **Escape is decided on `V`'s flow, before anything else.** `V` does not escape when every use,
+  through function-type casts and owning `let`s (their loads), is rc's bookkeeping, a borrow
+  (`isBorrow`: a call's argument the callee does not keep, `ts.GetThis`/`ts.GetMethod`), or a call.
+  Anything else escapes: a return (a store into the result slot), a store into a field, element,
+  global or another box, a push, a kept argument, a boxing cast. Cost if wrong: an escaping
+  closure read as borrowing would free its cells at the frame's exit, so anything unknown is an
+  escape.
+- **A non-escaping box borrows everything; an escaping box owns everything.** Mixing per field
+  would need a routine per closure site. Cost: a non-escaping closure cannot take ownership of a
+  value it is given; none needs to.
+- **The closure's uses** are every use of `V` (through casts and owning `let`s) and of `B` (other
+  than its own fill and `V`'s creation). A borrowing closure's uses may not run after an end of
+  anything it borrows:
+  - a cell this frame owns: its `ts.ReleaseCell`s, and the point it moves into an escaping box;
+  - a by-value fresh value: its releases and takers;
+  - a parameter's cell, an inherited cell, a parameter: nothing ends in this frame.
+  A by-value copy of a borrow (`const c = h.c; () => c.x`) is a store of a borrow: an error, as
+  today (`cannot be stored, returned or captured`). Recorded as a limit.
+- **A captured parameter's cell borrows its value.** Its `ts.ReleaseCell` frees the cell only
+  (`__own_cell_borrows`), and nothing may assign it: not the frame, not any closure body that holds
+  the cell, followed through nested captures. The lowering's "a captured variable cannot own its
+  value under -mm=own yet" goes. Cost if wrong: a parameter freed under its caller.
+- **Cells are places.** `ts.ReleaseSlot` of a cell is a drop of every chain whose root this
+  function does not own (a read of a cell is such a root). In the signature pass the same op makes
+  a function drop: the cell is reachable from its creator.
+- **A call given a closure may run it.** A call with an operand that is a closure value, a capture
+  box, or derived from one, drops every chain when its callee may drop: the box may hold anything.
+  Cost: spurious errors where a closure that cannot drop is passed to an unknown callee.
+- **An escaping capture is a move of each cell into the box** (5b). The `ts.RetainCell` is the
+  move: the frame's `ts.ReleaseCell`s it reaches must be dominated by it and are erased; a frame
+  use of the cell reachable after it is a use after move; a second capture reachable after it is
+  too; a capture in a loop of a cell declared outside is moved in a loop. A parameter's cell with
+  a heap value and an inherited cell cannot move: an error. A cell holding a number can. Spec
+  §2.5's "any use after the closure's creation" is this rule.
+- **Phasing.** 5a: non-escaping closures, parameter cells, the drop rules; an escaping closure is
+  `a closure that escapes is not supported by -mm=own yet`. 5b: escaping closures. One PR each.
+
+## Review Focus
+
+1. Escape classification: any use of `V` the classifier does not recognise must be an escape.
+2. A borrowing box's routine frees the box only; an owning box's routine is today's `tsrelcb_`.
+3. Every `ts.ReleaseCell` of a cell whose value is a parameter's carries `__own_cell_borrows`, or the
+   compile fails.
+4. The three drop holes: a frame assignment to a captured variable, a closure body's, and a call
+   that runs a closure assigning a field of a by-value capture. Each has a negative test.
+
+---
+
+### Task 1 (5a): the lowering marks
+
+**Files:** `lib/TypeScript/LowerToLLVM.cpp` (`CreateBoundFunctionOpLowering::getCaptureTypeTag`,
+`ReleaseCellOpLowering`, `VariableOpLowering`), `include/TypeScript/LowerToLLVM/OwnershipRoutineLogic.h`
+(`getOrCreateCaptureBoxFreeRoutine`, `emitReleaseCell` cell-only form),
+`include/TypeScript/LowerToLLVM/LLVMCodeHelper.h` (a descriptor keyed apart for the borrowing box),
+`lib/TypeScript/OwnershipFacts.h` (the mark names).
+
+- [ ] Under own, `__own_borrows_captures` picks a descriptor whose release routine frees the box
+      only (`tsrelcbb_`), keyed apart from the owning one.
+- [ ] Under own, `__own_cell_borrows` lowers `ts.ReleaseCell` to "free the cell if mortal".
+- [ ] `VariableOpLowering`: under own, a captured parameter's cell takes nothing; the error goes.
+
+### Task 2 (5a): non-escaping closures
+
+**Files:** `lib/TypeScript/OwnershipInferencePass.cpp`.
+
+- [ ] Classify every `__owns_capture` closure; an escaping one is the 5a error.
+- [ ] For a borrowing closure: erase each `ts.RetainCell`; treat each box-field store as a borrow;
+      check the closure's uses against the ends of everything it borrows; mark the closure.
+- [ ] Mark each `ts.ReleaseCell` of a parameter's heap cell; reject an assignment to such a cell in
+      the frame or in any closure body holding it.
+- [ ] Remove the blanket `ts.RetainCell` error.
+
+**Tests:** positives `own_closure_borrow.ts` (a local, a by-value `const`, a closure passed as an
+argument, in a loop, nested, `this` and a parameter, the frame assigning a captured local between
+calls), negatives `own_err_closure_outlives_cell.ts` (called after the cell's scope ended),
+`own_err_closure_param_assigned.ts`, `own_err_closure_borrowed_by_value.ts`.
+
+### Task 3 (5a): drops through cells and closures
+
+**Files:** `lib/TypeScript/OwnershipInferencePass.cpp` (`drops`, `dropsChain`),
+`lib/TypeScript/OwnershipSignaturePass.cpp` (`dropsInBody`).
+
+- [ ] `ts.ReleaseSlot` of a cell is a drop of chains rooted in this function's non-owned values.
+- [ ] A closure body assigning a cell may drop.
+- [ ] A call given a closure or a box drops every chain when its callee may drop.
+
+**Tests:** negatives `own_err_cell_assigned_in_frame.ts`, `own_err_cell_assigned_in_closure.ts`,
+`own_err_closure_assigns_captured_field.ts`.
+
+### Task 4 (5b): escaping closures
+
+- [ ] Cells move into an escaping box; by-value captures are takers; the errors of the rulings.
+
+**Tests:** positives `own_closure_escape.ts` (returned over an object local, over a number
+parameter, counter, pushed into an array), negatives `own_err_closure_escape_used.ts`,
+`own_err_closure_escape_twice.ts`, `own_err_closure_escape_param.ts`,
+`own_err_closure_escape_loop.ts`, `own_err_closure_escape_inherited.ts`.
+
+### Task 5: measure, teeth, corpus, spec
+
+- [ ] `measure.ps1` for the positives under all four models.
+- [ ] Teeth: the owning routine for a borrowing box; the frame's `ts.ReleaseCell` kept on a move.
+- [ ] Corpus before/after; spec §16; memory.

tslang/docs/superpowers/specs/2026-09-24-own-memory-model-design.md

@@ -4,7 +4,8 @@ Date: 2026-09-24. Status: design approved in conversation; written review 2026-0
 amendments folded in. Phase 0 merged as #399 (results §11); phase 1
 (moves by reachability) merged as #401 (results §12); phase 2 (borrows for locals) merged as #402
 (results §13); phase 3 (containers and unions) merged as #403 (results §14); phase 4 (function
-signatures and `any`) on branch `own-phase-4` (results §15). Plans in `docs/superpowers/plans/`.
+signatures and `any`) merged as #406-#410 (results §15); phase 5a (closures that do not escape)
+on branch `own-phase-5` (results §16). Plans in `docs/superpowers/plans/`.
 
 ## 1. Purpose
 
@@ -1007,3 +1008,99 @@ checks that the program is rejected against the rc library.
 - From §14.6: a read reached through an interface, borrow chains through assigned borrowers,
   moves after the last use of every borrower, and a temporary taken by a `let` and consumed
   elsewhere.
+
+## 16. Phase 5a results, 2026-10-02
+
+Plan: `docs/superpowers/plans/2026-10-02-own-phase-5.md`. Changed: `OwnershipInferencePass.cpp`,
+`OwnershipSignaturePass.cpp`, `OwnershipFacts.h`, and in the lowering the capture box's
+descriptor and `ts.ReleaseCell`. MLIRGen does not change.
+
+### 16.1 What phase 5a accepts
+
+A closure is `ts.CreateBoundFunction(box, @f) {__owns_capture}`. Its box holds a cell for each
+variable captured by reference and a copy of each `const` captured by value. The pass decides each
+closure over at least one cell before anything else, because the verdict changes what the box's
+stores are.
+
+- **Escape.** The closure value is followed through its views (a bound function and a hybrid one are
+  the same `{func, this, tag}`, so the cast between them is now a view) and through the locals that
+  hold it. It does not escape when every use is rc's bookkeeping, a call given it (not kept), or a
+  call through it (`ts.GetThis`/`ts.GetMethod`, or the devirtualized `ts.SymbolCallInternal(box)` of
+  a folded `const f`). Anything else is an escape: a return, a store into a field, element, global or
+  another box, a kept argument. An escape is `a closure that captures a variable and escapes here is
+  not supported by -mm=own yet` (5b).
+- **A closure that does not escape borrows everything its box holds.** rc's `ts.RetainCell` of each
+  cell and `ts.Retain` of each copy go. The box's stores of copies are reads, not takers
+  (`__own_capture_borrow`, this pass's own mark). The closure carries `__own_borrows_captures`, and
+  its tag names a routine that frees the box and nothing else (`tsfrecb_`).
+- **The closure's uses** are every call given it, called through it or through its box. None may run
+  after an end of what it borrows: the `ts.ReleaseCell`s of a cell this function owns, and the
+  releases and takers of a copy's owner (rootEnds of a fresh value or an owning local). A cell an
+  enclosing closure holds, and a parameter, end nowhere here. A copy of a read out of a container is
+  bounded by the read's own check: walkBorrowed adds the closure's uses to the read's.
+- **A local that owns nothing** (`let f: () => number; f = () => ...`) is an alias of the closure.
+  rc's birth reference reads to the move logic as the store's, which erased the closure's only
+  release. Now the closure keeps its releases, and nothing may run it after them. That holds for a
+  closure over copies only too, whose box otherwise owns its copies as in phase 4.
+- **A captured parameter's cell** (`this` included) borrows the caller's value. Its `ts.ReleaseCell`
+  carries `__own_cell_borrows` and frees the cell only. Nothing may assign the variable: not the
+  frame, and not a closure body. The signature pass pins `__own_assigns_captures` on each closure:
+  the box fields whose cell its body assigns, followed through the closures it builds over the same
+  cell. The inference reads it, because a per-function pass may not look into another function. A
+  non-owning cell with a heap value that is not a parameter's is
+  `'x' is captured by a closure, but -mm=own cannot tell who owns its value here`.
+- **Cells are places.** `ts.ReleaseSlot` of a cell, a captured `ts.Variable` in the frame or a cell
+  read out of a box in a closure body, is a drop of every chain whose root this function does not own.
+  A read of a cell is such a root. In the signature pass the closure-body form makes a function drop.
+- **A call given a closure may run it.** A call whose operand is a closure value, a capture box, or a
+  closure's `this` drops every chain when its callee may drop: the box may hold anything. Before, a
+  call given only the box was not "given" what the box held (`derivedFrom` follows results, and a
+  store has none).
+
+### 16.2 Measured
+
+AOT, `measure.ps1`, in MB: `own_closure_borrow` (thirteen shapes, 20000 rounds) reads gc 6.1, rc 4.3,
+none 1879.7, own 4.3.
+
+### 16.3 Teeth
+
+- With the owning routine for a borrowing box, `own_closure_borrow` fails under AOT (0xC0000005): the
+  closure in `innerBlock` is a `let` released at its block's end, and its box then destroys the cell
+  the frame still reads.
+- With `__own_cell_borrows` ignored, it fails under AOT (0xC0000374): the cell of `param`'s `c` and of
+  `D.read`'s `this` free the caller's objects, which `main` reads after a churn.
+- Under the JIT both pass: LLVM folds the reads of freed memory. The AOT runs are the evidence.
+- Each new negative compiles with its rule switched off. `own_err_closure_param_assigned` then
+  crashes (0xC0000374).
+
+### 16.4 Tests and the corpus
+
+- **Positive:** `own_closure_borrow` (a local, a copy beside a cell, an argument, a loop, nested, an
+  inner block, `this`, a parameter, the frame and the closure assigning the variable, a `let` holding
+  the closure, a `let` aliasing it, a `let` aliasing a closure over copies), under JIT, AOT and the
+  no-counting check.
+- **Negatives:** `own_err_closure` (now: escapes), `own_err_closure_outlives_cell`,
+  `own_err_closure_copy_moved`, `own_err_closure_param_assigned`, `own_err_closure_assigns_param`,
+  `own_err_cell_assigned_in_frame`, `own_err_cell_assigned_in_closure`,
+  `own_err_closure_assigns_captured_field`.
+- The capture box's descriptor has no retain routine under own, as no other descriptor does: it would
+  bring `__tslang_inc_ref` in.
+
+Corpus under `-mm=own --no-default-lib`: 310 of 591 before, 319 after, none lost. The nine new ones
+(`00arrow_generic`, `00capture_in_new_arguments`, `00funcs_expression_generic`,
+`00funcs_generic_arrow`, `00funcs_nesting_capture`, `00stack_test`, `22lambdas`, `241arrayforeach`,
+`25lamdacapture`) pass under test-runner `-mm=own`, AOT and JIT. Of the 25 files the old error
+stopped first: 9 compile, 4 escape (5b), 8 still meet a cell no closure claims (generators, which
+are phase 7, and object literals whose methods capture), 4 stop at an earlier limit.
+
+### 16.5 Known limits (the input to phase 5b)
+
+- A closure over a cell that escapes (§2.5's second half): 5b. A closure over copies only already
+  owns them, as in phase 4, escaping or not.
+- A copy of a borrowed value captured by a closure that does not escape works only when the borrow's
+  own check passes; a copy of a borrow into an escaping closure is an error.
+- A captured local cannot be borrowed by a `let` or moved out of (slotLoadOf and borrowerOf exclude
+  captured variables).
+- A non-escaping closure always borrows: one that could own its captures instead (called after the
+  cell's block ends, the variable not used again) is rejected.
+- A call given any function value drops every chain when its callee may drop.