-mm=own phase 7b: a generator over a parameter's block borrows it; .map and .filter

tslang/docs/superpowers/plans/2026-10-02-own-phase-7b.md

@@ -0,0 +1,146 @@
+# `-mm=own` Phase 7b Implementation Plan: generators that borrow
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development
+> (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use
+> checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Under `-mm=own`, a generator over a parameter that holds a block compiles: `function*
+each(a: number[]) { for (const v of a) yield v; }`. This is Rust's `fn each<'a>(a: &'a [f64]) ->
+impl Iterator + 'a`. The state object is owned by the caller, but it borrows the argument, and
+it may not outlive it. Then `.map`/`.filter`, which MLIRGen builds as such a generator over the
+source array.
+
+**Architecture:**
+
+1. **The signature pass decides which fields of a state object borrow.** It is a module pass, and
+   the per-function inference is parallel and cannot share state. It writes the decision as a
+   module attribute that three places read:
+   - the lowering, which builds the state object's release routine;
+   - the inference in `next`;
+   - the inference in callers.
+
+   A state type is unique to its generator (its `next` field's type names a per-site
+   `object_storage`), so a fact per type is a fact per generator.
+2. **The lowering** skips a borrowed field in the state object's release routine. For a borrowing
+   `.captured` box, it calls a box routine that frees each cell but not what the cell holds.
+3. **The maker's result is bounded by its arguments**: `__own_result_bounded = [indices]`. This is
+   not `__own_result_borrows`, which means "the result is a borrow" (no release, not fresh).
+   Bounded means owned (fresh, released as usual) and tenured: no use of the result may run
+   after an end of a bounding argument, and the result may not be stored, returned or captured.
+
+**Spec:** §18.6 (this phase's input). Results go into §19.
+
+## Global Constraints
+
+As in phase 7a (plan `2026-10-02-own-phase-7.md`). Also:
+
+- Clear `test/tester/own/__jit/<test>.ts.*` between JIT teeth runs. The cache ignores environment
+  variables.
+- Branch `own-phase-7b` from `origin/main` (af7b150f, with #439 merged).
+
+## Shapes (measured on af7b150f)
+
+- **The maker** of `each(a: number[])`:
+  1. `a`'s cell is `ts.Variable(%arg0) {captured}` (not owning), `ts.RetainCell`ed.
+  2. It is stored into the box `ts.Variable {captured, __capture_box}`, which is `ts.RetainCell`ed
+     and stored into `.captured` of the initial state.
+  3. The frame's `ts.ReleaseCell(cell)` follows.
+
+  Today this is 5b's "its value is the caller's".
+- **`next`** reads the cell out of the box (`ts.Load(ts.PropertyRef(box, i))`). The `for...of`
+  copies the array into a state field `.a`: `ts.Retain(v)`, `ts.Store(v, PropertyRef(state, .a))`.
+- **The caller** keeps the generator in an owning `let` and calls `next` through it, as in 7a.
+- **`.map(f)`**: the caller builds a closure whose box holds a copy of the source array (rc
+  `ts.Retain`s it) and the callback, and calls it at once (devirtualized `ts.SymbolCallInternal`).
+  That closure body (`..iter`) is the maker. It copies its box's fields into a new box (another
+  `ts.Retain`) and stores that box into the state object.
+
+## Rulings made while planning
+
+- **A borrowing field**, for a generator state type `T` (it has a `next` in `fieldFunctions`) whose
+  maker `M` is closed (private, every caller known):
+  - **The box.** The `.captured` box borrows when every cell `M` fills it with is a parameter's
+    cell. Its cells then move into the box, which frees them and not their values.
+  - **A state field `p`** borrows when every store into `p`, anywhere in the module, stores a
+    value rooted at a read of a borrowing cell or of another borrowing field. This is computed as
+    a fixpoint. One store of anything else, and `p` owns as today.
+- **`M`'s result is bounded** by each parameter whose cell holds a block and goes into a borrowing
+  box. `__own_result_bounded` is pinned on `M` and on each call that resolves to it.
+- **At a call with a bounded result**, the result is fresh, and owned by whatever holds it, as
+  today. Every use of it, followed through views and the `let`s that hold it, may not be reachable
+  after any end of a bounding argument's root:
+  - its releases;
+  - an assignment to it;
+  - a drop of the place it was read from.
+
+  A use that keeps the result (a store into a container, a return, a capture) is an error. Cost if
+  wrong: a generator that reads its argument after the caller freed it.
+- **In `next`:**
+  - a store into a borrowing field erases rc's retain and takes nothing;
+  - a `ts.ReleaseSlot` of a borrowing field frees nothing (`__own_slot_borrows`, read by the
+    lowering, as `__own_cell_borrows` is);
+  - a read of a borrowing cell or field is a borrow with no end in `next`, as a parameter cell's
+    is in 5a.
+- **Phasing:** 7b-1 is tasks 1-4 (parameters). 7b-2 is task 5 (`.map`/`.filter`). Yielding a borrow
+  stays a limit: the result record type `{value: C, done}` is shared between generators, so a fact
+  per type does not work for it.
+
+## Review Focus
+
+1. A field is marked borrowing only when nothing but borrows is ever stored into it.
+2. The bounded check follows every derived use of the result, and treats anything it does not
+   know as a keep.
+3. An open maker gets no facts, and stays the 5b error.
+
+---
+
+### Task 1: the decision (signature pass)
+
+- [x] Find each generator state type and its maker. Decide the borrowing box and fields. Write the
+      module attribute `__own_borrowing_fields`: per state type, the borrowing field indices, with
+      the `.captured` index meaning a borrowing box.
+- [x] Pin `__own_result_bounded` on the maker and on its calls.
+
+### Task 2: the lowering
+
+- [x] `releaseFields`, under own: skip a borrowing field. For a borrowing `.captured`, call the box
+      routine that frees cells only (`tsrelcbc_`).
+- [x] `ts.ReleaseSlot {__own_slot_borrows}` frees nothing.
+
+### Task 3: the maker and `next` (inference)
+
+- [x] The maker: a parameter's heap cell moves into a borrowing state box. There is no 5b error,
+      and the frame's `ts.ReleaseCell` goes.
+- [x] `next`: stores into borrowing fields, and their releases, as the rulings say.
+
+### Task 4: callers (inference)
+
+- [x] The bounded check.
+
+**Tests:** positive `own_generator_borrow.ts`:
+- an array parameter iterated;
+- a class parameter whose fields are yielded;
+- a string parameter, measured;
+- a generator given up early;
+- two generators over one argument.
+
+Negatives:
+- `own_err_generator_outlives_arg` (the argument is assigned while the generator is still used);
+- `own_err_generator_bounded_escapes` (returned from the function that owns the argument);
+- `own_err_generator_yields_borrow` (the parameter itself yielded), which replaced
+  `own_err_generator_bounded_stored`: a store is the same keep check as the return.
+
+### Task 5: `.map` and `.filter`
+
+- [x] The same, through the closure `..iter` and its by-value box. The caller's box is a borrowing
+      closure (5a), so the copy is a borrow: confirm this before assuming it.
+
+**Tests:** `own_generator_borrow.ts` gets `map`, `filter`, and a capturing `filter` (a chained `map`
+fails to compile under every model). Negative: `own_err_generator_map_outlives`.
+
+### Task 6: measure, teeth, corpus, spec §19, memory
+
+- [x] Teeth:
+  - the owning box routine for a borrowing box (the caller's argument freed twice);
+  - a borrowing field released (the same);
+  - the bounded check skipped (the negatives compile).

tslang/docs/superpowers/specs/2026-09-24-own-memory-model-design.md

@@ -6,7 +6,8 @@ amendments folded in. Phase 0 merged as #399 (results §11); phase 1
 (results §13); phase 3 (containers and unions) merged as #403 (results §14); phase 4 (function
 signatures and `any`) merged as #406-#410 (results §15); phase 5a (closures that do not escape)
 merged as #436 (results §16); phase 5b (closures that escape) merged as #437 (results §17); phase
-7a (generators, async) on branch `own-phase-7` (results §18). Plans in `docs/superpowers/plans/`.
+7a (generators, async) merged as #439 (results §18); phase 7b (generators that borrow) on branch
+`own-phase-7b` (results §19). Plans in `docs/superpowers/plans/`.
 
 ## 1. Purpose
 
@@ -1312,9 +1313,10 @@ Found, not fixed:
 - **A method of an object literal held in a `let`**, returning a block, called and kept, is a
   borrow error: `let o = {make(): C {...}}; const c = o.make()`. A `const o` works.
 - **`measure.ps1` reads `compile.bat`**, which only a test-runner run without `-mm` writes.
-- **`await f()` returns before `f`'s `for await` body has run**, under every model: with
-  `for await (const x of g()) keep.push(1)`, `keep` is empty after it (`none` fails the assert,
-  `gc` too).
+- **An `await` of a function that returns nothing never awaits** (#440, every model): MLIRGen
+  exits before building the `async.await` when the awaited call has no value, so `await f()`
+  continues at once and `f` may never run. That is what made the `for await` facts above
+  impossible to test at run time.
 
 ### 18.6 Known limits (the input to 7b)
 
@@ -1325,3 +1327,108 @@ Found, not fixed:
 - `.map`/`.filter`, which MLIRGen builds as generators over the source array and the callback, stop
   on "borrows a field and cannot be stored".
 - A yielded borrow (a parameter's element, a state field's value) is an error.
+
+## 19. Phase 7b results, 2026-10-02
+
+Plan: `docs/superpowers/plans/2026-10-02-own-phase-7b.md`. Changed: `OwnershipSignaturePass.cpp`,
+`OwnershipInferencePass.cpp`, `OwnershipFacts.h`, `Defines.h`, and `OwnershipRoutineLogic.h`
+(the lowering).
+
+### 19.1 What phase 7b accepts
+
+A generator over a parameter that holds a block, `function* each(a: number[])`, borrows it: Rust's
+`fn each<'a>(a: &'a [f64]) -> impl Iterator + 'a`. The caller owns the state object, which may not
+outlive the argument, and whose release gives the argument back to nobody.
+
+- **The decision is the signature pass's.** The release routine is built once per type, and the
+  inference is per function and parallel, so neither can make it. A state type is unique to its
+  generator: its `next`'s type names a per-site `object_storage`. For a state type whose maker is
+  closed, and is the type's only `ts.New`:
+  - the `.captured` box borrows when every fill is a parameter's cell, or a copy read out of a
+    parameter's field (`.map`'s maker is a closure body that copies its box);
+  - a state field borrows when everything stored into it, in the whole module, is a read of a
+    borrowing cell or field (a least fixpoint). The `for...of` copy of the array is such a field.
+
+  It writes `ts.own_borrowing_fields` on the module (per type, the box's index first), and pins
+  `__own_result_bounded`, the parameters whose blocks the state borrows, on the maker and its calls.
+  A maker is closed when it is private and its symbol is used only by direct calls, and by closures
+  that are only retained and released (`.map` builds one and calls its body through the box).
+- **The lowering**, under own, skips a borrowing field in the state object's release routine. A
+  borrowing box gets `tsrelcbc_`, which frees each cell and none of what the cells or by-value
+  fields hold.
+- **In the maker**, a parameter's cell moves into a borrowing box: no 5b error, and the frame's
+  `ts.ReleaseCell` goes. A copy stored into a borrowing box is a borrow.
+- **In `next`**:
+  - a store into a borrowing field takes nothing: rc's retain goes, and so does its release when
+    the field is overwritten;
+  - what is read out of a borrowing cell, field or box field is the caller's, and may only be
+    borrowed. Yielding it is an error (`own_err_generator_yields_borrow`): it would give the
+    caller's value a second owner, and rc retains nothing there for anything else to see;
+  - a read of a borrowing box's field is not a place read: only the maker fills the box, so no call
+    overwrites it.
+- **At the caller**, the generator is owned as any fresh value, and bounded:
+  - no use of it may come after an end of a bounding argument. Its uses are followed through the
+    locals that hold it and the `next` it is called through. An end is the argument's root's
+    releases, assignments and moves, or, for a read out of a place, what may destroy the place.
+    Its releases are not uses: the routine gives back nothing it borrows;
+  - it may not be stored, returned or captured (`own_err_generator_bounded_escapes`);
+  - a bounding capture box (`.map`) ends where the closures on it are released, and where what it
+    was filled from ends (`own_err_generator_map_outlives`).
+- **`.map` and `.filter`.** The caller's closure box holds a copy of the array. A box of copies
+  that does not escape, is given to a bounded maker, and holds nothing fresh borrows its copies, so
+  two `.map`s of one array no longer move it twice. A callback that captures is a fresh closure
+  made for the box: the box then owns its copies, and the array moves in. rc reads the array twice
+  for one copy and retains the first read: the second is the same value, both where a taker is
+  looked for and where a box field's retain is.
+
+### 19.2 Measured
+
+AOT, `measure.ps1`, in MB:
+
+| test | gc | rc | none | own |
+|---|---|---|---|---|
+| `own_generator_borrow`, 20000 iterations | 6.1 | 4.3 | 1441.5 | 4.3 |
+| the same, 200000 iterations | 6.1 | 4.3 | 14376.5 | 4.3 |
+
+The ten-times run is the leak check. It found one in this phase's own work: the first form of the
+copies rule borrowed a fresh capturing callback, which nobody then freed (own 10.5 MB, rc 4.3).
+
+### 19.3 Teeth
+
+Each made with a temporary `getenv` switch, the JIT cache cleared between runs:
+
+- **The owning box routine for a borrowing box:** `own_generator_borrow` fails, AOT
+  (0xC0000374) and JIT.
+- **A borrowing field released:** the same.
+- **The bounded check skipped:** `own_err_generator_outlives_arg`,
+  `own_err_generator_bounded_escapes` and `own_err_generator_map_outlives` compile.
+- **The borrowed-read check skipped:** `own_err_generator_yields_borrow` compiles.
+
+### 19.4 Tests and the corpus
+
+- **Positive:** `own_generator_borrow`. It covers an array, a class and a string parameter, a
+  generator given up early, two generators over one argument, an argument read out of a field,
+  `map`, `filter`, and a capturing `filter` last.
+- **Negatives:** `own_err_generator_outlives_arg`, `own_err_generator_bounded_escapes`,
+  `own_err_generator_yields_borrow`, `own_err_generator_map_outlives`,
+  `own_err_generator_place_overwritten` (the field the argument was read from is assigned).
+- **Removed:** `own_err_generator_param_object` compiles now and is gone.
+
+Corpus: 359 of 591 before, 364 after, none lost: `00funcs_generic_iterator`, `01disposable`,
+`02disposable` (an object literal whose methods capture a parameter is the same shape),
+`00filter`, `00class_iterator_super`. Each runs under test-runner `-mm=own`, AOT and JIT.
+
+Found, not fixed (every model): `a.map(f).map(g)` fails to compile ("the condition has no value").
+
+### 19.5 Known limits
+
+- **A yielded borrow** (an element of a parameter's array, the parameter itself) is an error. The
+  result record type `{value: C, done}` is shared between generators, so a fact per type cannot
+  say that a record borrows.
+- **A capturing `.map`/`.filter` callback** moves the array into the closure's box: a later use of
+  the array is a use after move.
+- **A bounded generator cannot be returned** past the argument it borrows, even when the
+  argument is the returning function's own parameter (the fact does not propagate).
+- **A copy a borrowing field takes of a sub-field** (`const v = c.v` in the generator) is not
+  borrowing, so it is the existing "cannot be stored" error.
+- **Without `--di`**, the bounded errors name "this value".

tslang/include/TypeScript/Defines.h

@@ -74,6 +74,12 @@
 // `this`), so the release frees the cell and leaves the value to its owner.
 #define OWN_BORROWS_CAPTURES_ATTR_NAME "__own_borrows_captures"
 #define OWN_CELL_BORROWS_ATTR_NAME "__own_cell_borrows"
+// -mm=own only, set by OwnershipSignaturePass on the module for the lowering and the inference: the
+// fields of a generator's state object that borrow what they hold from the generator's arguments,
+// as [[type, [field...]], ...] keyed by the state's tuple type, the `.captured` box's index first.
+// Its release routine skips them, and a borrowing `.captured` box frees its cells but not what
+// they hold.
+#define OWN_BORROWING_FIELDS_ATTR_NAME "ts.own_borrowing_fields"
 #define RETURN_VARIABLE_NAME ".return"
 #define CAPTURED_NAME ".captured"
 #define LABEL_ATTR_NAME "label"