Repository navigation
-mm=own phase 7b: a generator over a parameter's block borrows it; .map and .filter - #441
Merged
Merged
Conversation
`function* each(a: number[])` keeps `a`'s cell in a box in its state object's `.captured` field, and its `for...of` copies `a` into a state field. Under 5b that was "its value is the caller's". Now the state object borrows, Rust's `-> impl Iterator + 'a`: - The signature pass finds each generator state type (unique per site: its `next` names a per-site `object_storage`) whose closed maker fills its box with parameters' cells only, and decides which fields borrow: the box, and each field everything stored into is a read of a borrowing cell or field (a least fixpoint). It writes them as the module attribute `ts.own_borrowing_fields`, and pins `__own_result_bounded` (the parameters holding blocks) on the maker and its calls. - The lowering skips a borrowing field in the state object's release routine, and gives a borrowing box the new routine `tsrelcbc_`, which frees each cell but not what it holds. - The inference lets a parameter's heap cell move into a borrowing box. In `next`, a store into a borrowing field takes nothing (rc's retain goes, its release on overwrite goes), and what is read out of a borrowing cell or field may only be borrowed: yielding it would give the caller's value a second owner, and rc retains nothing there for anything else to notice (`own_err_generator_yields_borrow`). - At a call with a bounded result the generator is owned as usual, but no use of it - through its locals and the `next` it is called through - may come after an end of a bounding argument, and it may not be stored, returned or captured (`own_err_generator_outlives_arg`, `own_err_generator_bounded_escapes`). Its releases are not uses: the routine gives back nothing it borrows. `own_generator_borrow` covers an array, a class and a string parameter, a generator given up early and two over one argument. `own_err_generator_param_object` compiles now and is gone. Corpus: 359 -> 362 of 591 (`00funcs_generic_iterator`, `01disposable`, `02disposable`; an object literal whose methods capture a parameter is the same shape), each run under test-runner -mm=own, AOT and JIT, with gc's output. Plan: docs/superpowers/plans/2026-10-02-own-phase-7b.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MLIRGen builds `a.map(f)` as a closure whose box holds a copy of `a` and `f`, called at once
through the box: its body is the generator's maker, and it copies its box's fields by value into
the generator's own box.
- The signature pass takes a box copied from a parameter's fields as borrowing, bounded by that
parameter, and a maker whose symbol is used only by direct calls and closures nothing but
retains and releases as closed. In `next`, a by-value field of a borrowing box is a borrowing
read too.
- The inference: a store into a borrowing state box is a borrow; a read of a borrowing box's field
is the caller's (no place read: only the maker fills the box, so no call overwrites it - before,
a `filter` callback called in a loop "dropped" the field it was read from); a closure box that
holds copies only, does not escape, and is given to such a maker borrows its copies, so two
`.map`s of one array no longer move it twice.
- At the caller, a bounding box ends where the closures on it are released and where what it was
filled from ends (`own_err_generator_map_outlives`). The box is a `ref`, so it is checked before
the test that skips arguments holding no block.
- rc reads a place twice for one boxed copy and retains the first read: the second is the same
value, both where a taker is looked for and where rc's retain for a box field is.
`own_generator_borrow` gains `map` and `filter`. Corpus: 362 -> 364 of 591 (`00filter`,
`00class_iterator_super`), none lost, each run under test-runner -mm=own, AOT and JIT.
Found, not fixed (every model): `a.map(f).map(g)` fails to compile ("the condition has no
value").
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r of its own The previous commit let a copies-only closure box given to `.map`/`.filter`'s maker borrow everything it holds. A callback that captures (`a.filter((x) => x > n)`) is a closure made for that box, which rc hands its reference: borrowed, it was nobody's, a leak of 31 bytes a call that measure.ps1 showed (rc 4.3 MB, own 10.5 MB over 200000 calls). The box now borrows only when no copy it holds is fresh; otherwise it owns its copies as in phase 4, and the array moves into it. `own_generator_borrow` puts its capturing `filter` last, after the array's other uses, and makes its string in a helper (a temporary in front of a `yield` leaks under rc and own alike, spec 18.5). It measures flat: 4.3 MB at 20000 and at 200000 iterations, as rc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Results, teeth, measurements and limits of phase 7b. Section 18.5's await item is #440 now: an await of a function that returns nothing never awaits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ead's barrier The bounded check's branch for an argument read out of a place had no test. `each(h.items)` now runs in `own_generator_borrow`, and `own_err_generator_place_overwritten` assigns `h.items` while the generator is used. twinReadAfter stopped at a store into the place or a call; it now stops at any op given the place (a push writes through it too). MLIRGen emits no such op there today; the rule runs on every move out of a slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 7b of
-mm=own: generators that borrow. The plan isdocs/superpowers/plans/2026-10-02-own-phase-7b.mdand the results are in spec §19.A generator over a parameter that holds a block now compiles:
function* each(a: number[]) { for (const v of a) yield v; }. In Rust terms that'sfn each<'a>(a: &'a [f64]) -> impl Iterator + 'a. The caller owns the generator's state object, but the state object borrows the argument and may not outlive it..mapand.filter, which MLIRGen builds as such a generator over the source array, compile too.How it works
nextnames a per-siteobject_storage..capturedbox borrows when every fill is a parameter's cell, or a copy read out of a parameter's field.for...ofcopy of the array is such a field.ts.own_borrowing_fields, and__own_result_bounded(the parameters whose blocks the state borrows) on the maker and on each call to it.tsrelcbc_, which frees each cell but none of what the cells or the by-value fields hold..map, the bound is the closure box: it ends where the closures on it are released, and where what it holds ends..mapand.filter:.maps of one array no longer move it twice.For review
calledOnlyDirectlyis a second closed-world test, which disagrees withfindOpenby design.findOpenmarks.map's maker (..iter) open, because ats.SymbolRefnames it in ats.CreateBoundFunction. The bounded fact is pinned anyway. The argument is that the maker is private, it is the onlyts.Newof its state type, and every closure over it is dead: only retained and released, with the real call made directly through the box.ts.own_borrowing_fields) to pick a release routine. It's own-only:borrowingFieldsOfis empty under rc, gc and none, and whenever the attribute is absent.takerOfandretainBeforefor every move out of a slot. The evidence that it's safe is that the corpus didn't move (364 before and after it, none lost). Its barrier stops at any op given the place, and at any call.Verification
Release suite: 3389 of 3390 pass. The one failure,
test-compile-gc-defaultlib-collector, is the same stale installed default library as on -mm=own phase 5a: a closure that does not escape borrows what it captures #436, -mm=own phase 5b: a closure that escapes owns what it captures; a call in a try body is a call #437 and -mm=own phase 7a: generators and async; an interface is a view of its object #439. All 388 own tests pass.WSL/GCC: builds, and 387 of 387 own tests pass (Windows has one more, the Windows-only shared-library check).
Corpus: 359 → 364 of 591, none lost:
00funcs_generic_iterator,01disposable,02disposable(an object literal whose methods capture a parameter is the same shape),00filter,00class_iterator_super. Each passes under test-runner-mm=own, AOT and JIT.Memory:
own_generator_borrowmeasures flat, the same as rc, at both sizes:The ten-times run caught a leak in this PR's first form of the copies rule (a borrowed capturing callback, which nothing then freed). It's fixed in 12a7bd2.
Teeth: each new rule was undone with a temporary switch:
own_generator_borrow, AOT (0xC0000374) and JIT;Tests
own_generator_borrow: an array, a class and a string parameter; a generator given up early; two generators over one argument; an argument read out of a field;map;filter; and a capturingfilteras the array's last use.own_err_generator_outlives_arg,own_err_generator_bounded_escapes,own_err_generator_yields_borrow,own_err_generator_map_outlives,own_err_generator_place_overwritten.own_err_generator_param_object, which now compiles.Found, not fixed (every model)
a.map(f).map(g)fails to compile with "the condition has no value": #442. Without the default library, the built-in.mapreturns a generator, and the real error is lost in a condition.Known limits (spec §19.5)
.map/.filtercallback moves the array into the closure's box.🤖 Generated with Claude Code