Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/own-memory-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,10 @@ m4_record_err.ts:8:1: error: 'pairOf' returns a record that borrows 'a', 'b', wh
the generator, but it may not outlive `p`. It also goes on a function that returns such a
generator. On anything else it is an error.

A generator borrows its parameters without saying so, `this` among them: an extension
generator such as `function* keys<T>(this: T[])`, called `a.keys()`, may not outlive `a`. A function
that returns one says `@borrows("this")`, as `[Symbol.iterator]` of an array does.

```typescript
@final
export class Word {
Expand Down Expand Up @@ -742,6 +746,8 @@ a reference gives it back on every path out of a function.
as in JavaScript: `const seq = fibonacci().map(f)`. Over an iterator that borrows its container
(`s.values()` of a `Map` or a `Set`) a helper is bounded by the container too:
`for (const v of s.values().take(2))` works, but the result cannot be returned or kept past `s`.
An array's `values()`, `keys()` and `[Symbol.iterator]()` borrow it in the same way;
`entries()` does not compile yet, its `[i, a[i]]` holding a borrowed element.
A closure that captures, given to two of them, is moved into the first.
- `Shared<T>` cycles are never freed.
- There are no lifetime annotations: a borrow that the compiler cannot follow is an error, even
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4611,3 +4611,33 @@ JavaScript: `take(Infinity)` converted `Infinity` to an `index`, which is no par
line took all three under rc and none under own. `reduce`'s string accumulator, listed in
§38 as open, is DefaultLib #45 (its accumulator typed `V`, not the `any` that `?? iter.next().value`
gives).

## 40. A generator over its `this` parameter, 2026-10-27

`a.keys()` and `a.values()` of an array (DefaultLib `array_keys`, `array_values`, `array_iterator`)
were "'this' is a parameter, kept here, but -mm=own cannot make the callers of '___Array.keys' give it
up". Both are generators whose first parameter is `this` (`function* keys<T>(this: T[])`). The
generator lowering (`buildGeneratorWrapperDeclaration`) gives `next` the body's `this` through an
alias, `const .this = this`, because `next` has its own. For a method that is the class's `this`,
which the state holds by value (`readsParameterSlot`, §19). For a `this` parameter the alias was a
local of its own, initialised from the parameter, captured into the state's box as a cell: not the
parameter's cell (`isParameterCell` wants the cell to be the parameter), so the state owned a copy
of the caller's array, and the caller could not give it up.

The alias is now a `ConstRef` (`InternalFlags::ForceConstRef`) when `this` is a parameter: the name is
the parameter's storage, as for-in's `.a` is its array's. A `ConstRef` is otherwise never captured
(`setIgnoreCapturing`, for `.a`); `.this` is, so the closure captures the parameter itself - its
`ts.Param` marked captured, as any parameter a generator reads - and the state borrows it like one:
bounded by `this` (`__own_result_bounded`), walked while the array lives, returned only by a function
that says `@borrows("this")`. Under the other models the alias was a second reference to the same
array; now it is the parameter's cell, so nothing else changes.

`own_generator_this_param` walks an extension generator over a `this: C[]` in a `for...of`, a generic
one held in a local, one returned by a `@borrows("this")` function, and one over a temporary array in
the statement that made it, churning between steps, under own, rc, gc and none (without the change,
`@borrows("this")` on the function is refused: nothing it returns borrows). Negative:
`own_err_generator_this_param_returned`, returned by a function that does not say it.

DefaultLib: `array_keys`, `array_values` and `array_iterator` pass under own; `iterator<T>(this: T[])`
(`[Symbol.iterator]`), which returns `this.values()`, says `@borrows("this")`. Not here: `entries()`,
whose `yield [i, this[i]]` puts a borrowed element in a tuple the caller owns.
13 changes: 12 additions & 1 deletion tslang/lib/TypeScript/MLIRGenFunctions.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -867,6 +867,7 @@ namespace mlirgen
FunctionLikeDeclarationBase functionLikeDeclarationBaseAST, mlir::Location location)
{
auto fixThisReference = functionLikeDeclarationBaseAST == SyntaxKind::MethodDeclaration;
auto thisIsParameter = false;
if (functionLikeDeclarationBaseAST->parameters.size() > 0)
{
auto nameNode = functionLikeDeclarationBaseAST->parameters.front()->name;
Expand All @@ -876,6 +877,7 @@ namespace mlirgen
if (ident->escapedText == S(THIS_NAME))
{
fixThisReference = true;
thisIsParameter = true;
}
}
}
Expand Down Expand Up @@ -989,7 +991,16 @@ namespace mlirgen
// TODO: this is temp hack, add this alias as thisArg,
NodeArray<VariableDeclaration> _thisArgDeclarations;
auto _thisArg = nf.createIdentifier(S(THIS_ALIAS));
_thisArgDeclarations.push_back(nf.createVariableDeclaration(_thisArg, undefined, undefined, nf.createToken(SyntaxKind::ThisKeyword)));
auto _thisArgDeclaration = nf.createVariableDeclaration(_thisArg, undefined, undefined, nf.createToken(SyntaxKind::ThisKeyword));
// a `this` parameter (`function* keys<T>(this: T[])`): the alias is the parameter's own
// storage, so the generator captures the parameter, as it does any other - under
// -mm=own its state borrows it (spec 19) rather than owning a copy of it
if (thisIsParameter)
{
_thisArgDeclaration->internalFlags |= InternalFlags::ForceConstRef;
}

_thisArgDeclarations.push_back(_thisArgDeclaration);
auto _thisArgList = nf.createVariableDeclarationList(_thisArgDeclarations, NodeFlags::Const);

generatorStatements.push_back(nf.createVariableStatement(undefined, _thisArgList));
Expand Down
6 changes: 4 additions & 2 deletions tslang/lib/TypeScript/MLIRGenImpl.h
Original file line number Diff line number Diff line change
Expand Up @@ -2156,8 +2156,10 @@ class MLIRGenImpl
if (!isConst || varClass == VariableType::ConstRef || needsIdentityStorage)
{
varDecl->setReadWriteAccess();
// TODO: HACK: to mark var as local and ignore when capturing
if (varClass == VariableType::ConstRef)
// TODO: HACK: to mark var as local and ignore when capturing. Not a generator's alias
// of its `this` parameter (buildGeneratorWrapperDeclaration), which its `next`
// captures: it captures the parameter's storage
if (varClass == VariableType::ConstRef && fullName != THIS_ALIAS)
{
varDecl->setIgnoreCapturing();
}
Expand Down
10 changes: 6 additions & 4 deletions tslang/test/tester/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3937,7 +3937,8 @@ foreach(own_test own_fresh_string own_fresh_array own_return_new own_try_local o
own_call_field_drops own_immortal_keepers
own_generator_method_this own_string_copy_call own_param_kept_some_paths own_method_callbacks
own_loop_read_again own_length_drops_elements own_tuple_param_kept
own_for_of_borrowed_iterator own_generator_keeps own_generator_keeps_bounded)
own_for_of_borrowed_iterator own_generator_keeps own_generator_keeps_bounded
own_generator_this_param)
tslang_add_test(NAME test-jit-own-${own_test} COMMAND test-runner -jit -mm=own "${PROJECT_SOURCE_DIR}/test/tester/own/${own_test}.ts")
tslang_add_test(NAME test-compile-own-${own_test} COMMAND test-runner -mm=own "${PROJECT_SOURCE_DIR}/test/tester/own/${own_test}.ts")
# The promise: no counting at all in what own emits.
Expand Down Expand Up @@ -3965,9 +3966,9 @@ foreach(for_of_mm rc gc none)
endforeach()

# A generator whose `@keeps` makes its state own its arguments (spec 38), and one given a generator
# that borrows; the same programs under every other model.
# that borrows, and one over its `this` parameter; the same programs under every other model.
foreach(keeps_mm rc gc none)
foreach(keeps_test own_generator_keeps own_generator_keeps_bounded)
foreach(keeps_test own_generator_keeps own_generator_keeps_bounded own_generator_this_param)
tslang_add_test(NAME test-jit-${keeps_mm}-${keeps_test} COMMAND test-runner -jit -mm=${keeps_mm} "${PROJECT_SOURCE_DIR}/test/tester/own/${keeps_test}.ts")
tslang_add_test(NAME test-compile-${keeps_mm}-${keeps_test} COMMAND test-runner -mm=${keeps_mm} "${PROJECT_SOURCE_DIR}/test/tester/own/${keeps_test}.ts")
endforeach()
Expand Down Expand Up @@ -4641,7 +4642,8 @@ set(own_error_cases
"own_err_generator_keeps_missing|:9:1: error: '@keeps' of 'mapped<[^']*' names 'iter', but its body keeps 'iter', 'f'"
"own_err_generator_keeps_used_after|:15:11: error: '[^']*' is used here after its value was moved"
"own_err_generator_keeps_bounded|:14:17: error: '[^']*' borrows its generator's arguments and cannot be stored, returned or captured"
"own_err_generator_keeps_bounded_outlives|:20:11: error: '[^']*' borrows '[^']*' but is used here after '[^']*' is released or overwritten")
"own_err_generator_keeps_bounded_outlives|:20:11: error: '[^']*' borrows '[^']*' but is used here after '[^']*' is released or overwritten"
"own_err_generator_this_param_returned|:11:26: error: '[^']*' borrows its generator's arguments and cannot be stored, returned or captured")
foreach(own_error_case ${own_error_cases})
string(FIND "${own_error_case}" "|" own_error_sep)
string(SUBSTRING "${own_error_case}" 0 ${own_error_sep} own_error_name)
Expand Down
18 changes: 18 additions & 0 deletions tslang/test/tester/own/own_err_generator_this_param_returned.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
// -mm=own rejects: a generator that borrows its `this` parameter returned by a function that does not
// say `@borrows("this")`.
class C {
constructor(public x: number) {}
}

function* xs(this: C[]) {
for (let i = 0; i < this.length; i++) yield this[i].x;
}

function walk(this: C[]) {
return this.xs();
}

function main() {
const a = [new C(1)];
for (const v of a.walk()) print(v);
}
59 changes: 59 additions & 0 deletions tslang/test/tester/own/own_generator_this_param.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
// A generator whose first parameter is `this` (`function* keys<T>(this: T[])`, called `a.keys()`)
// borrows it as it does any other parameter (spec 19, 40): it is walked while the array lives, and a
// function that returns it says `@borrows("this")`. Each case churns between steps, so an array freed
// early, or a copy of it kept, reads garbage.
class C {
v: number[] = [];
constructor(public x: number) {}
}

function churn() {
const junk: C[] = [];
for (let i = 0; i < 64; i++) {
const c = new C(999);
c.v.push(99);
junk.push(c);
}
}

function* xs(this: C[]) {
for (let i = 0; i < this.length; i++) yield this[i].x;
}

function* indices<T>(this: T[]) {
for (let i = 0; i in this; i++) yield i;
}

@borrows("this")
function walk(this: C[]) {
return this.xs();
}

function make(n: number) {
const a: C[] = [];
for (let i = 1; i <= n; i++) a.push(new C(i));
return a;
}

function main() {
let t = 0;
for (let i = 0; i < 1000; i++) {
const a = make(3);
for (const v of a.xs()) {
churn();
t += v;
}

const g = a.indices();
churn();
for (const k of g) t += k;

for (const v of a.walk()) t += v;

// a temporary array, in the statement that walks it
t += make(2).xs().next().value;
}

assert(t == 1000 * (6 + 3 + 6 + 1), "t");
print("done.");
}
Loading