From 99f415247307e7b3b5fe6c4d922c4752adb1c6ee Mon Sep 17 00:00:00 2001 From: simontreanor Date: Mon, 28 Sep 2026 03:12:21 +0100 Subject: [PATCH] docs: tidy the native sections of ROADMAP and DESIGN Tonight's slices each appended to the same ROADMAP paragraph and DESIGN 5.6 until both read as a log. ROADMAP now states what landed, the measurement and what remains; DESIGN 5.6's second half is split into the CPython effect, annotations (including typed data class fields) and building. --- DESIGN.md | 40 +++++++++++++++++++++++----------------- ROADMAP.md | 31 ++++++++++++++----------------- 2 files changed, 37 insertions(+), 34 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 1ce0779..49f57a9 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -432,23 +432,29 @@ has no `match` statement left, and every recorded replay prints the same in both The ladder is also faster on CPython by itself, since a class pattern goes through `__match_args__` and positional binding at run time: on `bench/expr_eval` (ADT allocation and matching) the output goes from 2.30x to 1.63x of the hand-written baseline on CPython 3.14, before any compilation. -A data class field whose type has no Python class of its own (a sum type, a type variable) is -annotated `typing.Any` in native output rather than the default `object`, because mypyc cannot build -a dataclass with an `object` field. Top-level functions carry **annotations from their inferred -types** where Python can spell them (`int`, `float`, `bool`, `str`, `None`, `list`/`set`/`dict`/`tuple` -of those, a record's class, and a union alias per user sum type, `Expr = Num | Add | Mul | Neg`, -emitted after its classes); a type variable, a function type or an async result stays unannotated, -so the annotations never claim more than the checker proved. The output is written for mypy's newer -redefinition rule (`--allow-redefinition-new --local-partial-types`): a Python local reused across -match arms can hold differently narrowed types in different arms, which the default -first-assignment rule rejects. `pyfun build --native -o ` does the whole trip: it -compiles natively, runs mypyc with those flags, leaves the extension and a `__main__.py` in `` -(so the program runs as `python `), and needs `mypy` plus a C compiler in the environment. A -project builds the same way: every module is annotated from its own types and compiled by mypyc, -and the shared `_pyfun_rt.py` stays Python, imported by the compiled modules like any other. With both, `bench/mypyc_run.py` builds every benchmark that way: 1.3x to 13x faster than the same file on CPython, and on the -arithmetic-heavy one about 12x faster than hand-written Python. Native mode is opt-in and applies to every module of a project alike; the default emitter keeps -`match`/`case` because it reads as the program was written. Every example and the whole end-to-end suite produce identical -output in both modes. + +**Annotations.** Top-level functions carry annotations taken from their inferred types where Python +can spell them: `int`, `float`, `bool`, `str`, `None`, `list`/`set`/`dict`/`tuple` of those, a +record's class, and a union alias per user sum type (`Expr = Num | Add | Mul | Neg`, emitted after +its classes). A type variable, a function type or an async result stays unannotated, so the +annotations never claim more than the checker proved. Data class fields follow suit: a field +declared as a user sum type or record names it (`_0: "Expr"`, a string because the alias comes +later), and any other field without a Python class of its own is `typing.Any`, since mypyc cannot +build a dataclass with an `object` field. The output is written for mypy's newer redefinition rule +(`--allow-redefinition-new --local-partial-types`): a Python local reused across match arms can hold +differently narrowed types in different arms, which the default first-assignment rule rejects. + +**Building.** `pyfun build --native -o ` does the whole trip: it compiles natively, runs +mypyc with those flags, and leaves the extension and a `__main__.py` in ``, so the program runs +as `python `. It needs `mypy` and a C compiler in the environment. A project builds the same +way: every module is annotated from its own types and compiled by mypyc, and the shared +`_pyfun_rt.py` stays Python, imported by the compiled modules like any other. `bench/mypyc_run.py` +builds every benchmark this way and finds the mypyc build 1.3x to 13x faster than the same program +as plain Python; the int-arithmetic benchmark runs about 12x faster than hand-written Python. + +Native mode is opt-in and applies to every module of a project alike. The default emitter keeps +`match`/`case`, because it reads as the program was written. Every example and the whole +end-to-end suite produce identical output in both modes. ## 6. Python interop — the hard boundary diff --git a/ROADMAP.md b/ROADMAP.md index ccaa35a..6570e87 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -501,23 +501,20 @@ map_build 1.64× vs hand-written. with zero user toolchain. Weak spot: recursion (collatz 7.36× vs PyPy's own iterative baseline — absolute time still beats CPython). Docs line earned: "compute-bound? `--target 3.11` + PyPy." **CPython's own JIT** (experimental since 3.13) accrues to every program for free. -- **Typed-emit + mypyc AOT (`--native`)** (M to measure, L to ship; **gated on the measurement**; - **first slice SHIPPED 2026-09-28**: `pyfun compile --native` lowers every match built from - constructors/records/tuples/literals/`as` to an `if`/`isinstance` ladder, `DESIGN.md` §5.6, - expr_eval 2.30x → 1.63x on plain CPython; top-level or-patterns and list patterns followed the - same night, and so did nested or-patterns that bind no names. **First end-to-end mypyc - measurement (2026-09-28, `native-bench.yml`, ubuntu, CPython 3.12, untyped native output, data - class fields `Any` since mypyc rejects `object`):** all three benchmarks compile, run, and match - the baseline; mypyc is 1.24x / 1.28x / 1.11x faster than the same native file on CPython - (expr_eval / collatz / map_build), landing at 1.75x / **0.79x** / 1.58x of hand-written Python. - **Typed emit followed the same night** (top-level functions annotated from inferred types, a - union alias per sum type; mypyc run with `--allow-redefinition-new --local-partial-types`), and - the same job measured mypyc at 1.56x / **13.2x** / 1.32x faster than the native file on CPython, - landing at **1.40x / 0.08x / 1.34x of hand-written Python**: collatz, all int arithmetic, runs - about 12x faster than the hand-written baseline. `pyfun build --native -o ` (compile, mypyc, runnable - directory) followed too. Typed emit and `build` for projects followed too. Remaining, in - order: typed emit for block-local functions and for block-local functions, or-patterns that bind - names, then the closure/`seq`/`_pyfun_rt` audit (project mode followed the same night) — +- **Typed-emit + mypyc AOT (`--native`)** (L; **mostly SHIPPED 2026-09-28**, `DESIGN.md` §5.6). + What landed, in one night: `pyfun compile --native` lowers every match (constructors, records, + tuples, literals, `as`, list patterns, or-patterns, and the `return!` of a match in `async { }`) + to an `if`/`isinstance` ladder; top-level functions are annotated from their inferred types, with a + union alias per sum type and typed data class fields; `pyfun build --native -o ` runs + mypyc (`--allow-redefinition-new --local-partial-types`) and leaves a directory that runs as + `python `; all of it works for projects. `native-bench.yml` measures it on Linux, CPython + 3.12: mypyc runs the benchmarks 1.6x / **13x** / 1.3x faster than the same program as plain Python + (expr_eval / collatz / map_build), at **1.37x / 0.08x / 1.32x of hand-written Python**, so the + int-arithmetic benchmark beats the hand-written baseline about 12x. On plain CPython the ladders + alone take expr_eval from 2.30x to 1.63x. **Remaining:** annotations for block-local functions, + or-patterns whose alternatives bind names (the one match shape still left to `match`), trying a + mypyc build of a real program (the Scrabble game), and the closure/`seq` audit under mypyc. + The original reasoning, kept for the record: the checker knows every binding's inferred type, so the emitter could produce fully annotated Python whose annotations cannot lie, then compile it with mypyc into a C extension — native speed with the interop story intact (the result is still an ordinary extension module). Real blockers