Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 23 additions & 17 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -432,23 +432,29 @@ has no `match` statement left, and every recorded replay prints the same in both
The ladder is also faster on CPython by itself, since a class pattern goes through `__match_args__`
and positional binding at run time: on `bench/expr_eval` (ADT allocation and matching) the output
goes from 2.30x to 1.63x of the hand-written baseline on CPython 3.14, before any compilation.
A data class field whose type has no Python class of its own (a sum type, a type variable) is
annotated `typing.Any` in native output rather than the default `object`, because mypyc cannot build
a dataclass with an `object` field. Top-level functions carry **annotations from their inferred
types** where Python can spell them (`int`, `float`, `bool`, `str`, `None`, `list`/`set`/`dict`/`tuple`
of those, a record's class, and a union alias per user sum type, `Expr = Num | Add | Mul | Neg`,
emitted after its classes); a type variable, a function type or an async result stays unannotated,
so the annotations never claim more than the checker proved. The output is written for mypy's newer
redefinition rule (`--allow-redefinition-new --local-partial-types`): a Python local reused across
match arms can hold differently narrowed types in different arms, which the default
first-assignment rule rejects. `pyfun build --native <file> -o <dir>` does the whole trip: it
compiles natively, runs mypyc with those flags, leaves the extension and a `__main__.py` in `<dir>`
(so the program runs as `python <dir>`), and needs `mypy` plus a C compiler in the environment. A
project builds the same way: every module is annotated from its own types and compiled by mypyc,
and the shared `_pyfun_rt.py` stays Python, imported by the compiled modules like any other. With both, `bench/mypyc_run.py` builds every benchmark that way: 1.3x to 13x faster than the same file on CPython, and on the
arithmetic-heavy one about 12x faster than hand-written Python. Native mode is opt-in and applies to every module of a project alike; the default emitter keeps
`match`/`case` because it reads as the program was written. Every example and the whole end-to-end suite produce identical
output in both modes.

**Annotations.** Top-level functions carry annotations taken from their inferred types where Python
can spell them: `int`, `float`, `bool`, `str`, `None`, `list`/`set`/`dict`/`tuple` of those, a
record's class, and a union alias per user sum type (`Expr = Num | Add | Mul | Neg`, emitted after
its classes). A type variable, a function type or an async result stays unannotated, so the
annotations never claim more than the checker proved. Data class fields follow suit: a field
declared as a user sum type or record names it (`_0: "Expr"`, a string because the alias comes
later), and any other field without a Python class of its own is `typing.Any`, since mypyc cannot
build a dataclass with an `object` field. The output is written for mypy's newer redefinition rule
(`--allow-redefinition-new --local-partial-types`): a Python local reused across match arms can hold
differently narrowed types in different arms, which the default first-assignment rule rejects.

**Building.** `pyfun build --native <file> -o <dir>` does the whole trip: it compiles natively, runs
mypyc with those flags, and leaves the extension and a `__main__.py` in `<dir>`, so the program runs
as `python <dir>`. It needs `mypy` and a C compiler in the environment. A project builds the same
way: every module is annotated from its own types and compiled by mypyc, and the shared
`_pyfun_rt.py` stays Python, imported by the compiled modules like any other. `bench/mypyc_run.py`
builds every benchmark this way and finds the mypyc build 1.3x to 13x faster than the same program
as plain Python; the int-arithmetic benchmark runs about 12x faster than hand-written Python.

Native mode is opt-in and applies to every module of a project alike. The default emitter keeps
`match`/`case`, because it reads as the program was written. Every example and the whole
end-to-end suite produce identical output in both modes.

## 6. Python interop — the hard boundary

Expand Down
31 changes: 14 additions & 17 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -501,23 +501,20 @@ map_build 1.64× vs hand-written.
with zero user toolchain. Weak spot: recursion (collatz 7.36× vs PyPy's own iterative baseline —
absolute time still beats CPython). Docs line earned: "compute-bound? `--target 3.11` + PyPy."
**CPython's own JIT** (experimental since 3.13) accrues to every program for free.
- **Typed-emit + mypyc AOT (`--native`)** (M to measure, L to ship; **gated on the measurement**;
**first slice SHIPPED 2026-09-28**: `pyfun compile --native` lowers every match built from
constructors/records/tuples/literals/`as` to an `if`/`isinstance` ladder, `DESIGN.md` §5.6,
expr_eval 2.30x → 1.63x on plain CPython; top-level or-patterns and list patterns followed the
same night, and so did nested or-patterns that bind no names. **First end-to-end mypyc
measurement (2026-09-28, `native-bench.yml`, ubuntu, CPython 3.12, untyped native output, data
class fields `Any` since mypyc rejects `object`):** all three benchmarks compile, run, and match
the baseline; mypyc is 1.24x / 1.28x / 1.11x faster than the same native file on CPython
(expr_eval / collatz / map_build), landing at 1.75x / **0.79x** / 1.58x of hand-written Python.
**Typed emit followed the same night** (top-level functions annotated from inferred types, a
union alias per sum type; mypyc run with `--allow-redefinition-new --local-partial-types`), and
the same job measured mypyc at 1.56x / **13.2x** / 1.32x faster than the native file on CPython,
landing at **1.40x / 0.08x / 1.34x of hand-written Python**: collatz, all int arithmetic, runs
about 12x faster than the hand-written baseline. `pyfun build --native <file> -o <dir>` (compile, mypyc, runnable
directory) followed too. Typed emit and `build` for projects followed too. Remaining, in
order: typed emit for block-local functions and for block-local functions, or-patterns that bind
names, then the closure/`seq`/`_pyfun_rt` audit (project mode followed the same night) —
- **Typed-emit + mypyc AOT (`--native`)** (L; **mostly SHIPPED 2026-09-28**, `DESIGN.md` §5.6).
What landed, in one night: `pyfun compile --native` lowers every match (constructors, records,
tuples, literals, `as`, list patterns, or-patterns, and the `return!` of a match in `async { }`)
to an `if`/`isinstance` ladder; top-level functions are annotated from their inferred types, with a
union alias per sum type and typed data class fields; `pyfun build --native <file> -o <dir>` runs
mypyc (`--allow-redefinition-new --local-partial-types`) and leaves a directory that runs as
`python <dir>`; all of it works for projects. `native-bench.yml` measures it on Linux, CPython
3.12: mypyc runs the benchmarks 1.6x / **13x** / 1.3x faster than the same program as plain Python
(expr_eval / collatz / map_build), at **1.37x / 0.08x / 1.32x of hand-written Python**, so the
int-arithmetic benchmark beats the hand-written baseline about 12x. On plain CPython the ladders
alone take expr_eval from 2.30x to 1.63x. **Remaining:** annotations for block-local functions,
or-patterns whose alternatives bind names (the one match shape still left to `match`), trying a
mypyc build of a real program (the Scrabble game), and the closure/`seq` audit under mypyc.
The original reasoning, kept for the record:
the checker knows every binding's inferred type, so the emitter could produce fully annotated
Python whose annotations cannot lie, then compile it with mypyc into a C extension — native speed
with the interop story intact (the result is still an ordinary extension module). Real blockers
Expand Down
Loading