docs: INTERNALS for tonight's native lowering, spawn handler and packages

INTERNALS.md

@@ -26,18 +26,22 @@ src/
   python_emitter/  Python-AST IR → readable source (py311.rs = the --target 3.11 rewrite)
   diagnostics/     rustc-style errors: codes (E001…), levels, spans, notes
   project/         module graph: acyclic import resolution, cross-module checking
+    manifest.rs      pyfun.toml reader/editor, project root and vendored façades (DESIGN §6.2)
   lsp/             front-end-first language server (stdio JSON-RPC)
     json.rs        hand-rolled, dependency-free JSON value + parser + serializer
     resolve.rs     AST resolver: symbol_at / find_references / definitions
   repl.rs          read-eval-print loop over one long-lived Python worker
   kernel.rs        the REPL session inverted, for the Jupyter kernel
-  main.rs          the CLI: check/compile/run/parse/lsp/repl/kernel-engine
+  packages.rs      `pyfun add` / `install` / `remove`: façade packages (DESIGN §6.2)
+  native_build.rs  `pyfun build --native`: compile, then mypyc (DESIGN §5.6)
+  main.rs          the CLI: check/compile/run/build/add/install/remove/parse/lsp/repl/kernel-engine
 editors/vscode/    minimal VS Code client that launches `pyfun lsp`
 tests/             parser tests, compile tests, .pyfun fixtures (favor snapshot/golden tests)
 ```
 
-The shared Python runtime support (the `Option`/`Result` classes) is not a directory: it is
-generated by `lowering::runtime_module()` into a `_pyfun_rt.py` beside the emitted output.
+The shared Python runtime support (the `Option`/`Result` classes, and the `_pf_scope` context
+variable `Task.scope` publishes its TaskGroup through) is not a directory: it is generated by
+`lowering::runtime_module()` into a `_pyfun_rt.py` beside the emitted output.
 There are no dependencies — `Cargo.toml`'s `[dependencies]` is empty, so the CLI is hand-rolled
 argument parsing rather than clap.
 
@@ -390,6 +394,30 @@ old signatures by delegating and dropping the notes. Regression tests live in `t
 "self tail calls become loops", including two end-to-end: 50k/100k-deep recursion that only completes
 as a loop, and the closure case whose per-frame values the capture check exists to preserve.
 
+### Native lowering (`--native`) — implements DESIGN §5.6
+
+A `Lowerer.native` flag, set by `lowering::lower_native_typed` (single file, called from
+`pyfun::compile_with`) or `lower_in_project` (from `project::compile_with`). Three pieces:
+
+- **Match ladders.** `try_lower_native_match` runs before the `Option`/`Result` ladder at both
+  match call sites (return and value position). `native_pattern_ok` decides whether every arm can be
+  compiled to plain tests; `native_pattern` compiles one pattern against a subject expression into
+  `conds` (and-ed) and `binds` (assignments run once the arm is chosen). A top-level or-pattern is
+  expanded into one piece per alternative (the body lowered per alternative); a nested one is joined
+  with `or` when no alternative binds a name. The assembly (guards falling through in return
+  position, the last unguarded arm as a plain `else`) mirrors `try_lower_ladder_match`.
+- **Annotations.** `lower_module` post-processes the module: `any_for_object_fields` turns `object`
+  field annotations into `typing.Any` (mypyc cannot build an `object` field), and
+  `Lowerer::annotate_functions` adds a union alias per non-generic user sum type after its classes,
+  spells data class fields that name a user type (`"Expr"`), and rewrites each top-level
+  `PyStmt::FuncDef` whose binding type is known into a `PyStmt::TypedFuncDef` using
+  `py_type_annotation` over `Lowerer.binding_types` (span of the binding target → `types::Ty`, from
+  `TypeSpan::raw`). An unspellable type leaves its position unannotated.
+- **Build.** `src/native_build.rs` writes the native output to a directory, runs `python -m mypyc
+  --allow-redefinition-new --local-partial-types` over every module but `_pyfun_rt.py`, removes the
+  sources and writes `__main__.py`. `bench/mypyc_run.py` and `.github/workflows/native-bench.yml`
+  measure it.
+
 ### The `--target 3.11` emission pass
 
 Emitted code targets Python 3.12+ for exactly one construct: an f-string hole may carry a nested
@@ -555,6 +583,18 @@ becomes `import math as _pf_math` and the reference is re-rooted, leaving the us
 dotted import binds only its first segment (`import os.path` binds `os`), so the alias replaces as
 many segments as the module spec covers.
 
+## The `spawn` handler (`Task.scope`) — implements DESIGN §4, §8
+
+`EffLabel::Spawn` is performed by `Task.start` (its scheme, `seed_task_prelude`). The discharge
+is a rule in `Infer::infer_apply`, not in unification: for a saturated application whose head is
+the prelude's `Task.scope` (`qualified_name`, and no user binding named `Task`), the argument is
+inferred in a fresh effect context and `spawn` is removed from what it performed before the rest
+joins the enclosing effect (`infer_apply_plain` is the ordinary path). `types::run` collects each
+top-level evaluation's performed effect (value bindings, expressions, module value members) in
+`top_effects` and reports any that still contain `spawn` once inference is done. Lowering routes
+`Task.scope`/`Task.start` to `_pf_task_scope`/`_pf_task_start` (`async_prelude`) and
+`note_task_scope_var` defines or imports `_pf_scope`.
+
 ## Solving effect equations (`Infer::unify_eff`) — implements DESIGN §4
 
 An `Effect` is a set of concrete labels plus a set of effect variables, so unification is over an
@@ -626,6 +666,16 @@ relying on a theme's rendering of `invalid` keeps the hue consistent across them
 auto-switching, and avoids the semantic lie that these valid keywords are errors. Users can override the
 colour in their own `editor.tokenColorCustomizations`.
 
+## Façade packages — implements DESIGN §6.2
+
+`project::locate_module(dir, name)` is the one resolution rule, used by `build_from_path` (the CLI
+graph) and `resolve_exports` (the LSP): the sibling `name.pyfun`, else the same file in each
+directory under `<root>/.pyfun/facades/` for the nearest `pyfun.toml` (`manifest::find_root`,
+`manifest::facade_dirs`). `src/packages.rs` implements the commands: the installer is `uv pip`
+when `uv` is on PATH, else `python -m pip`; the façade's `.pyfun` files are found by asking the
+installed distribution for its file list (`importlib.metadata`), and `manifest::set`/`remove` edit
+`pyfun.toml` line by line.
+
 ## File-based modules — implements DESIGN §6.1
 
 The module system's semantics (one module per file, qualified use, an acyclic import graph, cross-module ADTs/records/externs/measures, all-public) are in `DESIGN.md` §6.1. The driver, cross-module checking, and project emit are built as follows.