Skip to content

Latest commit

 

History

History
220 lines (188 loc) · 30.3 KB

File metadata and controls

220 lines (188 loc) · 30.3 KB

Jett Compiler Progress

Statistics

  • Lines of Rust: ~33,000
  • Tests: 470+ passing
  • Crates: 15
  • VS Code extension: Yes

Implementation Status

Phase A: Foundation — COMPLETE

Component Crate Tests Status
FileId, Span, Symbol interner jett_common 3 Done
Diagnostics, error rendering, TOON output jett_diagnostics 12 Done
Project discovery, jett.proj parsing jett_project 7 Done
Lexer (indentation, interpolation, 90+ tokens) jett_lexer 62 Done
Parser (recursive descent + Pratt, direct source-spanned AST) jett_parser 67 Done; lossless CST-to-AST frontend is planned later
Formatter (canonical whitespace) jett_fmt 8 Done
Pipeline orchestration jett_driver 36 Done
CLI (format, build, run, test, lsp) jett_cli 0 Done

Phase B: Type System Core — COMPLETE

Component Crate Tests Status
Type interning, struct/enum/interface defs jett_types 18 Done
Name resolution, scoping, namespace export visibility jett_resolve 27 Done
Type checking (expressions, operators, generics) jett_typecheck 117 Done; checked output now includes an ordered, deduplicated concrete generic-instantiation manifest with per-instantiation expression and nested-call facts
Module/import/prelude registry and backend-neutral trusted origin jett_project, jett_resolve, later IR n/a Design selected by the module and trusted-origin contract; current block-local use, stdlib loading, and interpreter trust paths remain transitional

Phase C: Ownership and Capabilities — COMPLETE

Component Crate Tests Status
Ownership analysis (move/view/consume) jett_typecheck/ownership 9 Done
Capability purity enforcement jett_typecheck/capability 7 Done
Secret taint analysis Partial (refined-secret taint fixed by #18; content-constant interpreter comparison for string/bytes secrets defined by #33, with native lowering pending #20 and #22)

Phase D: Code Generation — STARTED

Component Crate Tests Status
HIR (monomorphization) jett_hir 28 Typed ordinary, generic, method, and actor receive-handler lowering implemented for the core subset, with canonical identity, deterministic IDs, separate per-instantiation facts, checked actor capability/state/message locals, named-argument normalization including actor spawn/messages, concrete source-method targets, struct/list/map construction, field access, pipeline normalization, generic calls, and core structured control flow; actor runtime entities and remaining constructs are staged by the HIR lowering plan and #20
MIR (control flow graph) jett_mir 13 Structured-control-flow lowering with source-spanned statements and terminators; structural entry/edge/ID validation and deterministic successor, predecessor, and reverse-postorder analysis are implemented. Definitive ownership/drop elaboration remains (tracked by #22)
LLVM native codegen jett_codegen_llvm Not started
Runtime library jett_runtime 5 Initial generation-checked opaque-resource registry implemented with isolated contexts, authority provenance validation, pending-operation cancellation, stale-callback defense, exactly-once finalization, and reverse-creation teardown; interpreter trusted-hook and control-flow integration remain staged
Core stdlib (.jett files) stdlib/ Partial (bootstrap loader plus extracted json, complete source-owned list, map, set, string, math, random, and time public APIs, and other modules that remain Rust-backed)

Phase E: Comptime and Verification — COMPLETE

Component Crate Tests Status
Comptime interpreter jett_comptime 137 Done
Explicit comptime expression evaluation jett_comptime, jett_driver Done (closed pure expressions are evaluated during the build and their values are consumed by runtime interpretation)
Verify blocks (compile-time) jett_comptime/verify Done
Comptime type reflection (type.name, type.kind, type.kind_tag, type.primitive_tag, type.has_secret, type.info, type.arg, type.fields, type.bitfield_layout, type.bitfield_fields, type.machine_layout, type.machine_states, type.machine_transitions, type.machine_state_value, type.machine_field_value, type.variants, type.variant_value, type.field_value, type.variant_field_value, type.construct_start, type.construct_variant_start, type.construct_machine_start, type.construct_put, type.construct_finish, trusted comptime type binding for roots, type args, struct/enum/machine field loops, TypeInfo.args, and TypeInfo.primitive_tag) jett_comptime Partial (struct/bitfield/enum/machine construction via TypeConstruction; structured TypeKind and TypePrimitive tags available; checked ReflectionMetadata snapshot now feeds metadata-only reflection builtins, value-sensitive field/variant/machine access, construction validation/finish, checked bitfield layout, checked machine state/transition metadata, trusted comptime type bindings for args/field/variant/machine-state loops, branch specialization for direct, immutable-local, and immutable helper-parameter TypeInfo / TypeKind / TypePrimitive generic reflection checks, selected-branch and direct top-level type.arg[T]() plus direct type.fields[T]() / type.variants[T]() / type.machine_states[T]() / type.machine_state_value[T](...).fields loop checking under concrete generic instantiations, top-level variant/state selection and TypeConstruction start/finish checking, and the json.serialize secret-containing type gate; AST fallback paths remain for bootstrap/direct-interpreter compatibility while metadata identity moves toward canonical checked records)
25+ stdlib builtins in interpreter jett_comptime Done

Phase F: Interpreter — COMPLETE

Component Crate Tests Status
Tree-walking interpreter (jett run) jett_comptime Done (reuses comptime interpreter)

Phase G: Advanced Type Features — PARTIAL

Feature Status
State machines (machine, transitions, at) Done (checked state-qualified types with explicit bare-annotation erasure, canonical Machine.transition(...) calls without generated alternatives, exact-state-only and local-variable-only branch narrowing, no implicit union-state or path facts, namespaced machines, reflection metadata, reflected machine construction, and JSON parse/serialize through the explicit state/payload envelope)
Refinement types (type X = T where ..., coarsen) Done
User-defined structs (constructors, field access, methods) Done
Match statements with enum destructuring Done
Pipeline operator (into, single-line and multi-line indented form) Done (qualified/generic steps, view steps, and step-local handle error: / handle: are covered)
String interpolation Done
Verify blocks (parsing + execution) Done
Property-based testing (property blocks) Done
Bitfield declarations Partial (constructors, field access, enum-annotated fields, payload fields, 64-bit uint64 fields, reflection, and interpreter to_bytes/from_bytes roundtrips are covered; native codegen still pending)
Actor model (actor, spawn, send, ask) Done (capability args, message args, state initializers, and responds values typecheck against declared types; exported namespaced actors support qualified and use-alias spawn)
Structured concurrency (run, join, cancel) Done
Interface/implement blocks Done
mutual blocks Done
Secret types (secret[T], declassify) Partial (refined-secret taint fixed by #18; secret.compare now accepts only compatible string/bytes secrets and avoids content-dependent early exits in the interpreter per #33; native lowering remains pending)
Handle blocks (handle error:, handle:, default) Done
Generic structs Done
Generic functions Done
Comptime struct/enum/bitfield/machine introspection (TypeInfo, TypeField, TypeBitfield, TypeBitfieldField, TypeMachine, TypeMachineState, TypeMachineTransition, TypeVariant, alias/refinement base metadata, field serialize names, checked field/variant/machine value access) Partial
String escape sequences (\", \\, \n, \t, \r) Done
Inline function expressions (function(x: T) returns U: body) Done
Higher-order list functions (filter, map, find, sort_by, all, any, count, sum, group_by) Done
Single-line handle: blocks Done
math.average, math.median Done
string.reverse, string.after, string.before, string.trim_start, string.trim_end Done
string.slugify, string.truncate, string.between, string.pad_left, string.is_not_empty Done
string.chars, string.words, string.lines (iterator builtins → list[string]) Done
string.index_of, string.count, string.to_upper_first, string.to_lower_first Done (search/count results are grapheme-boundary aware)
list.reduce, list.flat_map, list.chunk, list.sort_by_index, list.is_sorted, list.all_elements_in Done (higher-order callback return types are checked for explicit typed calls)
list.enumerate Done
Complete source-owned list API with typed list.Pair[A, B] and list.Indexed[T] Done
Complete source-owned map API and typed map.Entry[K, V] Done
Complete source-owned set API Done
Complete source-owned encoding API: byte-native Base64/hex plus URL/form components Done
Byte-native hex helpers (bytes.to_hex, bytes.from_hex) Done
Complete source-owned bytes API with move-only/view semantics Done
Closure captures (inline functions capture immutable enclosing scope) Done
Function type expressions (function(T) returns U in type annotations) Done
Dotted use paths (use net.http) Done
Strict inline imports for project/vendored namespaces (E0210; same-namespace and declaration-signature type access remain direct; compiler stdlib remains implicit pending prelude policy) Done
Namespace-local global constant initializers (E0211; cross-namespace project/vendored dependencies are forbidden) Done
Conservative generic reflection facts (predicate calls and detached booleans never authorize casts or skip branch checking; post-check folding is optimization only) Done
Namespace-qualified user function calls (helpers.f(), helpers.f[T]()) Done
Namespace-private declarations with explicit export for public APIs, including mutual declarations and qualified-only external access Done
Multi-file compilation (project-aware build/run with jett.proj) Done
range() builtin (1, 2, or 3 args) Done
For-in over strings, maps (with key, value destructuring), sets Done
and/or keyword operators for logical expressions Done
Unhandled result/optional detection (E0341, E0342) Done
Set value type and 12 set builtins (new, add, remove, contains, union, intersection, difference) Done
print/println builtins Partial (interpreter support, secret blocking, and E0362 release diagnostics are done; debug-only capability policy is decided, while debug-event isolation and future-backend conformance remain pending)
Type conversions: exact fallible int64.from_float64 and float64.from_int64, float64.from_string, string.from_float64, string.from_bool Done
time.now_ms, time.now_s Removed (replaced by explicit Clock.now)
os.env, os.args Removed (replaced by explicit Environment.get and Environment.args, with focused migration diagnostics)
Math: pi, e, sin, cos, tan, mod, is_even, is_odd, sum Done

Phase H: Agent Tooling — PARTIAL

Tracked by #35 for the remaining structured diagnostic context and agent-mode failure envelope.

Component Status
TOON output (--agent flag) Partial (build diagnostics include file, ok/error status, severity counts, ranged diagnostics/labels, and file-aware ranged suggested fixes; format status, run stdout/typed debug output, ranged verify/property test summaries, namespace/symbol/type-at/definition-at/references-at/completion/signature query results, file-symbol/completion parse failures, global namespace/signature project-parse failures, type-at compiler failures, and definition-at/references-at parse/resolution failures with known source context and cross-file labels/fixes are structured)
LSP server (save diagnostics with related compiler labels, whole-document formatting, semantic tokens, token-aware selection ranges, and open-buffer workspace symbol search) Done
LSP hover (type at cursor) Done
LSP go-to-definition Done
LSP find references Done (current document, with optional declaration inclusion)
LSP completions Done (latest in-memory source with prefix filtering, deterministic rank, project/stdlib candidates, signature details, and leaf-name filtering for qualified symbols)
LSP document highlights Done (current-document declaration and use occurrences with UTF-16 ranges)
LSP rename Done (semantic declaration and reference edits in the current document)
LSP signature help Done (source-defined and standard-library calls from the latest in-memory document, with nested-call active-parameter tracking)
LSP document symbols Done (top-level file outline from the latest in-memory document, with declaration kinds, signatures, full declaration ranges, and name-selection ranges in UTF-16 coordinates)
LSP folding ranges Done (lexer-backed nested block ranges from the latest in-memory document)
MCP server Not started; initial transport, tool/resource, and ASP handoff boundary tracked by #37
ASP query system Partial (jett query --agent --namespaces, --symbols, --type-at, --definition-at, --references-at, prefix-filtered --complete-at, and --signature are implemented; namespace, type, symbol, definition, reference, and completion rows include source ranges; file-symbol and completion parse failures, type-at parse/resolution/type-check failures, and definition-at/references-at parse/resolution failures with known source context preserve structured diagnostics and cross-file labels; completion rows also include deterministic rank, match kind, namespace, visibility, and source-level signatures where available)

Phase I: Testing and Profiling — PARTIAL

Component Status
Property-based test runner Done (basic: 100 generated iterations; all numeric primitives, bool/string/bytes/nothing, aliases/refinements, structs including generic structs, bitfields, enums, plus generic list/set/map/optional/result pools)
Input shrinking on failure Done (shrinking for int64, float64, string, bytes, list, set, map, optional, result, struct fields, enum payloads)
CPU profiler (--profile) Backend-neutral launch configuration, exact CLI validation, sample aggregation, exact thresholding, deterministic ranking, bounded hot-line and call-chain detail, bounded 128-frame stack normalization, manifest/secret-aware bounded source sanitization, human/TOON summary rendering, constant-space per-worker pending-request/coalescing gate, and suggestion rules implemented in jett_profiler; unsupported runtimes refuse setup before execution, while runtime sampling remains staged by the profiling contract
Memory profiler (--profile-memory) Backend-neutral allocation lifecycle validation, resize/free accounting, peak and retained-byte attribution, exact thresholding, deterministic ranking, and suggestion rules implemented in jett_profiler; rendering, CLI integration, metadata limits, and runtime allocation hooks remain staged by the profiling contract
trace keyword Partial (parses, typechecks, runtime type-tagged current-value output in jett run)
breakpoint keyword Partial (parses, typechecks, and emits conditional runtime debug snapshots with visible binding types in jett run; the decided pause/inspection protocol now has a shared typed lifecycle, request-validation, authentication, source-manifest, and failure-rendering model in jett_common, while interpreter transport/operations and future native-runtime stages remain unimplemented)

Phase J: Cross-Platform and Interop — PARTIAL

Component Status
Cross-compilation (--target) Not started
C binding generator (jett bind) Not started (initial syntax, safety boundary, supported subset, and staged implementation specified by the C FFI binding contract from #53; the foreign declaration frontend prerequisite is tracked by #173, while generator/CLI work remains pending)
jett bundle Working (resolver-derived whole-file dependency ordering, stable lexical tie-breaking, structured cycle and namespace-boundary diagnostics, line manifests, and validation-before-write)

Phase K: Full Standard Library — NOT STARTED

Module Status
string Done (all public declarations and compositional behavior are source-owned in stdlib/string.jett; only private trusted conversion, Unicode, grapheme, search, and text primitive kernels remain, and project code cannot call them; count/index/search/extraction helpers avoid partial grapheme matches)
list Done (all public declarations and compositional behavior are source-owned; observers use views, transformations consume inputs, zip/enumerate return typed records, global range is canonical, and only private trusted allocation/indexing/mutation/sorting/sum/callback kernels remain)
set Done (all public declarations and set algebra are source-owned; only private trusted storage/equality/cardinality kernels remain)
map Done (all public declarations, conversions, and higher-order operations are source-owned with typed map.Entry[K, V]; only private trusted storage/equality/lookup-update kernels remain)
math Done (all public declarations are source-owned in stdlib/math.jett; compositional helpers have Jett bodies, integer operations share the language's wrapping semantics and nonzero-divisor proof rule, while private trusted kernels preserve floating-point primitives, constants, exact numeric collection behavior, and remaining domain failures; project code cannot call the kernels, and the closed abs/min/max int64/float64 call policy remains compiler-enforced without creating general overloading)
json Partial (json.serialize, json.serialize_public, json.parse_exact, json.parse_raw and raw-tree accessors, compiler-owned public policy for parse/serialization; interpreter entrypoints require trusted stdlib-loaded reflected .jett hooks under namespace json; typed parsing routes through the stdlib json.JsonTree parser/decoder and exact parsing rejects unknown object fields recursively; reflected construction covers nested structs, enum-annotated bitfields, enums, machines, collections, wrappers, bytes, sized numeric primitives, null, secrets, aliases/refinements, and missing optional-field defaults; json.JsonTree is the sole raw representation and both former JsonValue aliases are rejected; checked reflection metadata feeds direct reflection, construction, runtime execution, and secret serialization policy)
random Done for the interpreter-backed compiler (all 5 public declarations are source-owned in stdlib/random.jett; every call borrows an explicit Random, integer sampling is half-open and unbiased, choice/shuffle preserve borrowed inputs, production state is runtime-injected, typed scripted samples make tests deterministic, and only private trusted sampling kernels remain; concurrent cancellation/clone sharing and later backends retain handoff obligations in the random contract)
crypto Done for the implemented digest and HMAC-SHA-256 surface (sha256, sha512, legacy-only md5, and key-first binary hmac_sha256 are source-owned in stdlib/crypto.jett; wrappers preserve exact bytes, HMAC accepts and returns secrets, only private digest kernels remain, and HMAC-SHA-512 stays reserved per the crypto contract)
encoding Done for the interpreter-backed compiler (all 8 public declarations are source-owned in stdlib/encoding.jett; Base64/hex operate on arbitrary bytes, all decoders return stable handled errors, URL and form component semantics are distinct, project code cannot call private kernels, and the future-backend handoff is recorded in the encoding contract)
bytes Done (all 9 public declarations are source-owned in stdlib/bytes.jett; observers use read-only views, slice returns independent owned bytes, concat consumes both inputs, and only private trusted raw-byte and UTF-8/hex kernels remain)
uuid Partial (uuid.new; generation and entropy contract tracked by #73)
time Done for the interpreter-backed compiler (Clock.now(view clock) reads an injected wall clock; time.Timestamp and time.Duration plus conversions, comparison, difference, and checked arithmetic are source-owned in stdlib/time.jett; deterministic raw clock samples cover pre-epoch flooring, backward movement, provider failure, exhaustion, and range checks; ambient clock builtins are removed with migration diagnostics; later backends retain the time/Clock contract)
Environment / os Done for the interpreter-backed compiler (Environment.get(view env, key) and Environment.args(view env) are source-owned over an immutable launch snapshot; production and deterministic test contexts inject launch data; missing, invalid-name, invalid-value, argument ordering/empties, independent-list, and capability restrictions are covered; private kernels cannot be called by project code; ambient os.env/os.args are removed with migration diagnostics; later backends retain the Environment and argument contract, implemented by #170)
graphics Initial interpreter-backed 2D session implemented: source-owned Config, Color, Rect, Text, Scene, Key, and run[State]; runtime-injected Graphics authority; checked pure callbacks; native minifb window, clipped rectangles and bitmap text; scripted input and scene capture for tests; host-local window cleanup on return/error. No source-owned window resources, animation clock, audio, textures, or native code generation. See the graphics contract.
net.http Not started (initial outbound client and Network capability contract tracked by #101)
net.socket Partial (TCP-first transport contract completed by #104; compiler-shipped resource Name declarations now have lexer/parser/AST, trusted-stdlib resolver, nominal type, reflection, formatter, query, and LSP support, including project-origin and clone rejection; jett_runtime now provides the generation-checked registry, authority validation, cancellation detachment, stale-callback defense, and reverse-order teardown substrate; trusted-hook dispatch, source control-flow cleanup, socket providers, and backend handoff remain pending under the opaque runtime resource contract from #175; see docs/open_design/net_socket_transport_contract.md)
csv Done for the interpreter-backed compiler (all 3 public declarations are source-owned in stdlib/csv.jett, with only private trusted parse/stringify kernels in the interpreter; parsing returns result[..., string], ignores one leading UTF-8 BOM, treats empty input as zero records, preserves blank records, whitespace, quoted data, Unicode, LF/CRLF endings, and ragged raw rows; malformed quoting, bare CR record endings, invalid headers, and header/data width mismatches fail explicitly; stringify emits canonical LF-separated records with no final newline; future backends retain the CSV format and failure contract)
regex Initial backend-neutral pattern compiler implemented in jett_regex (portable grammar validation, stable grapheme-indexed syntax errors, flags/capture metadata, and checked canonical NFA state preflight); matching, Unicode manifest integration, source declarations, and private trusted runtime/comptime kernels remain staged by the regular expression contract from #140
log Partial (the dedicated Log capability, source-owned event/level/field/error types and level wrappers, private trusted emission boundary, capability propagation, and secret-output classification are implemented; runtime filtering, checked sequence allocation, complete FileKey source identity, deterministic JSON records, injected sinks/captures, and RunOutput/TOON composition remain staged by the structured logging contract from #143)
test.mock Partial (the source-owned RandomStep, ClockStep, and EnvironmentEntry scripts plus the random, clock, and environment constructors are discoverable and typechecked, and constructor calls are restricted to direct property bodies; host-side Random and Clock adapters enforce exact script consumption after successful runs; per-attempt provider execution, mismatch schemas, isolation, replay/shrinking, and future-backend obligations remain staged by the capability mocking and deterministic test harness contract from #145)

Phase L: Incremental Compilation — NOT STARTED

Component Status
Salsa integration Initial whole-file parse-query slice implemented (the first jett_query boundary memoizes parser-owned direct ASTs by stable logical file identity; see the initial query and invalidation boundary from #147, with implementation tracked by #166)
Parallel compilation Initial bounded whole-file parse coordinator implemented with Salsa database snapshots, manifest-order publication, and stable invalid-worker/spawn/panic errors; cancellation, client worker controls, and namespace/body scheduling remain pending under the deterministic parallel compilation boundary, tracked by #151
Content-addressed caching Initial canonical parse-key and authenticated-envelope codecs implemented (SHA-256 identity, HMAC-SHA-256 authentication, exact v1 binary records, strict bounded decoding, constant-time tag verification, and current-source validation); compiler compatibility identity, authentication-key storage, parse artifact serialization, persistent read-through, atomic publication, and bounded cleanup remain pending under the content-addressed compilation cache contract from #153

VS Code Extension

Feature Status
Syntax highlighting (TextMate grammar) Done
Language configuration (brackets, indentation) Done
LSP integration (diagnostics, formatting, semantic highlighting, and quick fixes) Done (via jett lsp; tab-indentation diagnostics offer an apply-ready four-space fix)

CLI Commands

Command Status
jett format [--agent] [--check] file.jett Working
jett build [--agent] [--release] [--target T] file.jett Working (validates, no binary output)
jett run [--agent] file.jett Working (tree-walking interpreter; --agent captures stdout plus typed trace/breakpoint debug rows)
jett test [--agent] [file.jett] Working (verify + property blocks; --agent emits compact block tables)
jett lsp Working (diagnostics on save)
jett bind header.h Not started (contract specified in the C FFI binding contract from #53; the foreign declaration frontend prerequisite is tracked by #173, while generator/CLI work remains pending)
jett bundle Working (resolver-derived whole-file dependency ordering, stable lexical tie-breaking, structured cycle and namespace-boundary diagnostics, line manifests, and validation-before-write)
jett mcp Not started
jett query --agent --namespaces / --symbols file.jett / --type-at file:line:column / --definition-at file:line:column / --references-at file:line:column / --complete-at file:line:column / --signature name Partial (ranged namespace registry, file-local symbols with declaration ranges and function signatures, ranged type lookup with structured compiler failures and cross-file labels when source context is known, ranged definition lookup, ranged reference lookup, ranked prefix-filtered completion candidates with context metadata and ranges, and source-level function signatures)