diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 8efd3da..ef11030 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,25 @@ +## E-THE-CI-GAP-WAS-THE-TRIGGER-NOT-THE-COVERAGE-1 (2026-09-19) + +⊘ **Corrects `E-THE-VALHALLA-FLIP-COST-ONE-WORD-SIX-TIMES-…-1` below**, which +says the `RangeOutOfBounds` break went unseen *"because no CI job here compiles +the native crate."* **That is false, and checkable in one file.** +`.github/workflows/lint.yml` runs `clippy` and `rust-test` jobs that check out +sibling `AdaWorldAPI/lance-graph` (and `ndarray`, and `OGAR`) and then run +`cargo clippy --all-targets -- -D warnings` and `cargo test` from +`native/lgj-abi`. The native crate is compiled against the real sibling on +every push and PR **to this repository**. + +**The actual hazard is the TRIGGER.** The workflow is `on: {pull_request, +push}`, which fires for events in *this* repo only. An upstream-only merge in +lance-graph — exactly what added `ExecError::RangeOutOfBounds` — cannot start +it, so the break stayed invisible until the next push here. Coverage was never +missing; the *event* was. + +**Consequence for the next session:** do not add a CI job for this — one +exists. The open gap is upstream-change notification, and nothing in this +repository's triggers can close it. Raised by Codex P2 on #82, verified +against the workflow. + ## E-PANAMA-TIMES-VALHALLA-IS-ONE-MEMBRANE-AND-THE-LAB-FINALLY-HAS-ONE-VARIABLE-1 (2026-09-19) **Operator reframing, and it is the correct one:** this was never a JDK 26→28 @@ -131,14 +153,13 @@ tripwire, not a safety net:** it converts an upstream additive change into a downstream build failure, and nothing in either repo's CI noticed, because no CI job here compiles the native crate. -**Two false failures I nearly reported, both mine, both caught by reading.** -The flipped suite showed `1 FAILED` twice. Both times it was -`DoctrineFenceTest` refusing to run — it requires `java/src/main/java` AND -`native/lgj-abi/src` relative to CWD, and I had run the flipped build from a -scratch directory that had neither. The fence is *correct*: its first check is -*"a fence that cannot find its corpus must fail, not skip."* Running the flip -in the real tree (committed first, restored with `git checkout`) gave the true -result. **A harness artifact and a regression look identical in a summary +**Harness note, so the next run is not misread.** The flipped suite reports +`1 FAILED` when run from a directory lacking `java/src/main/java` AND +`native/lgj-abi/src` — `DoctrineFenceTest` resolves both relative to CWD and +refuses to run without them. The fence is behaving correctly: its first check +is *"a fence that cannot find its corpus must fail, not skip."* **Run the flip +in the real tree** (commit first; restore with `git checkout`) or the result is +not the suite's. **A harness artifact and a regression look identical in a summary line** — this is the ruff trap "a disable that does not apply is indistinguishable from a guard that is not load-bearing", met from the other direction. diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index d8f5e74..3c59d8f 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -1,3 +1,25 @@ +## 2026-09-19 — PR #81 merged (`07aa441`): production IS the Valhalla arm; Panama × Valhalla is one membrane + +**The frame, because it is easy to file this wrong:** this was not a JDK +26→28 upgrade. Production WAS Panama-only with Valhalla in a lab arm; they +are now **one production membrane**, with JDK 28 as the enabling toolchain. +Panama = the verb, Valhalla = the noun, **lance-graph / T0 owns storage**. + +- **Toolchain:** `/opt/jdks/jdk-28+16` (Temurin `28+16-ea`). FFM final and + flag-free; JEP 401 preview-gated, so `--release 28 --enable-preview` is part + of the shipped build contract — **the flag is Valhalla's, never FFM's.** +- **Production vocabulary:** the six types are `public value record`, + `isValue() == true`. Gates: **183 native tests**, **409 Java checks**, + identical to the unflipped baseline on the same JDK. +- **Carry this forward:** the **8-byte flattening cliff reproduces on JDK 28**, + and `RowRange` / `Row` (16 B) are on the wrong side of it. Two allocation + measurements got WORSE under value semantics (non-escaping construction, + `Row` hydration) — both honest, neither on a production path. The FFM path is + byte-identical across the object model. +- **Still owed:** `bench/` JMH numbers have NOT been re-taken on JDK 28 + (`bench/lib` absent here); `bench/RESULTS.md` is JDK 26 historical. +- Full record: `PR_ARC_INVENTORY.md` PR #81. + ## 2026-09-16 — `hop_cached_vs_gather`: the M1b tile pays off on the second hop; the scatter walk is the access-shape question **PR #79**, `native/lgj-abi/examples/hop_cached_vs_gather.rs` only — no ABI diff --git a/.claude/board/PR_ARC_INVENTORY.md b/.claude/board/PR_ARC_INVENTORY.md index 7158ff0..668fcb3 100644 --- a/.claude/board/PR_ARC_INVENTORY.md +++ b/.claude/board/PR_ARC_INVENTORY.md @@ -8,6 +8,87 @@ > anti-pattern the imported board rules name. Backfilled below in one > pass rather than left stale; PR #4 onward gets its entry at merge time. +## PR #81 — Panama × Valhalla as ONE production membrane (merged `07aa441`, branch `claude/great-pascal-k96kok`, head `1c66091`, 3 commits) + +**Entry written at merge**, not backfilled. + +- **Frame, and it is the point:** NOT a JDK 26→28 migration. The two halves + that were split — Panama FFM in production, Valhalla in the lab — became one + production membrane. JDK 28 is the enabling toolchain, nothing more. Panama + carries the VERB (reach into canonical bytes); Valhalla carries the NOUN + (identity-free semantic addresses over them); **lance-graph / T0 still owns + the only canonical copy.** +- **Added:** `CLAUDE.md` § P0 (the operator mandate); the six vocabulary types + as `public value record` (`LaneId`, `MaskId`, `Ordinal`, `FacetId`, + `RowRange`, `WideFieldMask`); `plan_lower::range_falsifier:: + no_opcode_lowers_to_pred_range`; JDK 28 + preview across `bench/run.sh`, + `bench/gate-run.sh`, `java/README.md`, `README.md`, `docs/panama.md`, + `panama-bridge-engineer.md`, `BOOT.md`; `jdk-toolchain-facts.md` § "JDK 28 — + INSTALLED AND MEASURED" with the obtain route. +- **Fixed (this blocked everything):** `lgj-abi` did not compile against + lance-graph `main`. `mask_risc::ExecError` gained `RangeOutOfBounds` with the + `Pred::Range` work and `exec_error_to_status` matches exhaustively. Mapped + into the documented *"bug in THIS file"* family — no new public status, no + ABI bump — and fenced by the falsifier above, **disable-verified + red-then-green**. *Standing hazard, corrected before merge (Codex P2 on #82, + verified against `.github/workflows/lint.yml`): an exhaustive match across a + repo boundary turns an upstream ADDITIVE change into a downstream build + failure. ⊘ The first draft blamed the wrong thing — it said "no CI job here + compiles the native crate", which is FALSE: the `clippy` and `rust-test` + jobs check out sibling `AdaWorldAPI/lance-graph` and run + `cargo clippy --all-targets` / `cargo test` from `native/lgj-abi`, so CI + compiles it on every push and PR **to this repo**. The real gap is the + TRIGGER, not the coverage: `on: {pull_request, push}` fires on this + repository only, so an upstream-only merge in lance-graph cannot start this + workflow and the break stays invisible until someone pushes here. Naming it + as missing coverage would have sent the next session to add a job that + already exists.* +- **Measured** (Temurin `28+16-ea`, fresh `liblgj_abi.so` abi 0.11 avx512): + **183 native tests**; **`ALL PASSED (409 checks)`** — byte-for-byte the same + 409 as the unflipped baseline **on the same JDK**, which is the comparison + that matters. All six types `isValue() == true`; substitutability holds. + **The 8-byte flattening cliff REPRODUCES on JDK 28:** `LaneId`/`Ordinal` + (4 B) and `MaskId` (8 B) FLAT; **`RowRange` (2 long, 16 B) and `Row` (16 B) + NOT-FLAT.** Allocation, record → value, 1M ops: construct-into-array + 15.26 → **1.87 MiB** (8.2×); allocate+fill 19.07 → 5.65 MiB; `Descriptor` + 45.78 → 27.89 MiB; ⊘ **never-escaping 4.73 → 5.75 MiB (WORSE** — EA already + erased the record case**)**; ⊘ **hydrate 65,536 `Row` 1.50 → 2.00 MiB + (WORSE** — `Row` does not flatten**)**. **FFM path byte-identical across + arms:** 712 B/query, 0 Java objects per row, same native bytes. +- **Locked:** `value semantics ≠ flattening` — `RowRange` is a production type + on the wrong side of the cliff and no performance claim may be read off its + `isValue()`. Valhalla changes what the VOCABULARY costs; it does not touch + the membrane. No second graph representation, and **no PRODUCTION row + hydration** introduced — scoped deliberately, because this entry reports a + `hydrate 65,536 Row` number two bullets up and an unqualified claim would + contradict its own measurement. That number is a LAB probe of what hydration + would cost; production takes no such path. +- **The harness is now a REPRESENTATION probe suite, not a "Valhalla lab" — + Valhalla graduated to production infrastructure and is no longer the thing + under test.** `valhalla-lab/` keeps its directory name for path + compatibility only; a doc banner says so. What it measures is object-model + and layout consequence: `record` vs `value record` **on one JDK, against the + same Valhalla-enabled production API**, with flattening and EA toggled. + **The re-scope also removed a confound the harness always carried:** the old + A/B was `(record, JDK 26)` vs `(value record, JDK 27 EA)` — two variables. + Both arms now run JDK 28, so the object model is the only difference. Two things it forced: **preview marking is transitive** (the + `record` arm also runs `--enable-preview`; it is a record arm on a preview + JVM, not a preview-free one), and **`ValueClass.isFlatArray` narrowed from + `Object` to `Object[]`** since 27-jep401ea3, which broke the flattening probe + until adapted. +- **Struck:** the *"production targets a shipped GA JDK, no preview flags — a + real, deliberate strength"* decision, in place in `jdk-toolchain-facts.md`. + It dressed a release constraint as an architectural virtue and had already + licensed the false claim *"production lgj does not depend on Valhalla at + all."* +- **Deferred, stated rather than implied:** `bench/` cannot run here — + `bench/lib` does not exist, so the JMH jars are absent. `bench/RESULTS.md` + numbers remain **JDK 26 historicals, correct when taken**, deliberately + untouched and **NOT re-taken on JDK 28**. `gate-run.sh` carries the new pin + and flags, so the re-run is one command once the jars are present. +- **Confidence:** high on everything measured in-container; the JMH + crossing/allocation figures are unverified on JDK 28 and are labelled so. + ## PR #79 — `hop_cached_vs_gather`: the M1b tile pays off on hop two; the scatter walk is the access-shape question (opened 2026-09-16, merged `9cb63e9`, head `c10029b`) > **Post-merge backfill, owned (2026-09-16):** this entry was NOT written at diff --git a/README.md b/README.md index e169570..558b6ed 100644 --- a/README.md +++ b/README.md @@ -28,9 +28,11 @@ Rust, SoA layout, SIMD, FFM, or graph-engine internals. | View / Mask / Lens DSL <- lazy, monotonic-narrowing | - stable-JDK implementation - | - Valhalla laboratory <- the tiny vocabulary only + JDK 28: Panama FFM + JEP 401 <- value-typed vocabulary, + | production, not a lab arm + representation probes <- object-model / layout + measurements (historical + directory: valhalla-lab/) ====================== PANAMA FFM ====================== diff --git a/docs/panama.md b/docs/panama.md index 7f5b953..356e1b4 100644 --- a/docs/panama.md +++ b/docs/panama.md @@ -80,16 +80,20 @@ remains the only thing Panama asks for. What changed is that production deliberately uses Valhalla value classes (JEP 401, preview on this build), so `--release 28 --enable-preview` is now part of the shipped build contract. Read every "no preview" claim below as scoped to FFM: **Panama needs no -preview; the vocabulary types do.** The Valhalla lab -(`valhalla-lab/src/valhalla`) is compiled *separately*, with its own -`-source 27 --enable-preview`, into its own output directory -(`results/valhalla-lab/`), and is never on the classpath the production -tests or the bench harness run against. `--enable-preview`-compiled -classfiles carry a preview marker that poisons every consumer that loads -them; keeping the two trees physically separate (rather than, say, -compiling once and gating features at runtime) is what makes this a -structural guarantee rather than a discipline someone could accidentally -violate. See `.claude/knowledge/jdk-toolchain-facts.md` for the exact +preview; the vocabulary types do.** ⊘ **The paragraph that stood here is doubly stale and is replaced.** It said +the lab compiles *separately* with `-source 27 --enable-preview` and is +*"never on the classpath the production tests run against"*, and it argued +that physical separation kept preview marking out of production. All three +are now false: both probe arms run **JDK 28** with `-source 28 -target 28`, +both **compile and load the production API**, and that API is itself +preview-marked — so there is nothing to quarantine production *from*. The +separation argument was load-bearing only while production was preview-free, +and it stopped being that when production adopted JEP 401. + +The isolation that remains is ordinary and smaller: the probe harness builds +into its own output directory and is not on the classpath of the shipped +artifact. It is not a structural guarantee against preview poisoning, because +preview marking is now a property of production itself. See `.claude/knowledge/jdk-toolchain-facts.md` for the exact verified flag matrix across all three JDKs this project touches. ## What Panama did NOT need to solve here diff --git a/docs/valhalla-lab.md b/docs/valhalla-lab.md index 1e118dd..c9cecda 100644 --- a/docs/valhalla-lab.md +++ b/docs/valhalla-lab.md @@ -1,4 +1,15 @@ -# The Valhalla lab — synthesis +# Representation probes — synthesis + +> ⊘ **Legacy name.** The directory is still `valhalla-lab/` and this file is +> still `valhalla-lab.md`, because renaming paths buys churn rather than +> clarity. **The name no longer describes the experiment.** Valhalla is +> production infrastructure — the six vocabulary types are `value record`s on +> JDK 28 (PR #81) — so it is not what is being tested. What this harness +> measures is the CONSEQUENCE of object-model and layout realizations: +> `record` vs `value record` **on the same JDK, against the same +> Valhalla-enabled production API**, with flattening and escape analysis +> toggled. Read every "Valhalla arm" below as "value-record arm", and never as +> evidence that production is fundamentally Panama with Valhalla bolted on. > Companion to `valhalla-lab/README.md` and `valhalla-lab/docs/three-truths.md` > (the raw findings and numbers) and `.claude/knowledge/valhalla-three-truths-method.md`