Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 29 additions & 8 deletions .claude/board/EPIPHANIES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,25 @@
## E-THE-CI-GAP-WAS-THE-TRIGGER-NOT-THE-COVERAGE-1 (2026-09-19)

⊘ **Corrects `E-THE-VALHALLA-FLIP-COST-ONE-WORD-SIX-TIMES-…-1` below**, which
says the `RangeOutOfBounds` break went unseen *"because no CI job here compiles
the native crate."* **That is false, and checkable in one file.**
`.github/workflows/lint.yml` runs `clippy` and `rust-test` jobs that check out
sibling `AdaWorldAPI/lance-graph` (and `ndarray`, and `OGAR`) and then run
`cargo clippy --all-targets -- -D warnings` and `cargo test` from
`native/lgj-abi`. The native crate is compiled against the real sibling on
every push and PR **to this repository**.

**The actual hazard is the TRIGGER.** The workflow is `on: {pull_request,
push}`, which fires for events in *this* repo only. An upstream-only merge in
lance-graph — exactly what added `ExecError::RangeOutOfBounds` — cannot start
it, so the break stayed invisible until the next push here. Coverage was never
missing; the *event* was.

**Consequence for the next session:** do not add a CI job for this — one
exists. The open gap is upstream-change notification, and nothing in this
repository's triggers can close it. Raised by Codex P2 on #82, verified
against the workflow.

## E-PANAMA-TIMES-VALHALLA-IS-ONE-MEMBRANE-AND-THE-LAB-FINALLY-HAS-ONE-VARIABLE-1 (2026-09-19)

**Operator reframing, and it is the correct one:** this was never a JDK 26→28
Expand Down Expand Up @@ -131,14 +153,13 @@ tripwire, not a safety net:** it converts an upstream additive change into a
downstream build failure, and nothing in either repo's CI noticed, because no
CI job here compiles the native crate.

**Two false failures I nearly reported, both mine, both caught by reading.**
The flipped suite showed `1 FAILED` twice. Both times it was
`DoctrineFenceTest` refusing to run — it requires `java/src/main/java` AND
`native/lgj-abi/src` relative to CWD, and I had run the flipped build from a
scratch directory that had neither. The fence is *correct*: its first check is
*"a fence that cannot find its corpus must fail, not skip."* Running the flip
in the real tree (committed first, restored with `git checkout`) gave the true
result. **A harness artifact and a regression look identical in a summary
**Harness note, so the next run is not misread.** The flipped suite reports
`1 FAILED` when run from a directory lacking `java/src/main/java` AND
`native/lgj-abi/src` — `DoctrineFenceTest` resolves both relative to CWD and
refuses to run without them. The fence is behaving correctly: its first check
is *"a fence that cannot find its corpus must fail, not skip."* **Run the flip
in the real tree** (commit first; restore with `git checkout`) or the result is
not the suite's. **A harness artifact and a regression look identical in a summary
line** — this is the ruff trap "a disable that does not apply is
indistinguishable from a guard that is not load-bearing", met from the other
direction.
Expand Down
22 changes: 22 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,25 @@
## 2026-09-19 — PR #81 merged (`07aa441`): production IS the Valhalla arm; Panama × Valhalla is one membrane

**The frame, because it is easy to file this wrong:** this was not a JDK
26→28 upgrade. Production WAS Panama-only with Valhalla in a lab arm; they
are now **one production membrane**, with JDK 28 as the enabling toolchain.
Panama = the verb, Valhalla = the noun, **lance-graph / T0 owns storage**.

- **Toolchain:** `/opt/jdks/jdk-28+16` (Temurin `28+16-ea`). FFM final and
flag-free; JEP 401 preview-gated, so `--release 28 --enable-preview` is part
of the shipped build contract — **the flag is Valhalla's, never FFM's.**
- **Production vocabulary:** the six types are `public value record`,
`isValue() == true`. Gates: **183 native tests**, **409 Java checks**,
identical to the unflipped baseline on the same JDK.
- **Carry this forward:** the **8-byte flattening cliff reproduces on JDK 28**,
and `RowRange` / `Row` (16 B) are on the wrong side of it. Two allocation
measurements got WORSE under value semantics (non-escaping construction,
`Row` hydration) — both honest, neither on a production path. The FFM path is
byte-identical across the object model.
- **Still owed:** `bench/` JMH numbers have NOT been re-taken on JDK 28
(`bench/lib` absent here); `bench/RESULTS.md` is JDK 26 historical.
- Full record: `PR_ARC_INVENTORY.md` PR #81.

## 2026-09-16 — `hop_cached_vs_gather`: the M1b tile pays off on the second hop; the scatter walk is the access-shape question

**PR #79**, `native/lgj-abi/examples/hop_cached_vs_gather.rs` only — no ABI
Expand Down
81 changes: 81 additions & 0 deletions .claude/board/PR_ARC_INVENTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,87 @@
> anti-pattern the imported board rules name. Backfilled below in one
> pass rather than left stale; PR #4 onward gets its entry at merge time.

## PR #81 — Panama × Valhalla as ONE production membrane (merged `07aa441`, branch `claude/great-pascal-k96kok`, head `1c66091`, 3 commits)

**Entry written at merge**, not backfilled.

- **Frame, and it is the point:** NOT a JDK 26→28 migration. The two halves
that were split — Panama FFM in production, Valhalla in the lab — became one
production membrane. JDK 28 is the enabling toolchain, nothing more. Panama
carries the VERB (reach into canonical bytes); Valhalla carries the NOUN
(identity-free semantic addresses over them); **lance-graph / T0 still owns
the only canonical copy.**
- **Added:** `CLAUDE.md` § P0 (the operator mandate); the six vocabulary types
as `public value record` (`LaneId`, `MaskId`, `Ordinal`, `FacetId`,
`RowRange`, `WideFieldMask`); `plan_lower::range_falsifier::
no_opcode_lowers_to_pred_range`; JDK 28 + preview across `bench/run.sh`,
`bench/gate-run.sh`, `java/README.md`, `README.md`, `docs/panama.md`,
`panama-bridge-engineer.md`, `BOOT.md`; `jdk-toolchain-facts.md` § "JDK 28 —
INSTALLED AND MEASURED" with the obtain route.
- **Fixed (this blocked everything):** `lgj-abi` did not compile against
lance-graph `main`. `mask_risc::ExecError` gained `RangeOutOfBounds` with the
`Pred::Range` work and `exec_error_to_status` matches exhaustively. Mapped
into the documented *"bug in THIS file"* family — no new public status, no
ABI bump — and fenced by the falsifier above, **disable-verified
red-then-green**. *Standing hazard, corrected before merge (Codex P2 on #82,
verified against `.github/workflows/lint.yml`): an exhaustive match across a
repo boundary turns an upstream ADDITIVE change into a downstream build
failure. ⊘ The first draft blamed the wrong thing — it said "no CI job here
compiles the native crate", which is FALSE: the `clippy` and `rust-test`
jobs check out sibling `AdaWorldAPI/lance-graph` and run
`cargo clippy --all-targets` / `cargo test` from `native/lgj-abi`, so CI
compiles it on every push and PR **to this repo**. The real gap is the
TRIGGER, not the coverage: `on: {pull_request, push}` fires on this
repository only, so an upstream-only merge in lance-graph cannot start this
workflow and the break stays invisible until someone pushes here. Naming it
as missing coverage would have sent the next session to add a job that
already exists.*
- **Measured** (Temurin `28+16-ea`, fresh `liblgj_abi.so` abi 0.11 avx512):
**183 native tests**; **`ALL PASSED (409 checks)`** — byte-for-byte the same
409 as the unflipped baseline **on the same JDK**, which is the comparison
that matters. All six types `isValue() == true`; substitutability holds.
**The 8-byte flattening cliff REPRODUCES on JDK 28:** `LaneId`/`Ordinal`
(4 B) and `MaskId` (8 B) FLAT; **`RowRange` (2 long, 16 B) and `Row` (16 B)
NOT-FLAT.** Allocation, record → value, 1M ops: construct-into-array
15.26 → **1.87 MiB** (8.2×); allocate+fill 19.07 → 5.65 MiB; `Descriptor`
45.78 → 27.89 MiB; ⊘ **never-escaping 4.73 → 5.75 MiB (WORSE** — EA already
erased the record case**)**; ⊘ **hydrate 65,536 `Row` 1.50 → 2.00 MiB
(WORSE** — `Row` does not flatten**)**. **FFM path byte-identical across
arms:** 712 B/query, 0 Java objects per row, same native bytes.
- **Locked:** `value semantics ≠ flattening` — `RowRange` is a production type
on the wrong side of the cliff and no performance claim may be read off its
`isValue()`. Valhalla changes what the VOCABULARY costs; it does not touch
the membrane. No second graph representation, and **no PRODUCTION row
hydration** introduced — scoped deliberately, because this entry reports a
`hydrate 65,536 Row` number two bullets up and an unqualified claim would
contradict its own measurement. That number is a LAB probe of what hydration
would cost; production takes no such path.
- **The harness is now a REPRESENTATION probe suite, not a "Valhalla lab" —
Valhalla graduated to production infrastructure and is no longer the thing
under test.** `valhalla-lab/` keeps its directory name for path
compatibility only; a doc banner says so. What it measures is object-model
and layout consequence: `record` vs `value record` **on one JDK, against the
same Valhalla-enabled production API**, with flattening and EA toggled.
**The re-scope also removed a confound the harness always carried:** the old
A/B was `(record, JDK 26)` vs `(value record, JDK 27 EA)` — two variables.
Both arms now run JDK 28, so the object model is the only difference. Two things it forced: **preview marking is transitive** (the
`record` arm also runs `--enable-preview`; it is a record arm on a preview
JVM, not a preview-free one), and **`ValueClass.isFlatArray` narrowed from
`Object` to `Object[]`** since 27-jep401ea3, which broke the flattening probe
until adapted.
- **Struck:** the *"production targets a shipped GA JDK, no preview flags — a
real, deliberate strength"* decision, in place in `jdk-toolchain-facts.md`.
It dressed a release constraint as an architectural virtue and had already
licensed the false claim *"production lgj does not depend on Valhalla at
all."*
- **Deferred, stated rather than implied:** `bench/` cannot run here —
`bench/lib` does not exist, so the JMH jars are absent. `bench/RESULTS.md`
numbers remain **JDK 26 historicals, correct when taken**, deliberately
untouched and **NOT re-taken on JDK 28**. `gate-run.sh` carries the new pin
and flags, so the re-run is one command once the jars are present.
- **Confidence:** high on everything measured in-container; the JMH
crossing/allocation figures are unverified on JDK 28 and are labelled so.

## PR #79 — `hop_cached_vs_gather`: the M1b tile pays off on hop two; the scatter walk is the access-shape question (opened 2026-09-16, merged `9cb63e9`, head `c10029b`)

> **Post-merge backfill, owned (2026-09-16):** this entry was NOT written at
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,11 @@ Rust, SoA layout, SIMD, FFM, or graph-engine internals.
|
View / Mask / Lens DSL <- lazy, monotonic-narrowing
|
stable-JDK implementation
|
Valhalla laboratory <- the tiny vocabulary only
JDK 28: Panama FFM + JEP 401 <- value-typed vocabulary,
| production, not a lab arm
representation probes <- object-model / layout
measurements (historical
directory: valhalla-lab/)

====================== PANAMA FFM ======================

Expand Down
24 changes: 14 additions & 10 deletions docs/panama.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,16 +80,20 @@ remains the only thing Panama asks for. What changed is that production
deliberately uses Valhalla value classes (JEP 401, preview on this build), so
`--release 28 --enable-preview` is now part of the shipped build contract.
Read every "no preview" claim below as scoped to FFM: **Panama needs no
preview; the vocabulary types do.** The Valhalla lab
(`valhalla-lab/src/valhalla`) is compiled *separately*, with its own
`-source 27 --enable-preview`, into its own output directory
(`results/valhalla-lab/`), and is never on the classpath the production
tests or the bench harness run against. `--enable-preview`-compiled
classfiles carry a preview marker that poisons every consumer that loads
them; keeping the two trees physically separate (rather than, say,
compiling once and gating features at runtime) is what makes this a
structural guarantee rather than a discipline someone could accidentally
violate. See `.claude/knowledge/jdk-toolchain-facts.md` for the exact
preview; the vocabulary types do.** ⊘ **The paragraph that stood here is doubly stale and is replaced.** It said
the lab compiles *separately* with `-source 27 --enable-preview` and is
*"never on the classpath the production tests run against"*, and it argued
that physical separation kept preview marking out of production. All three
are now false: both probe arms run **JDK 28** with `-source 28 -target 28`,
both **compile and load the production API**, and that API is itself
preview-marked — so there is nothing to quarantine production *from*. The
separation argument was load-bearing only while production was preview-free,
and it stopped being that when production adopted JEP 401.

The isolation that remains is ordinary and smaller: the probe harness builds
into its own output directory and is not on the classpath of the shipped
artifact. It is not a structural guarantee against preview poisoning, because
preview marking is now a property of production itself. See `.claude/knowledge/jdk-toolchain-facts.md` for the exact
verified flag matrix across all three JDKs this project touches.

## What Panama did NOT need to solve here
Expand Down
13 changes: 12 additions & 1 deletion docs/valhalla-lab.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,15 @@
# The Valhalla lab — synthesis
# Representation probes — synthesis

> ⊘ **Legacy name.** The directory is still `valhalla-lab/` and this file is
> still `valhalla-lab.md`, because renaming paths buys churn rather than
> clarity. **The name no longer describes the experiment.** Valhalla is
> production infrastructure — the six vocabulary types are `value record`s on
> JDK 28 (PR #81) — so it is not what is being tested. What this harness
> measures is the CONSEQUENCE of object-model and layout realizations:
> `record` vs `value record` **on the same JDK, against the same
> Valhalla-enabled production API**, with flattening and escape analysis
> toggled. Read every "Valhalla arm" below as "value-record arm", and never as
> evidence that production is fundamentally Panama with Valhalla bolted on.

> Companion to `valhalla-lab/README.md` and `valhalla-lab/docs/three-truths.md`
> (the raw findings and numbers) and `.claude/knowledge/valhalla-three-truths-method.md`
Expand Down
Loading