Skip to content

Commit 07aa441

Browse files
authored
Merge pull request #81 from AdaWorldAPI/claude/great-pascal-k96kok
P0: JDK 28 + Valhalla value classes + Panama — implemented, 409 checks green
2 parents f5fa792 + 1c66091 commit 07aa441

34 files changed

Lines changed: 756 additions & 287 deletions

.claude/agents/BOOT.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,11 @@ matched to the actual size of the problem.
2424
`jextract`/`cbindgen` out of habit).
2525
5. **`.claude/knowledge/jdk-toolchain-facts.md`** — which JDK path to
2626
use for which purpose. Getting this wrong (e.g. using `/usr/bin/java`
27-
instead of `/opt/jdks/jdk-26.0.2`) produces confusing preview-flag
27+
instead of `/opt/jdks/jdk-28+16`) produces confusing preview-flag
2828
errors that look like a design problem but are a toolchain-selection
29-
mistake.
29+
mistake. On JDK 28 the production build carries `--release 28
30+
--enable-preview` for the Valhalla value types — FFM itself needs no
31+
preview flag.
3032
6. **`.claude/knowledge/agent-cargo-hygiene.md`** — operator directive:
3133
spawned agents do NOT run `cargo` in any form (build/check/test/
3234
clippy), ever. Only the orchestrating main thread compiles. This

.claude/agents/panama-bridge-engineer.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -63,9 +63,12 @@ on both sides.
6363
Java side actually implements its half.
6464
7. **`--enable-native-access` is required and documented** in
6565
`java/README.md`'s exact command lines — verify the commands there
66-
actually run against `/opt/jdks/jdk-26.0.2` (see
67-
`.claude/knowledge/jdk-toolchain-facts.md`) without additional flags
68-
beyond that one.
66+
actually run against `/opt/jdks/jdk-28+16` (see
67+
`.claude/knowledge/jdk-toolchain-facts.md`). **Two flags, two owners:**
68+
`--enable-native-access=ALL-UNNAMED` is FFM's and the only one Panama
69+
needs; `--enable-preview` is VALHALLA's, required because the production
70+
vocabulary types are `value record`s. Never attribute the preview flag to
71+
FFM — on JDK 28 FFM is final.
6972

7073
## What you are not
7174

.claude/board/EPIPHANIES.md

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,187 @@
1+
## E-PANAMA-TIMES-VALHALLA-IS-ONE-MEMBRANE-AND-THE-LAB-FINALLY-HAS-ONE-VARIABLE-1 (2026-09-19)
2+
3+
**Operator reframing, and it is the correct one:** this was never a JDK 26→28
4+
migration. It is **Panama-only production + Valhalla-lab arm → one production
5+
Panama × Valhalla membrane.** JDK 28 is the enabling toolchain, not the point.
6+
Panama answers *where are the bytes and how does Java reach them without
7+
copying*; Valhalla answers *how does Java name and carry semantic values over
8+
those bytes without identity-heavy heap objects*. lance-graph / T0 remains the
9+
only owner of canonical storage.
10+
11+
### Measured on JDK 28 (`/opt/jdks/jdk-28+16`), production value-shaped
12+
13+
| gate | result |
14+
|---|---|
15+
| native `lgj-abi` | **183 tests** green (182 + the new Range falsifier) |
16+
| Java `AllTests`, production as `value record` | **ALL PASSED (409 checks)** — identical to the unflipped baseline on the same JDK |
17+
| all six vocabulary types | `isValue() == true`; substitutability holds |
18+
19+
### The flattening cliff is UNCHANGED on JDK 28 — and it bites one of our six
20+
21+
Measured by the lab, both arms on one JDK:
22+
23+
| type | payload | array |
24+
|---|---|---|
25+
| `LaneId` (1 int) | 4 B | **FLAT** |
26+
| `Ordinal` (1 int) | 4 B | **FLAT** |
27+
| `MaskId` (1 long) | 8 B | **FLAT** |
28+
| `RowRange` (2 long) | 16 B | **NOT-FLAT** |
29+
| `Row` (1 long + 2 int) | 16 B | **NOT-FLAT** |
30+
31+
The ≤8-byte cliff measured on the 27-jep401ea3 build **reproduces on JDK 28**.
32+
So `value semantics` and `flattening` remain two different claims, and
33+
**`RowRange` is a real production type on the wrong side of the cliff.** No
34+
performance claim may be made for it from `isValue() == true`.
35+
36+
### Allocation deltas, record arm → value arm (1M ops, same JDK)
37+
38+
| measurement | record | value | delta |
39+
|---|---|---|---|
40+
| construct N `LaneId` into an array | 15.26 MiB | **1.87 MiB** | **8.2× less** (16 B → 1.96 B each) |
41+
| allocate+fill `LaneId[N]` | 19.07 MiB | **5.65 MiB** | 3.4× less |
42+
| construct N `Descriptor` (2 wrappers) | 45.78 MiB | **27.89 MiB** | 1.6× less |
43+
| construct N `LaneId`, never escaping | 4.73 MiB | **5.75 MiB** |**value arm allocates MORE** — escape analysis already erased the record case |
44+
| hydrate 65,536 `Row` | 1.50 MiB | **2.00 MiB** |**worse**`Row` is 16 B, NOT-FLAT |
45+
46+
**Two of the five went the wrong way, and both are honest.** Where EA already
47+
won, value classes add nothing; where the shape does not flatten, hydration
48+
costs more. Production does not hydrate rows, so the second is a warning about
49+
a path we do not take — not a regression we shipped.
50+
51+
### The Panama path is byte-identical across the object model
52+
53+
| FFM measurement | record arm | value arm |
54+
|---|---|---|
55+
| Java bytes allocated per query (warm) | 712 B | **712 B** |
56+
| Java objects per row | 0 | **0** |
57+
| native lane bytes / mask bytes | 1.00 MiB / 8.0 KiB | **identical** |
58+
59+
**That is the integration's central invariant, measured rather than asserted:**
60+
Valhalla changes what the *vocabulary* costs; it does not touch the membrane.
61+
No second graph representation appeared, and no row hydration was introduced
62+
to "use Valhalla".
63+
64+
### The lab's re-scope removed a confound it had carried since it was built
65+
66+
The old A/B was `(record, JDK 26)` vs `(value record, JDK 27 EA)`**two
67+
variables**. Both arms now run on JDK 28, so the object model is the only
68+
difference and the diff is attributable. The lab stops being
69+
present-vs-future and becomes a VM/representation probe suite over the real
70+
production vocabulary.
71+
72+
Two things the re-scope forced, both real:
73+
- **Preview marking is transitive.** Production classfiles are preview-marked,
74+
so the `record` arm *also* runs `--enable-preview`. It is a record arm on a
75+
preview-enabled JVM, not a preview-free arm; the "stable" label survives only
76+
as a path.
77+
- **A JDK 28 internal-API change.** `jdk.internal.value.ValueClass.isFlatArray`
78+
narrowed from `Object` to `Object[]` since 27-jep401ea3; the flattening probe
79+
was adapted (a non-`Object[]` argument answers `NOT-FLAT` without reaching
80+
the internal API).
81+
82+
### The `RangeOutOfBounds` fix is compatibility plumbing, with a falsifier
83+
84+
`mask_risc::ExecError` gained `RangeOutOfBounds` with lance-graph's
85+
`Pred::Range` work and `exec_error_to_status` matches exhaustively, so
86+
`lgj-abi` **did not compile against lance-graph `main`**. Mapped into the
87+
documented *"bug in THIS file"* family — no new public status, no ABI bump —
88+
and backed by `plan_lower::range_falsifier::no_opcode_lowers_to_pred_range`,
89+
which sweeps every opcode through the real `lower_plan` and reads every
90+
emitted predicate. **Disable-verified red-then-green:** pointing `LGJ_OP_EQ_U32`
91+
at `Pred::Range` fails it with the intended message; restoring passes. When LGJ
92+
gains a Range lowering this goes red and forces a deliberate caller-visible
93+
mapping instead of inheriting `LGJ_ERR_ALLOCATION_FAILED`, which would then be
94+
a lie.
95+
96+
### What could not be measured here, stated rather than implied
97+
98+
`bench/` cannot run in this container — `bench/lib` does not exist, so the JMH
99+
jars are absent. The crossing-count and per-call allocation figures in
100+
`bench/RESULTS.md` are **historical JDK 26 measurements, correct when taken**,
101+
and are deliberately left untouched; they have NOT been re-taken on JDK 28.
102+
`gate-run.sh` carries the new pin and flags so the re-run is one command once
103+
the jars are present.
104+
105+
## E-THE-VALHALLA-FLIP-COST-ONE-WORD-SIX-TIMES-AND-THE-GATES-DID-NOT-MOVE-1 (2026-09-19)
106+
107+
**Finding, measured end to end today.** The P0 mandate (JDK 28 + Valhalla +
108+
Panama) is implemented. `/opt/jdks/jdk-28+16`, Temurin `28+16-ea`.
109+
110+
| arm | result |
111+
|---|---|
112+
| JDK 28, sources unchanged, no preview | compiles; **409 checks, 0 failed** |
113+
| JDK 28, six types `public value record`, `--enable-preview` | compiles; **409 checks, 0 failed** |
114+
| `isValue()` on all six at runtime | **true**; `new LaneId(7) == new LaneId(7)` → true |
115+
| FFM without any preview flag | final; `SysVx64Linker`; `MemorySegment` round-trip clean |
116+
117+
**The lab's central claim is now demonstrated on production sources:** the
118+
migration is one word, six times, and the gate suite does not move. It was
119+
previously proven only on `valhalla-lab`'s parallel vocabulary.
120+
121+
**A cross-repo break found on the way, and it was blocking everything.**
122+
`lgj-abi` did not compile against current lance-graph `main`:
123+
`mask_risc::ExecError` gained `RangeOutOfBounds { lo, hi, n_rows }` with the
124+
`Pred::Range` work, and `exports.rs`'s `exec_error_to_status` maps `ExecError`
125+
**exhaustively**. Fixed by joining the documented *"would be a bug in THIS
126+
file, not in a caller's plan"* family (`LGJ_ERR_ALLOCATION_FAILED`) — the ABI
127+
lowering emits no `Pred::Range`, exactly as it emits no sum terminal and no
128+
blend — and the doc comment's enumeration was extended so the list stays
129+
exhaustive in prose too. **An exhaustive match across a repo boundary is a
130+
tripwire, not a safety net:** it converts an upstream additive change into a
131+
downstream build failure, and nothing in either repo's CI noticed, because no
132+
CI job here compiles the native crate.
133+
134+
**Two false failures I nearly reported, both mine, both caught by reading.**
135+
The flipped suite showed `1 FAILED` twice. Both times it was
136+
`DoctrineFenceTest` refusing to run — it requires `java/src/main/java` AND
137+
`native/lgj-abi/src` relative to CWD, and I had run the flipped build from a
138+
scratch directory that had neither. The fence is *correct*: its first check is
139+
*"a fence that cannot find its corpus must fail, not skip."* Running the flip
140+
in the real tree (committed first, restored with `git checkout`) gave the true
141+
result. **A harness artifact and a regression look identical in a summary
142+
line** — this is the ruff trap "a disable that does not apply is
143+
indistinguishable from a guard that is not load-bearing", met from the other
144+
direction.
145+
146+
## E-VALHALLA-IS-THE-STORAGE-MEMBRANE-AND-IT-IS-MANDATORY-1 (2026-09-19)
147+
148+
**Operator ruling.** *LGJ MUST use JDK 28 and MUST use Valhalla and Panama.*
149+
Recorded as canon in `CLAUDE.md` § P0; the superseded decision is struck in
150+
place in `.claude/knowledge/jdk-toolchain-facts.md`.
151+
152+
**The violation it corrects, in my own words this session:** *"Production lgj
153+
does not depend on Valhalla at all."* That sentence is wrong twice. It is
154+
wrong about the design — the vocabulary types are written identity-free
155+
precisely so the same source compiles as `value record` with one word changed,
156+
which is a dependency in shape that `valhalla-lab/docs/three-truths.md`
157+
already measured (every behaviour the API uses is identical across both object
158+
models; the rows that differ — `==`, identity hash, `synchronized`,
159+
null-restricted arrays — are exactly the ones the API never touches). And it
160+
is wrong about the architecture: **Panama carries the verb, Valhalla carries
161+
the noun, lance-graph owns the reality.** Valhalla is the STORAGE MEMBRANE,
162+
never the storage owner. Drop it and Java can still reach the bytes, but it
163+
reaches them as offset/stride/segment/handle — it has not adopted the
164+
substrate's storage vocabulary, and zero-copy stops being a programming model
165+
and goes back to being an FFI trick.
166+
167+
**How the error got in, which is the transferable part.** The knowledge doc
168+
called the GA-JDK target *"a real, deliberate strength of the design."* A
169+
release constraint dressed as a virtue is an architectural claim, and a later
170+
session (this one) read it back as one. A toolchain doc may record what a
171+
toolchain CAN do; the moment it says what the design SHOULD therefore be, it
172+
has started legislating outside its evidence.
173+
174+
**Unaffected by the ruling, stated so the next session does not over-apply
175+
it:** E4 — the Vector API remains a lab arm; JDK 28 finalizing it gives Java
176+
no backend, because Java has no backends (kernels stay in `ndarray::simd`).
177+
`--enable-preview` remains classfile-poisoning, so the flag posture is
178+
repo-wide and uniform, never mixed. Value classes make a DESCRIPTOR cheap;
179+
they never make a population crossable.
180+
181+
**Status: MANDATED, NOT IMPLEMENTED.** No JDK 28 in this container
182+
(`/opt/jdks` absent, system `java` 21.0.10) and `jdk.java.net` is proxy-blocked,
183+
so the migration is filed, not done: `ISS-LGJ-TOOLCHAIN-MUST-BE-JDK28-VALHALLA-PANAMA`.
184+
1185
## E-THE-PIN-THAT-CLAIMED-ONE-PLACE-CREATED-A-SECOND-1 (2026-09-05)
2186

3187
**Finding.** The first CI gate this repo ever had went red on its first

.claude/board/ISSUES.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,67 @@
1+
## ISS-LGJ-TOOLCHAIN-MUST-BE-JDK28-VALHALLA-PANAMA — UNBLOCKED; JDK 28 installed and the flip proven (2026-09-19)
2+
3+
⊘ The entry below says the migration is blocked because no JDK 28 can be
4+
fetched in this container. **That was true of apt and of every JDK
5+
distribution host — and wrong as a conclusion.** GitHub release *download*
6+
paths pass the gateway (`releases.atom` and the API do not), so
7+
`git ls-remote` gave the tag and a direct download gave the tarball:
8+
**`/opt/jdks/jdk-28+16`** (Temurin `28+16-ea`). Route, asset name and every
9+
probe are recorded in `.claude/knowledge/jdk-toolchain-facts.md` § "JDK 28 —
10+
INSTALLED AND MEASURED". Steps 1 and 2 of the plan below are **done and
11+
green**: JEP 401 is preview-gated on this build, FFM is final and flag-free,
12+
and all six vocabulary types flip with one word and report
13+
`isValue() == true` at runtime **on the production sources**, which compile
14+
clean under JDK 28 both as-is and flipped.
15+
16+
**What remains, and it is only this:** the flip is proven in a scratch copy,
17+
not committed — landing it needs the gate suite (`ApiSurfaceTest`,
18+
`GraphHopTest` allowlist + G2, `AbiContractTest`, `g11_contract_import_fence`,
19+
the allocation gates) run against a freshly built `liblgj_abi.so` with
20+
re-pinned allocation numbers, plus the build scripts carrying
21+
`--enable-preview` repo-wide, plus `valhalla-lab`'s A/B re-scoped in the same
22+
PR. Nothing architectural is open.
23+
24+
## ISS-LGJ-TOOLCHAIN-MUST-BE-JDK28-VALHALLA-PANAMA
25+
26+
**Status:** OPEN. **Filed:** 2026-09-19. **Severity:** high — the repo's
27+
mandated toolchain and its actual toolchain differ.
28+
29+
**Mandate** (operator, `CLAUDE.md` § P0): LGJ MUST use JDK 28 and MUST use
30+
Valhalla and Panama. **Blocked in the filing container, not by the work:**
31+
`/opt/jdks` does not exist here (the JDK 26/27 paths in
32+
`.claude/knowledge/jdk-toolchain-facts.md` were pinned in a different
33+
environment), system `java` is 21.0.10, and `jdk.java.net` is refused by the
34+
egress proxy (`connect_rejected`, organization policy), so no JDK 28 can be
35+
fetched or verified from this session.
36+
37+
**Steps, in order, for a session that has a JDK 28:**
38+
39+
1. **Verify by execution, never by doc** (this repo's standing rule): does
40+
JEP 401 require `--enable-preview` on that build? Record `Class::isValue()`
41+
on a `value record` and the exact build string in
42+
`jdk-toolchain-facts.md`. If the answer is preview, the flag is repo-wide.
43+
2. **Flip the vocabulary types to `value`**`LaneId`, `MaskId`, `Ordinal`,
44+
`FacetId`, `RowRange`, `WideFieldMask`. The lab proved this is a one-word
45+
change per type; `run.sh` step 0 already enforces that the two sources
46+
differ by exactly that word.
47+
3. **Re-run the whole gate suite**`ApiSurfaceTest` (no FFM type in a public
48+
signature), `GraphHopTest` reflective allowlist + G2 no-per-row-engine,
49+
`AbiContractTest` layout parity, `g11_contract_import_fence`, and the
50+
allocation gates. The allocation numbers are the ones expected to MOVE;
51+
re-pin them from measurement, never predict them (measure-then-pin).
52+
4. **Re-measure the 8-byte flattening cliff** from `three-truths.md` on the
53+
new build before quoting the old number anywhere.
54+
5. **`valhalla-lab/` loses its reason to exist as an A/B** once production IS
55+
the Valhalla arm. Retire or re-scope it deliberately, in the same PR, with
56+
its measurements preserved on the board — do not leave a lab whose stable
57+
arm is the thing production no longer is.
58+
6. **E4 stays**: no Vector API in `src/main`, whatever JDK 28 finalizes.
59+
60+
**Falsifier for "done":** the production build runs on JDK 28, every
61+
vocabulary type reports `isValue() == true` at runtime in a test, all gates
62+
are green with re-pinned allocation numbers, and no doc still states the
63+
JDK-26 framing as current.
64+
165
# Issues Log — Open + Resolved (double-entry, append-only)
266

367
## ISS-LGJ-CROSS-REPO-CITATION-GOES-STALE-SILENTLY (2026-09-03) — OPEN

0 commit comments

Comments
 (0)