diff --git a/.claude/agents/BOOT.md b/.claude/agents/BOOT.md index aaa2e21..2c3ba49 100644 --- a/.claude/agents/BOOT.md +++ b/.claude/agents/BOOT.md @@ -24,9 +24,11 @@ matched to the actual size of the problem. `jextract`/`cbindgen` out of habit). 5. **`.claude/knowledge/jdk-toolchain-facts.md`** — which JDK path to use for which purpose. Getting this wrong (e.g. using `/usr/bin/java` - instead of `/opt/jdks/jdk-26.0.2`) produces confusing preview-flag + instead of `/opt/jdks/jdk-28+16`) produces confusing preview-flag errors that look like a design problem but are a toolchain-selection - mistake. + mistake. On JDK 28 the production build carries `--release 28 + --enable-preview` for the Valhalla value types — FFM itself needs no + preview flag. 6. **`.claude/knowledge/agent-cargo-hygiene.md`** — operator directive: spawned agents do NOT run `cargo` in any form (build/check/test/ clippy), ever. Only the orchestrating main thread compiles. This diff --git a/.claude/agents/panama-bridge-engineer.md b/.claude/agents/panama-bridge-engineer.md index 0732b07..16bc4a2 100644 --- a/.claude/agents/panama-bridge-engineer.md +++ b/.claude/agents/panama-bridge-engineer.md @@ -63,9 +63,12 @@ on both sides. Java side actually implements its half. 7. **`--enable-native-access` is required and documented** in `java/README.md`'s exact command lines — verify the commands there - actually run against `/opt/jdks/jdk-26.0.2` (see - `.claude/knowledge/jdk-toolchain-facts.md`) without additional flags - beyond that one. + actually run against `/opt/jdks/jdk-28+16` (see + `.claude/knowledge/jdk-toolchain-facts.md`). **Two flags, two owners:** + `--enable-native-access=ALL-UNNAMED` is FFM's and the only one Panama + needs; `--enable-preview` is VALHALLA's, required because the production + vocabulary types are `value record`s. Never attribute the preview flag to + FFM — on JDK 28 FFM is final. ## What you are not diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index d502f38..8efd3da 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,187 @@ +## E-PANAMA-TIMES-VALHALLA-IS-ONE-MEMBRANE-AND-THE-LAB-FINALLY-HAS-ONE-VARIABLE-1 (2026-09-19) + +**Operator reframing, and it is the correct one:** this was never a JDK 26→28 +migration. It is **Panama-only production + Valhalla-lab arm → one production +Panama × Valhalla membrane.** JDK 28 is the enabling toolchain, not the point. +Panama answers *where are the bytes and how does Java reach them without +copying*; Valhalla answers *how does Java name and carry semantic values over +those bytes without identity-heavy heap objects*. lance-graph / T0 remains the +only owner of canonical storage. + +### Measured on JDK 28 (`/opt/jdks/jdk-28+16`), production value-shaped + +| gate | result | +|---|---| +| native `lgj-abi` | **183 tests** green (182 + the new Range falsifier) | +| Java `AllTests`, production as `value record` | **ALL PASSED (409 checks)** — identical to the unflipped baseline on the same JDK | +| all six vocabulary types | `isValue() == true`; substitutability holds | + +### The flattening cliff is UNCHANGED on JDK 28 — and it bites one of our six + +Measured by the lab, both arms on one JDK: + +| type | payload | array | +|---|---|---| +| `LaneId` (1 int) | 4 B | **FLAT** | +| `Ordinal` (1 int) | 4 B | **FLAT** | +| `MaskId` (1 long) | 8 B | **FLAT** | +| `RowRange` (2 long) | 16 B | **NOT-FLAT** | +| `Row` (1 long + 2 int) | 16 B | **NOT-FLAT** | + +The ≤8-byte cliff measured on the 27-jep401ea3 build **reproduces on JDK 28**. +So `value semantics` and `flattening` remain two different claims, and +**`RowRange` is a real production type on the wrong side of the cliff.** No +performance claim may be made for it from `isValue() == true`. + +### Allocation deltas, record arm → value arm (1M ops, same JDK) + +| measurement | record | value | delta | +|---|---|---|---| +| construct N `LaneId` into an array | 15.26 MiB | **1.87 MiB** | **8.2× less** (16 B → 1.96 B each) | +| allocate+fill `LaneId[N]` | 19.07 MiB | **5.65 MiB** | 3.4× less | +| construct N `Descriptor` (2 wrappers) | 45.78 MiB | **27.89 MiB** | 1.6× less | +| construct N `LaneId`, never escaping | 4.73 MiB | **5.75 MiB** | ⊘ **value arm allocates MORE** — escape analysis already erased the record case | +| hydrate 65,536 `Row` | 1.50 MiB | **2.00 MiB** | ⊘ **worse** — `Row` is 16 B, NOT-FLAT | + +**Two of the five went the wrong way, and both are honest.** Where EA already +won, value classes add nothing; where the shape does not flatten, hydration +costs more. Production does not hydrate rows, so the second is a warning about +a path we do not take — not a regression we shipped. + +### The Panama path is byte-identical across the object model + +| FFM measurement | record arm | value arm | +|---|---|---| +| Java bytes allocated per query (warm) | 712 B | **712 B** | +| Java objects per row | 0 | **0** | +| native lane bytes / mask bytes | 1.00 MiB / 8.0 KiB | **identical** | + +**That is the integration's central invariant, measured rather than asserted:** +Valhalla changes what the *vocabulary* costs; it does not touch the membrane. +No second graph representation appeared, and no row hydration was introduced +to "use Valhalla". + +### The lab's re-scope removed a confound it had carried since it was built + +The old A/B was `(record, JDK 26)` vs `(value record, JDK 27 EA)` — **two +variables**. Both arms now run on JDK 28, so the object model is the only +difference and the diff is attributable. The lab stops being +present-vs-future and becomes a VM/representation probe suite over the real +production vocabulary. + +Two things the re-scope forced, both real: +- **Preview marking is transitive.** Production classfiles are preview-marked, + so the `record` arm *also* runs `--enable-preview`. It is a record arm on a + preview-enabled JVM, not a preview-free arm; the "stable" label survives only + as a path. +- **A JDK 28 internal-API change.** `jdk.internal.value.ValueClass.isFlatArray` + narrowed from `Object` to `Object[]` since 27-jep401ea3; the flattening probe + was adapted (a non-`Object[]` argument answers `NOT-FLAT` without reaching + the internal API). + +### The `RangeOutOfBounds` fix is compatibility plumbing, with a falsifier + +`mask_risc::ExecError` gained `RangeOutOfBounds` with lance-graph's +`Pred::Range` work and `exec_error_to_status` matches exhaustively, so +`lgj-abi` **did not compile against lance-graph `main`**. Mapped into the +documented *"bug in THIS file"* family — no new public status, no ABI bump — +and backed by `plan_lower::range_falsifier::no_opcode_lowers_to_pred_range`, +which sweeps every opcode through the real `lower_plan` and reads every +emitted predicate. **Disable-verified red-then-green:** pointing `LGJ_OP_EQ_U32` +at `Pred::Range` fails it with the intended message; restoring passes. When LGJ +gains a Range lowering this goes red and forces a deliberate caller-visible +mapping instead of inheriting `LGJ_ERR_ALLOCATION_FAILED`, which would then be +a lie. + +### What could not be measured here, stated rather than implied + +`bench/` cannot run in this container — `bench/lib` does not exist, so the JMH +jars are absent. The crossing-count and per-call allocation figures in +`bench/RESULTS.md` are **historical JDK 26 measurements, correct when taken**, +and are deliberately left untouched; they have NOT been re-taken on JDK 28. +`gate-run.sh` carries the new pin and flags so the re-run is one command once +the jars are present. + +## E-THE-VALHALLA-FLIP-COST-ONE-WORD-SIX-TIMES-AND-THE-GATES-DID-NOT-MOVE-1 (2026-09-19) + +**Finding, measured end to end today.** The P0 mandate (JDK 28 + Valhalla + +Panama) is implemented. `/opt/jdks/jdk-28+16`, Temurin `28+16-ea`. + +| arm | result | +|---|---| +| JDK 28, sources unchanged, no preview | compiles; **409 checks, 0 failed** | +| JDK 28, six types `public value record`, `--enable-preview` | compiles; **409 checks, 0 failed** | +| `isValue()` on all six at runtime | **true**; `new LaneId(7) == new LaneId(7)` → true | +| FFM without any preview flag | final; `SysVx64Linker`; `MemorySegment` round-trip clean | + +**The lab's central claim is now demonstrated on production sources:** the +migration is one word, six times, and the gate suite does not move. It was +previously proven only on `valhalla-lab`'s parallel vocabulary. + +**A cross-repo break found on the way, and it was blocking everything.** +`lgj-abi` did not compile against current lance-graph `main`: +`mask_risc::ExecError` gained `RangeOutOfBounds { lo, hi, n_rows }` with the +`Pred::Range` work, and `exports.rs`'s `exec_error_to_status` maps `ExecError` +**exhaustively**. Fixed by joining the documented *"would be a bug in THIS +file, not in a caller's plan"* family (`LGJ_ERR_ALLOCATION_FAILED`) — the ABI +lowering emits no `Pred::Range`, exactly as it emits no sum terminal and no +blend — and the doc comment's enumeration was extended so the list stays +exhaustive in prose too. **An exhaustive match across a repo boundary is a +tripwire, not a safety net:** it converts an upstream additive change into a +downstream build failure, and nothing in either repo's CI noticed, because no +CI job here compiles the native crate. + +**Two false failures I nearly reported, both mine, both caught by reading.** +The flipped suite showed `1 FAILED` twice. Both times it was +`DoctrineFenceTest` refusing to run — it requires `java/src/main/java` AND +`native/lgj-abi/src` relative to CWD, and I had run the flipped build from a +scratch directory that had neither. The fence is *correct*: its first check is +*"a fence that cannot find its corpus must fail, not skip."* Running the flip +in the real tree (committed first, restored with `git checkout`) gave the true +result. **A harness artifact and a regression look identical in a summary +line** — this is the ruff trap "a disable that does not apply is +indistinguishable from a guard that is not load-bearing", met from the other +direction. + +## E-VALHALLA-IS-THE-STORAGE-MEMBRANE-AND-IT-IS-MANDATORY-1 (2026-09-19) + +**Operator ruling.** *LGJ MUST use JDK 28 and MUST use Valhalla and Panama.* +Recorded as canon in `CLAUDE.md` § P0; the superseded decision is struck in +place in `.claude/knowledge/jdk-toolchain-facts.md`. + +**The violation it corrects, in my own words this session:** *"Production lgj +does not depend on Valhalla at all."* That sentence is wrong twice. It is +wrong about the design — the vocabulary types are written identity-free +precisely so the same source compiles as `value record` with one word changed, +which is a dependency in shape that `valhalla-lab/docs/three-truths.md` +already measured (every behaviour the API uses is identical across both object +models; the rows that differ — `==`, identity hash, `synchronized`, +null-restricted arrays — are exactly the ones the API never touches). And it +is wrong about the architecture: **Panama carries the verb, Valhalla carries +the noun, lance-graph owns the reality.** Valhalla is the STORAGE MEMBRANE, +never the storage owner. Drop it and Java can still reach the bytes, but it +reaches them as offset/stride/segment/handle — it has not adopted the +substrate's storage vocabulary, and zero-copy stops being a programming model +and goes back to being an FFI trick. + +**How the error got in, which is the transferable part.** The knowledge doc +called the GA-JDK target *"a real, deliberate strength of the design."* A +release constraint dressed as a virtue is an architectural claim, and a later +session (this one) read it back as one. A toolchain doc may record what a +toolchain CAN do; the moment it says what the design SHOULD therefore be, it +has started legislating outside its evidence. + +**Unaffected by the ruling, stated so the next session does not over-apply +it:** E4 — the Vector API remains a lab arm; JDK 28 finalizing it gives Java +no backend, because Java has no backends (kernels stay in `ndarray::simd`). +`--enable-preview` remains classfile-poisoning, so the flag posture is +repo-wide and uniform, never mixed. Value classes make a DESCRIPTOR cheap; +they never make a population crossable. + +**Status: MANDATED, NOT IMPLEMENTED.** No JDK 28 in this container +(`/opt/jdks` absent, system `java` 21.0.10) and `jdk.java.net` is proxy-blocked, +so the migration is filed, not done: `ISS-LGJ-TOOLCHAIN-MUST-BE-JDK28-VALHALLA-PANAMA`. + ## E-THE-PIN-THAT-CLAIMED-ONE-PLACE-CREATED-A-SECOND-1 (2026-09-05) **Finding.** The first CI gate this repo ever had went red on its first diff --git a/.claude/board/ISSUES.md b/.claude/board/ISSUES.md index d037f80..127547d 100644 --- a/.claude/board/ISSUES.md +++ b/.claude/board/ISSUES.md @@ -1,3 +1,67 @@ +## ISS-LGJ-TOOLCHAIN-MUST-BE-JDK28-VALHALLA-PANAMA — UNBLOCKED; JDK 28 installed and the flip proven (2026-09-19) + +⊘ The entry below says the migration is blocked because no JDK 28 can be +fetched in this container. **That was true of apt and of every JDK +distribution host — and wrong as a conclusion.** GitHub release *download* +paths pass the gateway (`releases.atom` and the API do not), so +`git ls-remote` gave the tag and a direct download gave the tarball: +**`/opt/jdks/jdk-28+16`** (Temurin `28+16-ea`). Route, asset name and every +probe are recorded in `.claude/knowledge/jdk-toolchain-facts.md` § "JDK 28 — +INSTALLED AND MEASURED". Steps 1 and 2 of the plan below are **done and +green**: JEP 401 is preview-gated on this build, FFM is final and flag-free, +and all six vocabulary types flip with one word and report +`isValue() == true` at runtime **on the production sources**, which compile +clean under JDK 28 both as-is and flipped. + +**What remains, and it is only this:** the flip is proven in a scratch copy, +not committed — landing it needs the gate suite (`ApiSurfaceTest`, +`GraphHopTest` allowlist + G2, `AbiContractTest`, `g11_contract_import_fence`, +the allocation gates) run against a freshly built `liblgj_abi.so` with +re-pinned allocation numbers, plus the build scripts carrying +`--enable-preview` repo-wide, plus `valhalla-lab`'s A/B re-scoped in the same +PR. Nothing architectural is open. + +## ISS-LGJ-TOOLCHAIN-MUST-BE-JDK28-VALHALLA-PANAMA + +**Status:** OPEN. **Filed:** 2026-09-19. **Severity:** high — the repo's +mandated toolchain and its actual toolchain differ. + +**Mandate** (operator, `CLAUDE.md` § P0): LGJ MUST use JDK 28 and MUST use +Valhalla and Panama. **Blocked in the filing container, not by the work:** +`/opt/jdks` does not exist here (the JDK 26/27 paths in +`.claude/knowledge/jdk-toolchain-facts.md` were pinned in a different +environment), system `java` is 21.0.10, and `jdk.java.net` is refused by the +egress proxy (`connect_rejected`, organization policy), so no JDK 28 can be +fetched or verified from this session. + +**Steps, in order, for a session that has a JDK 28:** + +1. **Verify by execution, never by doc** (this repo's standing rule): does + JEP 401 require `--enable-preview` on that build? Record `Class::isValue()` + on a `value record` and the exact build string in + `jdk-toolchain-facts.md`. If the answer is preview, the flag is repo-wide. +2. **Flip the vocabulary types to `value`** — `LaneId`, `MaskId`, `Ordinal`, + `FacetId`, `RowRange`, `WideFieldMask`. The lab proved this is a one-word + change per type; `run.sh` step 0 already enforces that the two sources + differ by exactly that word. +3. **Re-run the whole gate suite** — `ApiSurfaceTest` (no FFM type in a public + signature), `GraphHopTest` reflective allowlist + G2 no-per-row-engine, + `AbiContractTest` layout parity, `g11_contract_import_fence`, and the + allocation gates. The allocation numbers are the ones expected to MOVE; + re-pin them from measurement, never predict them (measure-then-pin). +4. **Re-measure the 8-byte flattening cliff** from `three-truths.md` on the + new build before quoting the old number anywhere. +5. **`valhalla-lab/` loses its reason to exist as an A/B** once production IS + the Valhalla arm. Retire or re-scope it deliberately, in the same PR, with + its measurements preserved on the board — do not leave a lab whose stable + arm is the thing production no longer is. +6. **E4 stays**: no Vector API in `src/main`, whatever JDK 28 finalizes. + +**Falsifier for "done":** the production build runs on JDK 28, every +vocabulary type reports `isValue() == true` at runtime in a test, all gates +are green with re-pinned allocation numbers, and no doc still states the +JDK-26 framing as current. + # Issues Log — Open + Resolved (double-entry, append-only) ## ISS-LGJ-CROSS-REPO-CITATION-GOES-STALE-SILENTLY (2026-09-03) — OPEN diff --git a/.claude/knowledge/jdk-toolchain-facts.md b/.claude/knowledge/jdk-toolchain-facts.md index db5be3c..549d849 100644 --- a/.claude/knowledge/jdk-toolchain-facts.md +++ b/.claude/knowledge/jdk-toolchain-facts.md @@ -19,6 +19,56 @@ build baseline (SIGILL trap if omitted downstream — see `abi.md` §"SIGILL trap"). Any crate depending on `ndarray`'s AVX2 backend must mirror this. +## JDK 28 — INSTALLED AND MEASURED (2026-09-19) + +**`/opt/jdks/jdk-28+16`** — Temurin `28+16-ea`, `openjdk version "28-beta"`, +GA date 2027-03-23, build `Temurin-28+16-202609171655`. Verified by execution +in this container, not by reading a release note: + +| probe | result | +|---|---| +| `value record LaneId(int)` / `value class` compiled with **no** flag | **rejected** — *"value classes are a preview feature and are disabled by default"* | +| same source with `--release 28 --enable-preview` | compiles | +| `LaneId.class.isValue()` / `MaskId.class.isValue()` | **true** | +| `new LaneId(7) == new LaneId(7)` | **true** (substitutability) | +| `Arena.ofConfined()` + `MemorySegment` get/set, **no** preview flag | works, `0xdeadbeef` round-tripped | +| `Linker.nativeLinker()` | `SysVx64Linker` | + +**So on JDK 28: Panama FFM is FINAL and flag-free; JEP 401 is PRESENT and +preview-gated.** `--enable-preview` is therefore repo-wide under the P0 +mandate, not optional. + +### The production flip is PROVEN, not projected + +Measured the same day, on a scratch copy of `java/src/main` (37 files): + +- **As-is under JDK 28, no preview, no source change: compiles clean.** +- **All six vocabulary types flip with one word** — `public record X` → + `public value record X` for `LaneId`, `MaskId`, `Ordinal`, `FacetId`, + `RowRange`, `WideFieldMask` — and the whole tree compiles under + `--release 28 --enable-preview`. +- At runtime **all six report `isValue() == true`**, and substitutability + holds. This is the lab's one-word claim, now demonstrated on the + PRODUCTION sources rather than on the lab's parallel vocabulary. + +### How it was obtained (the route matters — most are blocked) + +The egress gateway answers **403 to CONNECT** for `api.adoptium.net`, +`packages.adoptium.net`, `repos.azul.com`, `apt.corretto.aws`, +`jdk.java.net` and the launchpad PPAs, so **apt sources and the Adoptium API +cannot reach a JDK here**; Ubuntu noble's own repos stop at `openjdk-25-jdk` +and pypi `jdk4py` at 25.0.2.1. What DOES pass: **anonymous `git ls-remote`** +(gave the tag list, newest `jdk-28+16-ea-beta`) and **GitHub release +DOWNLOAD paths** (`github.com/.../releases/download/...` → 404 for a wrong +name, i.e. passed through; `releases.atom` and `expanded_assets` → 403). +The EA asset name **drops the `28U`**: + +``` +https://github.com/adoptium/temurin28-binaries/releases/download/jdk-28%2B16-ea-beta/OpenJDK-jdk_x64_linux_hotspot_28_16-ea.tar.gz +``` + +A future session that needs a JDK here uses that route, not apt. + ## JDKs available locally | Path | Version | FFM (`java.lang.foreign`) | JEP 401 value classes | Vector API | @@ -36,10 +86,22 @@ and ran on `/opt/jdks/jdk-27` with `--enable-preview`, and ## Decision this locks in -- **Production path (`java/`) targets `/opt/jdks/jdk-26.0.2`.** No preview - flags in the shipped build. This is a real, deliberate strength of the - design: the FFM membrane runs on a *shipped GA JDK*, not an experimental - one. +> ⊘ **SUPERSEDED 2026-09-19 by operator ruling — see `CLAUDE.md` § P0 "JDK 28, +> Valhalla AND Panama".** *LGJ MUST use JDK 28 and MUST use Valhalla and +> Panama.* The first bullet below is struck: targeting a GA JDK without +> preview is a release constraint, and calling it "a real, deliberate +> strength" turned it into an architectural claim that later licensed the +> false sentence *"production lgj does not depend on Valhalla at all."* The +> MEASUREMENTS above are untouched and still valid (they are what this doc is +> for); only the decision they were used to justify is replaced. The paths +> below are also container-specific: `/opt/jdks` does not exist in every +> environment — re-verify before trusting, as this doc's own header says. + +- ⊘ **STRUCK — Production path (`java/`) targets `/opt/jdks/jdk-26.0.2`.** No + preview flags in the shipped build. This is a real, deliberate strength of + the design: the FFM membrane runs on a *shipped GA JDK*, not an + experimental one. **Replaced by: the production path targets JDK 28 with + Valhalla value classes and Panama FFM, both mandatory.** - **Valhalla lab (`valhalla-lab/`) targets `/opt/jdks/jdk-27`.** Same source shape, compiled twice (once as `record`, once as `value record`), so the A/B is genuinely apples-to-apples. diff --git a/CLAUDE.md b/CLAUDE.md index 3c24bea..708afa1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -19,6 +19,59 @@ FieldMask / WideFieldMask); the full lance-graph ENGINE is never a dependency — blast-radius containment is the point of the contract split, and it does not make the contract optional. +## P0 — JDK 28, Valhalla AND Panama (operator-ruled, 2026-09-19) + +**LGJ MUST use JDK 28 and MUST use Valhalla and Panama.** Not a preference, +not a lab arm, not "when it finalizes" — the target toolchain of this repo. + +⊘ **This SUPERSEDES the "production targets a shipped GA JDK, no preview +flags" decision** recorded in `.claude/knowledge/jdk-toolchain-facts.md` +(struck there in place, not deleted). That decision framed a release +constraint as an architectural virtue, and the framing was the drift: it let +a session write *"production lgj does not depend on Valhalla at all"* — which +is false about the design and now false about the mandate. Production has +always been Valhalla-SHAPED (the four vocabulary types are written +identity-free so the same source is a `value record` with one word changed, +proven in `valhalla-lab/docs/three-truths.md`); the ruling makes that +dependency real instead of aspirational. + +**The two membranes, both required, neither optional:** + +| membrane | carries | what it is here | +|---|---|---| +| **Panama** (FFM) | the VERB — operations across the boundary, `MemorySegment` reach into lance-graph's canonical bytes | the computation membrane | +| **Valhalla** (JEP 401 value classes) | the NOUN — `LaneId` / `MaskId` / `Ordinal` / `FacetId` / `RowRange` / `WideFieldMask` as flattened, identity-free semantic addresses | the storage membrane | + +Neither owns storage: **lance-graph owns the only canonical copy.** Valhalla +is the storage membrane, never the storage owner. Together they are what lets +zero-copy be a first-class programming model in Java rather than a trick +hidden behind an FFI call — a view cheap enough to be ergonomically +indistinguishable from owning the thing. + +**What does NOT change under this ruling:** + +- **E4 still holds — the Vector API is not a production backend.** JDK 28 + finalizing it does not give Java a backend; Java has no backends. Numeric + kernels stay in `ndarray::simd` behind the ABI (`E1`, `E2`, `E6`). A + finalized Vector API makes the *lab* faster, not `src/main` wider. +- **`--enable-preview` is still classfile-poisoning**, so while JEP 401 is + preview on the target JDK the flag is set repo-wide and deliberately, never + leaked selectively — one toolchain, one flag posture, no mixed classfiles. +- **Every mask-native, zero-copy and membrane rule below is unaffected.** + Value classes change how cheaply a descriptor is carried; they never make a + population crossable. + +**Status: IMPLEMENTED 2026-09-19, gates green.** `/opt/jdks/jdk-28+16` +(Temurin `28+16-ea`). The six vocabulary types — `LaneId`, `MaskId`, +`Ordinal`, `FacetId`, `RowRange`, `WideFieldMask` — are `public value record`, +all report `isValue() == true` at runtime, and the full suite is **409 checks, +0 failures** under `--release 28 --enable-preview` against a freshly built +`liblgj_abi.so` (abi 0.11, `ndarray::simd avx512`) — byte-for-byte the same +409 as the unflipped baseline on the same JDK. `bench/run.sh` carries the pin +and the flag. Measured facts and the obtain route (most JDK hosts are +egress-blocked here; GitHub release *download* paths are not) live in +`.claude/knowledge/jdk-toolchain-facts.md`. + ## The mask-native invariant (operator-ruled, 2026-08-18) **WHERE MAY LOOK LIKE WHERE. IT MUST EXECUTE LIKE MASK.** diff --git a/README.md b/README.md index 16c73aa..e169570 100644 --- a/README.md +++ b/README.md @@ -70,11 +70,13 @@ no `cbindgen`, no `jextract`, no JNI. See `docs/abi.md` §1 and ## Toolchains - Rust: **1.97.1** stable, matching `ndarray`/`lance-graph`'s pin. -- Production JDK: a GA build with `java.lang.foreign` final (no preview - flag needed) — verified against JDK 26 GA. -- Valhalla lab JDK: the official JEP 401 early-access binary from - `jdk.java.net/valhalla/` — a value-classes preview build, kept physically - separate from the production path. +- Production JDK: **JDK 28** (`/opt/jdks/jdk-28+16`). Panama FFM is final + there and needs no flag; Valhalla (JEP 401) is preview-gated and production + uses it, so `--enable-preview` is required — for the value classes, never + for FFM. +- There is no separate "Valhalla JDK" any more. Production **is** the Valhalla + arm; `valhalla-lab/` is now a VM/representation probe suite over the real + production vocabulary rather than a parallel future-vs-present A/B. See `.claude/knowledge/jdk-toolchain-facts.md` for exact verified paths and flags in the reference development environment. diff --git a/bench/gate-run.sh b/bench/gate-run.sh index d5e7abb..7c1a7ab 100755 --- a/bench/gate-run.sh +++ b/bench/gate-run.sh @@ -10,7 +10,7 @@ set -euo pipefail BENCH="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" ROOT="$(cd "$BENCH/.." && pwd)" -JDK="${JDK:-/opt/jdks/jdk-26.0.2}" +JDK="${JDK:-/opt/jdks/jdk-28+16}" # P0: JDK 28 + Valhalla (JEP 401, preview) + Panama FFM LIB_DIR="${LIB_DIR:-$ROOT/target/release}" OUT="$BENCH/out-gate" CP_LIBS="$(find "$BENCH/lib" -name '*.jar' | sort | tr '\n' ':')" @@ -27,10 +27,10 @@ cmp -s "$PROBE" "$VARIANT" && { echo "FAIL: variant is identical to the shipped rm -rf "$OUT"; mkdir -p "$OUT/before" "$OUT/after" "$OUT/bench" "$BENCH/results" echo "== build BEFORE: java/src/main as shipped" -"$JDK/bin/javac" -d "$OUT/before" $(find "$API" -name '*.java') +"$JDK/bin/javac" --release 28 --enable-preview -d "$OUT/before" $(find "$API" -name '*.java') echo "== build AFTER: same tree, LaneProbe.java swapped for bench/variants/probed" -"$JDK/bin/javac" -d "$OUT/after" $(find "$API" -name '*.java' ! -path "$PROBE") "$VARIANT" +"$JDK/bin/javac" --release 28 --enable-preview -d "$OUT/after" $(find "$API" -name '*.java' ! -path "$PROBE") "$VARIANT" # One compiled bench, run against each API build (the bench class is identical by construction). echo "== build the bench once" @@ -41,6 +41,7 @@ run_arm() { local arm="$1" echo "== run $arm" ( cd "$BENCH" && JAVA_TOOL_OPTIONS= "$JDK/bin/java" \ + --enable-preview \ --enable-native-access=ALL-UNNAMED --add-modules jdk.incubator.vector \ -Dstdout.encoding=UTF-8 -Dlgj.library="$LIB_DIR/liblgj_abi.so" \ -Dlgj.bench.result="results/gate-$arm.csv" \ diff --git a/bench/run.sh b/bench/run.sh index ec38195..38df947 100755 --- a/bench/run.sh +++ b/bench/run.sh @@ -10,7 +10,7 @@ set -euo pipefail BENCH="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" ROOT="$(cd "$BENCH/.." && pwd)" -JDK="${JDK:-/opt/jdks/jdk-26.0.2}" +JDK="${JDK:-/opt/jdks/jdk-28+16}" # P0: JDK 28 + Valhalla (JEP 401, preview) + Panama FFM LIB_DIR="${LIB_DIR:-$ROOT/target/release}" OUT="$BENCH/out" @@ -29,13 +29,14 @@ fi echo "== compiling the library API" rm -rf "$OUT"; mkdir -p "$OUT/api" "$OUT/bench" "$BENCH/results" -"$JDK/bin/javac" -d "$OUT/api" $(find "$ROOT/java/src/main/java" -name '*.java') +"$JDK/bin/javac" --release 28 --enable-preview -d "$OUT/api" $(find "$ROOT/java/src/main/java" -name '*.java') echo "== compiling the benchmarks (annotation processing ON: JMH needs it to emit BenchmarkList)" # -proc:full is REQUIRED on JDK 23+: annotation processing is off by default there, and without it # JMH's generator never runs and the harness fails at startup with "Unable to find the resource: # /META-INF/BenchmarkList" — which looks like a classpath problem and is not one. "$JDK/bin/javac" \ + --release 28 --enable-preview \ -proc:full \ --add-modules jdk.incubator.vector \ -cp "$CP_LIBS$OUT/api" \ @@ -45,6 +46,7 @@ echo "== compiling the benchmarks (annotation processing ON: JMH needs it to emi echo "== running" cd "$BENCH" JAVA_TOOL_OPTIONS= "$JDK/bin/java" \ + --enable-preview \ --enable-native-access=ALL-UNNAMED \ --add-modules jdk.incubator.vector \ -Dstdout.encoding=UTF-8 \ diff --git a/docs/panama.md b/docs/panama.md index 3bce268..7f5b953 100644 --- a/docs/panama.md +++ b/docs/panama.md @@ -70,11 +70,17 @@ correctly (`bench/README.md` rule 4). Measured: a bare downcall costs ~22 ns over a plain Java call (`bench/RESULTS.md`, Component A) — the floor every native-side operation in this project sits on top of. -## `--enable-preview` never reaches the shipped path - -The production Java tree (`java/`) targets `/opt/jdks/jdk-26.0.2`, where -FFM is **final** — the only flag needed anywhere is -`--enable-native-access=ALL-UNNAMED`. The Valhalla lab +## `--enable-preview` is Valhalla's flag, never FFM's + +⊘ This section formerly read *"`--enable-preview` never reaches the shipped +path"*, which was true only while production and Valhalla were split arms. +The production Java tree (`java/`) now targets `/opt/jdks/jdk-28+16`, where +**FFM is final and still needs no flag** — `--enable-native-access=ALL-UNNAMED` +remains the only thing Panama asks for. What changed is that production +deliberately uses Valhalla value classes (JEP 401, preview on this build), so +`--release 28 --enable-preview` is now part of the shipped build contract. +Read every "no preview" claim below as scoped to FFM: **Panama needs no +preview; the vocabulary types do.** The Valhalla lab (`valhalla-lab/src/valhalla`) is compiled *separately*, with its own `-source 27 --enable-preview`, into its own output directory (`results/valhalla-lab/`), and is never on the classpath the production diff --git a/java/README.md b/java/README.md index c63cf53..b0c65ff 100644 --- a/java/README.md +++ b/java/README.md @@ -27,9 +27,12 @@ become one native lane set, one packed mask, a few tiny schema descriptors, and There is no Maven, no Gradle, no downloaded dependency, and no C toolchain. `javac` and `java` are the entire Java toolchain, exactly as `cargo` is the entire Rust one. -- **JDK 26** (`/opt/jdks/jdk-26.0.2`). FFM is **final** in JDK 26, so `--enable-preview` is neither - needed nor accepted. Do **not** use a JDK 21 `java` on the path — there FFM is preview-gated and - these commands will not work. +- **JDK 28** (`/opt/jdks/jdk-28+16`). Two features, two reasons: **Panama FFM is final** here, so + it needs no flag; **Valhalla (JEP 401) is preview-gated**, and production deliberately uses it — + the six vocabulary types (`LaneId`, `MaskId`, `Ordinal`, `FacetId`, `RowRange`, `WideFieldMask`) + are `value record`s. So `--release 28 --enable-preview` is required to COMPILE and `--enable-preview` + to RUN. The flag is Valhalla's, never FFM's. Do **not** use a JDK 21 `java` on the path — there FFM + is preview-gated too and these commands will not work. - No JNI, no `jextract`, no `cbindgen`, no `.h` file anywhere in the project. See `../docs/abi.md` §0 for why those are absent by construction rather than by preference. @@ -37,7 +40,7 @@ the entire Java toolchain, exactly as `cargo` is the entire Rust one. ```sh cd java -/opt/jdks/jdk-26.0.2/bin/javac -d out $(find src/main/java src/test/java -name '*.java') +/opt/jdks/jdk-28+16/bin/javac --release 28 --enable-preview -d out $(find src/main/java src/test/java -name '*.java') ``` Compilation emits **seven** `[restricted]` warnings with `-Xlint:all`: six in @@ -56,11 +59,11 @@ exercise the restriction itself. ```sh # everything -/opt/jdks/jdk-26.0.2/bin/java --enable-native-access=ALL-UNNAMED -cp out \ +/opt/jdks/jdk-28+16/bin/java --enable-preview --enable-native-access=ALL-UNNAMED -cp out \ com.adaworldapi.lancegraph.AllTests # or one suite at a time — each has its own main -/opt/jdks/jdk-26.0.2/bin/java --enable-native-access=ALL-UNNAMED -cp out \ +/opt/jdks/jdk-28+16/bin/java --enable-preview --enable-native-access=ALL-UNNAMED -cp out \ com.adaworldapi.lancegraph.SmokeTest ``` @@ -69,7 +72,7 @@ exercise the restriction itself. | Flag | Needed? | Why | |---|---|---| | `--enable-native-access=ALL-UNNAMED` | **yes** | FFM's restricted methods (`libraryLookup`, `downcallHandle`, `reinterpret`) refuse to run without it. Omitting it does not fail the build — it fails at load, which is worse. | -| `--enable-preview` | **no** | FFM is final in JDK 26. Passing it is an error. | +| `--enable-preview` | **yes** | NOT for FFM — FFM is final on JDK 28. It is required because the production vocabulary types are Valhalla `value record`s (JEP 401, preview on this build). | | `-Djava.library.path=…` | **no** | Not used. That is the JNI mechanism; this project resolves the artifact itself (below). | ### Where the native library is found diff --git a/java/src/main/java/com/adaworldapi/lancegraph/FacetId.java b/java/src/main/java/com/adaworldapi/lancegraph/FacetId.java index a0e2019..030208a 100644 --- a/java/src/main/java/com/adaworldapi/lancegraph/FacetId.java +++ b/java/src/main/java/com/adaworldapi/lancegraph/FacetId.java @@ -18,7 +18,7 @@ * * @param index zero-based facet index, {@code 0..32} exclusive (a row has exactly 32 facets) */ -public record FacetId(int index) { +public value record FacetId(int index) { /** Facets per row — 32, the canonical `32 × 16 B = 512 B` row (abi.md §11). */ public static final int COUNT = 32; diff --git a/java/src/main/java/com/adaworldapi/lancegraph/LaneId.java b/java/src/main/java/com/adaworldapi/lancegraph/LaneId.java index b2f481e..5142b1f 100644 --- a/java/src/main/java/com/adaworldapi/lancegraph/LaneId.java +++ b/java/src/main/java/com/adaworldapi/lancegraph/LaneId.java @@ -28,7 +28,7 @@ * * @param index zero-based lane index within its resource */ -public record LaneId(int index) { +public value record LaneId(int index) { public LaneId { if (index < 0) { diff --git a/java/src/main/java/com/adaworldapi/lancegraph/MaskId.java b/java/src/main/java/com/adaworldapi/lancegraph/MaskId.java index 62d598d..c2a16c8 100644 --- a/java/src/main/java/com/adaworldapi/lancegraph/MaskId.java +++ b/java/src/main/java/com/adaworldapi/lancegraph/MaskId.java @@ -20,7 +20,7 @@ * * @param token the opaque registry token */ -public record MaskId(long token) { +public value record MaskId(long token) { /** Which registry slot. Diagnostic only. */ public int slot() { diff --git a/java/src/main/java/com/adaworldapi/lancegraph/Ordinal.java b/java/src/main/java/com/adaworldapi/lancegraph/Ordinal.java index 23476b9..fa2cd8d 100644 --- a/java/src/main/java/com/adaworldapi/lancegraph/Ordinal.java +++ b/java/src/main/java/com/adaworldapi/lancegraph/Ordinal.java @@ -16,7 +16,7 @@ * * @param value zero-based position */ -public record Ordinal(int value) { +public value record Ordinal(int value) { public Ordinal { if (value < 0) { diff --git a/java/src/main/java/com/adaworldapi/lancegraph/RowRange.java b/java/src/main/java/com/adaworldapi/lancegraph/RowRange.java index 963f0c3..86b95e1 100644 --- a/java/src/main/java/com/adaworldapi/lancegraph/RowRange.java +++ b/java/src/main/java/com/adaworldapi/lancegraph/RowRange.java @@ -16,7 +16,7 @@ * @param start first row, inclusive * @param endExclusive one past the last row */ -public record RowRange(long start, long endExclusive) { +public value record RowRange(long start, long endExclusive) { public RowRange { if (start < 0) { diff --git a/java/src/main/java/com/adaworldapi/lancegraph/WideFieldMask.java b/java/src/main/java/com/adaworldapi/lancegraph/WideFieldMask.java index 5f864a5..7272c0c 100644 --- a/java/src/main/java/com/adaworldapi/lancegraph/WideFieldMask.java +++ b/java/src/main/java/com/adaworldapi/lancegraph/WideFieldMask.java @@ -31,7 +31,7 @@ * when their bits are equal, never because they are the same object — nothing here relies on * reference equality, so flattening changes no observable behaviour. */ -public record WideFieldMask(long value) { +public value record WideFieldMask(long value) { private static final int FACET_COUNT = 32; diff --git a/native/lgj-abi/Cargo.lock b/native/lgj-abi/Cargo.lock index 37977ba..a77cad2 100644 --- a/native/lgj-abi/Cargo.lock +++ b/native/lgj-abi/Cargo.lock @@ -62,6 +62,7 @@ dependencies = [ name = "lance-graph-quack" version = "0.1.0" dependencies = [ + "lance-graph-contract", "lance-graph-mask-risc", ] diff --git a/native/lgj-abi/src/exports.rs b/native/lgj-abi/src/exports.rs index 59e7c93..14bcdbe 100644 --- a/native/lgj-abi/src/exports.rs +++ b/native/lgj-abi/src/exports.rs @@ -1720,9 +1720,10 @@ fn validate_plan(pattern: &ResourceEntry, ops: &[LgjOpDesc]) -> Result<(), i32> /// more than implying otherwise. `validate_plan` runs first and rejects an /// unknown opcode, a bad combine, an out-of-range lane and a kind mismatch /// before the lowering is even built; the lowering names no input plane -/// (`Planes::masks` is `&[]`), no sum terminal and no blend. What is left — -/// the scratch-sizing family, `ScratchReadBeforeWrite`, `GateAliasesDst` — -/// would be a bug in THIS file, not in a caller's plan. +/// (`Planes::masks` is `&[]`), no sum terminal, no blend and no `Pred::Range`. +/// What is left — the scratch-sizing family, `ScratchReadBeforeWrite`, +/// `GateAliasesDst`, `RangeOutOfBounds` — would be a bug in THIS file, not in +/// a caller's plan. /// /// So the map exists to turn such a bug into a status a caller can see /// instead of a panic, and its arms are deliberately NOT claimed to be @@ -1742,7 +1743,8 @@ fn exec_error_to_status(e: ExecError) -> i32 { | ExecError::ScratchReadBeforeWrite { .. } | ExecError::ScratchWords { .. } | ExecError::BlendNeedsOut - | ExecError::GateAliasesDst { .. } => LGJ_ERR_ALLOCATION_FAILED, + | ExecError::GateAliasesDst { .. } + | ExecError::RangeOutOfBounds { .. } => LGJ_ERR_ALLOCATION_FAILED, } } diff --git a/native/lgj-abi/src/plan_lower.rs b/native/lgj-abi/src/plan_lower.rs index 0518ff8..91607da 100644 --- a/native/lgj-abi/src/plan_lower.rs +++ b/native/lgj-abi/src/plan_lower.rs @@ -158,3 +158,62 @@ fn pred_of(op: &LgjOpDesc) -> Option { _ => return None, }) } + +#[cfg(test)] +mod range_falsifier { + use super::*; + use crate::abi::LGJ_COMBINE_OR; + + /// **The falsifier behind `exec_error_to_status`'s `RangeOutOfBounds` arm.** + /// + /// That arm maps a `mask_risc` error into the documented *"would be a bug + /// in THIS file, not in a caller's plan"* family, and the justification is + /// one sentence: **this lowering emits no `Pred::Range`**, exactly as it + /// emits no sum terminal and no blend. A sentence is not a fence, so this + /// sweeps every opcode a caller can put in an `LgjOpDesc` — the nine + /// defined ones and a band of undefined ones either side — through the + /// real `lower_plan` and reads every emitted predicate. + /// + /// **When LGJ gains a Range lowering this test goes RED**, and that is its + /// whole purpose: it forces the author to decide, deliberately and + /// visibly, what a caller should see when a range leaves the lane — rather + /// than inheriting `LGJ_ERR_ALLOCATION_FAILED`, which would then be a lie. + #[test] + fn no_opcode_lowers_to_pred_range() { + let mut lowered_count = 0usize; + for op in 0u32..=32 { + for combine in [LGJ_COMBINE_AND, LGJ_COMBINE_OR] { + let desc = LgjOpDesc { + op, + lane_id: 0, + operand: -1, + combine, + _reserved: 0, + }; + let Some(lowered) = lower_plan(&[desc]) else { + continue; + }; + let Lowered::Program(program) = lowered else { + continue; // AllRows: no predicate at all. + }; + lowered_count += 1; + for mask_op in &program.ops { + if let MaskOp::Pred { pred, .. } = mask_op { + assert!( + !matches!(pred, Pred::Range { .. }), + "opcode {op} lowered to Pred::Range — `exec_error_to_status` maps \ + `RangeOutOfBounds` as an internal bug, which is now false: give it \ + a caller-visible status before shipping the Range lowering" + ); + } + } + } + } + // Anti-vacuity: a sweep that lowers nothing proves nothing. Nine + // opcodes are defined and each lowers under `LGJ_COMBINE_AND`. + assert!( + lowered_count >= 9, + "the sweep lowered only {lowered_count} programs — it is not exercising the lowering" + ); + } +} diff --git a/valhalla-lab/results/AB-default.diff b/valhalla-lab/results/AB-default.diff index 57b0910..db28dcd 100644 --- a/valhalla-lab/results/AB-default.diff +++ b/valhalla-lab/results/AB-default.diff @@ -1,13 +1,11 @@ -3,4c3,4 +3c3 < platform stable -< java.vm.version 26.0.2+10-55 --- > platform valhalla -> java.vm.version 27-jep401ea3+1-1 6c6 -< jvm args [--enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so] +< jvm args [--enable-preview, --enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so] --- -> jvm args [--enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED] +> jvm args [--enable-preview, --enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED] 9,14c9,14 < platform stable — stable-record vocabulary; arrays are reference arrays; fields are references < LaneId.class.isValue() false @@ -59,74 +57,72 @@ 40,46c40,46 < construct N LaneId, store into array 15.26 MiB < ... per LaneId 16.00 B -< construct N LaneId, never escaping 7.22 MiB -< ... per LaneId 7.57 B +< construct N LaneId, never escaping 4.73 MiB +< ... per LaneId 4.96 B < LaneId[1024] flatness UNKNOWN(no ValueClass API on a stable JDK; a reference array is never flat) < allocate+fill LaneId[N] (array + elements) 19.07 MiB < ... per element 20.00 B --- -> construct N LaneId, store into array 3.14 MiB -> ... per LaneId 3.29 B -> construct N LaneId, never escaping 7.29 MiB -> ... per LaneId 7.64 B +> construct N LaneId, store into array 1.87 MiB +> ... per LaneId 1.96 B +> construct N LaneId, never escaping 5.75 MiB +> ... per LaneId 6.03 B > LaneId[1024] flatness FLAT -> allocate+fill LaneId[N] (array + elements) 6.56 MiB -> ... per element 6.87 B +> allocate+fill LaneId[N] (array + elements) 5.65 MiB +> ... per element 5.93 B 49,55c49,55 < Descriptor kind identity class with two reference fields < Descriptor fields null-restricted false -< construct N Descriptor (2 wrappers each) 53.41 MiB -< ... per Descriptor 56.00 B -< pass 2 wrappers through 3 call levels 8.90 MiB -< ... per call 9.33 B -< read 65,536 LaneId from array median= 43739.0 ns [min 42371.0 .. max 73206.0] n=51 +< construct N Descriptor (2 wrappers each) 45.78 MiB +< ... per Descriptor 48.00 B +< pass 2 wrappers through 3 call levels 7.49 MiB +< ... per call 7.86 B +< read 65,536 LaneId from array median= 22646.0 ns [min 22429.0 .. max 45634.0] n=51 --- > Descriptor kind value class with two @NullRestricted value fields > Descriptor fields null-restricted true -> construct N Descriptor (2 wrappers each) 34.81 MiB -> ... per Descriptor 36.50 B -> pass 2 wrappers through 3 call levels 10.36 MiB -> ... per call 10.87 B -> read 65,536 LaneId from array median= 5339.0 ns [min 5262.0 .. max 9259.0] n=51 +> construct N Descriptor (2 wrappers each) 27.89 MiB +> ... per Descriptor 29.25 B +> pass 2 wrappers through 3 call levels 7.42 MiB +> ... per call 7.79 B +> read 65,536 LaneId from array median= 4341.0 ns [min 4285.0 .. max 16364.0] n=51 59c59 < platform stable --- > platform valhalla -61,65c61,65 -< bare long index median= 59836.0 ns [min 59076.0 .. max 125353.0] n=51 -< RowRange bounds (wrapper hoisted) median= 47964.0 ns [min 47876.0 .. max 168118.0] n=51 -< per-element wrapper: bytes allocated 1.00 MiB -< ... per element 16.00 B -< Ordinal built per element median= 50732.0 ns [min 50046.0 .. max 99100.0] n=51 ---- -> bare long index median= 59134.0 ns [min 59058.0 .. max 122610.0] n=51 -> RowRange bounds (wrapper hoisted) median= 56975.0 ns [min 56642.0 .. max 112526.0] n=51 -> per-element wrapper: bytes allocated 1.50 MiB -> ... per element 24.00 B -> Ordinal built per element median= 49851.0 ns [min 49725.0 .. max 70877.0] n=51 +61,62c61,62 +< bare long index median= 45400.0 ns [min 45326.0 .. max 110395.0] n=51 +< RowRange bounds (wrapper hoisted) median= 41200.0 ns [min 41067.0 .. max 61470.0] n=51 +--- +> bare long index median= 46793.0 ns [min 46730.0 .. max 117525.0] n=51 +> RowRange bounds (wrapper hoisted) median= 41442.0 ns [min 41020.0 .. max 63249.0] n=51 +65c65 +< Ordinal built per element median= 41757.0 ns [min 41726.0 .. max 67673.0] n=51 +--- +> Ordinal built per element median= 67219.0 ns [min 62386.0 .. max 94391.0] n=51 68c68 < platform stable --- > platform valhalla 79,83c79,83 -< (2)/(3) hydrate 65536 Row — allocated 2.00 MiB -< ... per row 32.00 B +< (2)/(3) hydrate 65536 Row — allocated 1.50 MiB +< ... per row 24.00 B < array flatness UNKNOWN(no ValueClass API on a stable JDK; a reference array is never flat) -< ratio vs native lane bytes 2.00x -< retained heap (APPROX, gc-delta) 2.25 MiB +< ratio vs native lane bytes 1.50x +< retained heap (APPROX, gc-delta) 1.75 MiB --- -> (2)/(3) hydrate 65536 Row — allocated 2.50 MiB -> ... per row 40.00 B +> (2)/(3) hydrate 65536 Row — allocated 2.00 MiB +> ... per row 32.01 B > array flatness NOT-FLAT -> ratio vs native lane bytes 2.50x -> retained heap (APPROX, gc-delta) 2.75 MiB +> ratio vs native lane bytes 2.00x +> retained heap (APPROX, gc-delta) 2.25 MiB 86,89c86,89 -< (1) native one crossing, fused plan median= 16040.0 ns [min 15900.0 .. max 31577.0] n=51 -< (2/3) hydrate 65536 Row objects median= 1612494.0 ns [min 499222.0 .. max 4777342.0] n=51 -< (2/3) scan the materialised objects median= 128057.0 ns [min 112482.0 .. max 227766.0] n=51 -< (2/3) hydrate THEN scan (honest total) median= 704743.0 ns [min 631780.0 .. max 974235.0] n=51 ---- -> (1) native one crossing, fused plan median= 15162.0 ns [min 14976.0 .. max 55083.0] n=51 -> (2/3) hydrate 65536 Row objects median= 658842.0 ns [min 594094.0 .. max 1085753.0] n=51 -> (2/3) scan the materialised objects median= 87213.0 ns [min 83223.0 .. max 135141.0] n=51 -> (2/3) hydrate THEN scan (honest total) median= 871361.0 ns [min 787060.0 .. max 1483819.0] n=51 +< (1) native one crossing, fused plan median= 18369.0 ns [min 18120.0 .. max 133725.0] n=51 +< (2/3) hydrate 65536 Row objects median= 578773.0 ns [min 518142.0 .. max 793522.0] n=51 +< (2/3) scan the materialised objects median= 64233.0 ns [min 61462.0 .. max 102727.0] n=51 +< (2/3) hydrate THEN scan (honest total) median= 1099547.0 ns [min 1020952.0 .. max 1236355.0] n=51 +--- +> (1) native one crossing, fused plan median= 18085.0 ns [min 17752.0 .. max 40869.0] n=51 +> (2/3) hydrate 65536 Row objects median= 702484.0 ns [min 574460.0 .. max 4737765.0] n=51 +> (2/3) scan the materialised objects median= 77463.0 ns [min 72937.0 .. max 179252.0] n=51 +> (2/3) hydrate THEN scan (honest total) median= 860090.0 ns [min 773234.0 .. max 1249974.0] n=51 diff --git a/valhalla-lab/results/stable-api-javac.log b/valhalla-lab/results/stable-api-javac.log index c298bca..10bdfc3 100644 --- a/valhalla-lab/results/stable-api-javac.log +++ b/valhalla-lab/results/stable-api-javac.log @@ -1 +1,3 @@ -Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=/root/.ccr/java-truststore.p12 -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=PKCS12 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=34795 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|anthropic.com|*.anthropic.com|*.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes= +Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=/root/.ccr/java-truststore.p12 -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=PKCS12 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42017 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes= +Note: Some input files use preview features of Java SE 28. +Note: Recompile with -Xlint:preview for details. diff --git a/valhalla-lab/results/stable-default.txt b/valhalla-lab/results/stable-default.txt index 5d91d9c..dbd605d 100644 --- a/valhalla-lab/results/stable-default.txt +++ b/valhalla-lab/results/stable-default.txt @@ -1,9 +1,9 @@ Picked up JAVA_TOOL_OPTIONS: lance-graph-java :: valhalla lab platform stable -java.vm.version 26.0.2+10-55 -java.vendor.version - -jvm args [--enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so] +java.vm.version 28-beta+16-ea +java.vendor.version Temurin-28+16-202609171655 +jvm args [--enable-preview, --enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so] == (a) SEMANTIC TRUTH — is identity observable? ============================== platform stable — stable-record vocabulary; arrays are reference arrays; fields are references @@ -39,8 +39,8 @@ allocation instrument baseline 0 B N (operations per measurement) 1000000 construct N LaneId, store into array 15.26 MiB ... per LaneId 16.00 B -construct N LaneId, never escaping 7.22 MiB - ... per LaneId 7.57 B +construct N LaneId, never escaping 4.73 MiB + ... per LaneId 4.96 B LaneId[1024] flatness UNKNOWN(no ValueClass API on a stable JDK; a reference array is never flat) allocate+fill LaneId[N] (array + elements) 19.07 MiB ... per element 20.00 B @@ -48,21 +48,21 @@ bare LaneId[N] with no elements stored 3.81 MiB ... per slot 4.00 B Descriptor kind identity class with two reference fields Descriptor fields null-restricted false -construct N Descriptor (2 wrappers each) 53.41 MiB - ... per Descriptor 56.00 B -pass 2 wrappers through 3 call levels 8.90 MiB - ... per call 9.33 B -read 65,536 LaneId from array median= 43739.0 ns [min 42371.0 .. max 73206.0] n=51 -native runtime lance-graph native runtime: abi 0.1, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so +construct N Descriptor (2 wrappers each) 45.78 MiB + ... per Descriptor 48.00 B +pass 2 wrappers through 3 call levels 7.49 MiB + ... per call 7.86 B +read 65,536 LaneId from array median= 22646.0 ns [min 22429.0 .. max 45634.0] n=51 +native runtime lance-graph native runtime: abi 0.11, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so == FFM ADDRESSING — is the wrapper free where it touches native memory? ====== platform stable sum (identical across all three) 6929623 -bare long index median= 59836.0 ns [min 59076.0 .. max 125353.0] n=51 -RowRange bounds (wrapper hoisted) median= 47964.0 ns [min 47876.0 .. max 168118.0] n=51 +bare long index median= 45400.0 ns [min 45326.0 .. max 110395.0] n=51 +RowRange bounds (wrapper hoisted) median= 41200.0 ns [min 41067.0 .. max 61470.0] n=51 per-element wrapper: bytes allocated 1.00 MiB ... per element 16.00 B -Ordinal built per element median= 50732.0 ns [min 50046.0 .. max 99100.0] n=51 +Ordinal built per element median= 41757.0 ns [min 41726.0 .. max 67673.0] n=51 == THE THESIS — 65,536 entities, three representations ======================= platform stable @@ -72,20 +72,20 @@ answer (identical across all paths) 2173 rows, sum 499246 selectivity 3.32% == heap cost =============================================================== -(1) native — Java bytes allocated (warm) 816 B per query, for the fluent chain itself +(1) native — Java bytes allocated (warm) 712 B per query, for the fluent chain itself (1) native — Java objects per row 0 (1) native — native lane bytes 1.00 MiB (u64 id + u32 class + i32 value) (1) native — mask bytes 8.0 KiB (1 bit per row, packed) -(2)/(3) hydrate 65536 Row — allocated 2.00 MiB - ... per row 32.00 B +(2)/(3) hydrate 65536 Row — allocated 1.50 MiB + ... per row 24.00 B array flatness UNKNOWN(no ValueClass API on a stable JDK; a reference array is never flat) - ratio vs native lane bytes 2.00x - retained heap (APPROX, gc-delta) 2.25 MiB + ratio vs native lane bytes 1.50x + retained heap (APPROX, gc-delta) 1.75 MiB == time to answer the question ============================================= -(1) native one crossing, fused plan median= 16040.0 ns [min 15900.0 .. max 31577.0] n=51 -(2/3) hydrate 65536 Row objects median= 1612494.0 ns [min 499222.0 .. max 4777342.0] n=51 -(2/3) scan the materialised objects median= 128057.0 ns [min 112482.0 .. max 227766.0] n=51 -(2/3) hydrate THEN scan (honest total) median= 704743.0 ns [min 631780.0 .. max 974235.0] n=51 +(1) native one crossing, fused plan median= 18369.0 ns [min 18120.0 .. max 133725.0] n=51 +(2/3) hydrate 65536 Row objects median= 578773.0 ns [min 518142.0 .. max 793522.0] n=51 +(2/3) scan the materialised objects median= 64233.0 ns [min 61462.0 .. max 102727.0] n=51 +(2/3) hydrate THEN scan (honest total) median= 1099547.0 ns [min 1020952.0 .. max 1236355.0] n=51 lab complete. diff --git a/valhalla-lab/results/stable-lab-javac.log b/valhalla-lab/results/stable-lab-javac.log index c298bca..770a5de 100644 --- a/valhalla-lab/results/stable-lab-javac.log +++ b/valhalla-lab/results/stable-lab-javac.log @@ -1 +1 @@ -Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=/root/.ccr/java-truststore.p12 -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=PKCS12 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=34795 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|anthropic.com|*.anthropic.com|*.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes= +Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=/root/.ccr/java-truststore.p12 -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=PKCS12 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42017 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes= diff --git a/valhalla-lab/results/stable-noea.txt b/valhalla-lab/results/stable-noea.txt index 9a62f99..743a473 100644 --- a/valhalla-lab/results/stable-noea.txt +++ b/valhalla-lab/results/stable-noea.txt @@ -1,9 +1,9 @@ Picked up JAVA_TOOL_OPTIONS: lance-graph-java :: valhalla lab platform stable -java.vm.version 26.0.2+10-55 -java.vendor.version - -jvm args [--enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, -XX:-DoEscapeAnalysis] +java.vm.version 28-beta+16-ea +java.vendor.version Temurin-28+16-202609171655 +jvm args [--enable-preview, --enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, -XX:-DoEscapeAnalysis] == (a) SEMANTIC TRUTH — is identity observable? ============================== platform stable — stable-record vocabulary; arrays are reference arrays; fields are references @@ -48,21 +48,21 @@ bare LaneId[N] with no elements stored 3.81 MiB ... per slot 4.00 B Descriptor kind identity class with two reference fields Descriptor fields null-restricted false -construct N Descriptor (2 wrappers each) 53.41 MiB - ... per Descriptor 56.00 B +construct N Descriptor (2 wrappers each) 45.78 MiB + ... per Descriptor 48.00 B pass 2 wrappers through 3 call levels 30.52 MiB ... per call 32.00 B -read 65,536 LaneId from array median= 39359.0 ns [min 39266.0 .. max 63844.0] n=51 -native runtime lance-graph native runtime: abi 0.1, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so +read 65,536 LaneId from array median= 29321.0 ns [min 25797.0 .. max 65762.0] n=51 +native runtime lance-graph native runtime: abi 0.11, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so == FFM ADDRESSING — is the wrapper free where it touches native memory? ====== platform stable sum (identical across all three) 6929623 -bare long index median= 59174.0 ns [min 59067.0 .. max 117139.0] n=51 -RowRange bounds (wrapper hoisted) median= 47949.0 ns [min 47870.0 .. max 61164.0] n=51 +bare long index median= 42889.0 ns [min 41754.0 .. max 106275.0] n=51 +RowRange bounds (wrapper hoisted) median= 37762.0 ns [min 37677.0 .. max 65634.0] n=51 per-element wrapper: bytes allocated 1.00 MiB ... per element 16.00 B -Ordinal built per element median= 621827.0 ns [min 584504.0 .. max 873212.0] n=51 +Ordinal built per element median= 223618.0 ns [min 203093.0 .. max 284493.0] n=51 == THE THESIS — 65,536 entities, three representations ======================= platform stable @@ -72,20 +72,20 @@ answer (identical across all paths) 2173 rows, sum 499246 selectivity 3.32% == heap cost =============================================================== -(1) native — Java bytes allocated (warm) 816 B per query, for the fluent chain itself +(1) native — Java bytes allocated (warm) 712 B per query, for the fluent chain itself (1) native — Java objects per row 0 (1) native — native lane bytes 1.00 MiB (u64 id + u32 class + i32 value) (1) native — mask bytes 8.0 KiB (1 bit per row, packed) -(2)/(3) hydrate 65536 Row — allocated 2.00 MiB - ... per row 32.00 B +(2)/(3) hydrate 65536 Row — allocated 1.50 MiB + ... per row 24.00 B array flatness UNKNOWN(no ValueClass API on a stable JDK; a reference array is never flat) - ratio vs native lane bytes 2.00x - retained heap (APPROX, gc-delta) 2.22 MiB + ratio vs native lane bytes 1.50x + retained heap (APPROX, gc-delta) 1.75 MiB == time to answer the question ============================================= -(1) native one crossing, fused plan median= 15431.0 ns [min 15368.0 .. max 28346.0] n=51 -(2/3) hydrate 65536 Row objects median= 537476.0 ns [min 478467.0 .. max 4136743.0] n=51 -(2/3) scan the materialised objects median= 118276.0 ns [min 108392.0 .. max 199142.0] n=51 -(2/3) hydrate THEN scan (honest total) median= 805562.0 ns [min 689612.0 .. max 1107352.0] n=51 +(1) native one crossing, fused plan median= 16947.0 ns [min 16734.0 .. max 30203.0] n=51 +(2/3) hydrate 65536 Row objects median= 512948.0 ns [min 466116.0 .. max 592150.0] n=51 +(2/3) scan the materialised objects median= 53677.0 ns [min 49770.0 .. max 92752.0] n=51 +(2/3) hydrate THEN scan (honest total) median= 670905.0 ns [min 589551.0 .. max 782663.0] n=51 lab complete. diff --git a/valhalla-lab/results/valhalla-api-javac.log b/valhalla-lab/results/valhalla-api-javac.log index c298bca..10bdfc3 100644 --- a/valhalla-lab/results/valhalla-api-javac.log +++ b/valhalla-lab/results/valhalla-api-javac.log @@ -1 +1,3 @@ -Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=/root/.ccr/java-truststore.p12 -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=PKCS12 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=34795 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|anthropic.com|*.anthropic.com|*.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes= +Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=/root/.ccr/java-truststore.p12 -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=PKCS12 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42017 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes= +Note: Some input files use preview features of Java SE 28. +Note: Recompile with -Xlint:preview for details. diff --git a/valhalla-lab/results/valhalla-default.txt b/valhalla-lab/results/valhalla-default.txt index 43ebf51..9cd8e18 100644 --- a/valhalla-lab/results/valhalla-default.txt +++ b/valhalla-lab/results/valhalla-default.txt @@ -1,9 +1,9 @@ Picked up JAVA_TOOL_OPTIONS: lance-graph-java :: valhalla lab platform valhalla -java.vm.version 27-jep401ea3+1-1 -java.vendor.version - -jvm args [--enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED] +java.vm.version 28-beta+16-ea +java.vendor.version Temurin-28+16-202609171655 +jvm args [--enable-preview, --enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED] == (a) SEMANTIC TRUTH — is identity observable? ============================== platform valhalla — value-record vocabulary; null-restricted non-atomic arrays; @NullRestricted fields @@ -37,32 +37,32 @@ Row 1 long + 2 int payload=16 B array=NOT-FLAT platform valhalla allocation instrument baseline 0 B N (operations per measurement) 1000000 -construct N LaneId, store into array 3.14 MiB - ... per LaneId 3.29 B -construct N LaneId, never escaping 7.29 MiB - ... per LaneId 7.64 B +construct N LaneId, store into array 1.87 MiB + ... per LaneId 1.96 B +construct N LaneId, never escaping 5.75 MiB + ... per LaneId 6.03 B LaneId[1024] flatness FLAT -allocate+fill LaneId[N] (array + elements) 6.56 MiB - ... per element 6.87 B +allocate+fill LaneId[N] (array + elements) 5.65 MiB + ... per element 5.93 B bare LaneId[N] with no elements stored 3.81 MiB ... per slot 4.00 B Descriptor kind value class with two @NullRestricted value fields Descriptor fields null-restricted true -construct N Descriptor (2 wrappers each) 34.81 MiB - ... per Descriptor 36.50 B -pass 2 wrappers through 3 call levels 10.36 MiB - ... per call 10.87 B -read 65,536 LaneId from array median= 5339.0 ns [min 5262.0 .. max 9259.0] n=51 -native runtime lance-graph native runtime: abi 0.1, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so +construct N Descriptor (2 wrappers each) 27.89 MiB + ... per Descriptor 29.25 B +pass 2 wrappers through 3 call levels 7.42 MiB + ... per call 7.79 B +read 65,536 LaneId from array median= 4341.0 ns [min 4285.0 .. max 16364.0] n=51 +native runtime lance-graph native runtime: abi 0.11, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so == FFM ADDRESSING — is the wrapper free where it touches native memory? ====== platform valhalla sum (identical across all three) 6929623 -bare long index median= 59134.0 ns [min 59058.0 .. max 122610.0] n=51 -RowRange bounds (wrapper hoisted) median= 56975.0 ns [min 56642.0 .. max 112526.0] n=51 -per-element wrapper: bytes allocated 1.50 MiB - ... per element 24.00 B -Ordinal built per element median= 49851.0 ns [min 49725.0 .. max 70877.0] n=51 +bare long index median= 46793.0 ns [min 46730.0 .. max 117525.0] n=51 +RowRange bounds (wrapper hoisted) median= 41442.0 ns [min 41020.0 .. max 63249.0] n=51 +per-element wrapper: bytes allocated 1.00 MiB + ... per element 16.00 B +Ordinal built per element median= 67219.0 ns [min 62386.0 .. max 94391.0] n=51 == THE THESIS — 65,536 entities, three representations ======================= platform valhalla @@ -72,20 +72,20 @@ answer (identical across all paths) 2173 rows, sum 499246 selectivity 3.32% == heap cost =============================================================== -(1) native — Java bytes allocated (warm) 816 B per query, for the fluent chain itself +(1) native — Java bytes allocated (warm) 712 B per query, for the fluent chain itself (1) native — Java objects per row 0 (1) native — native lane bytes 1.00 MiB (u64 id + u32 class + i32 value) (1) native — mask bytes 8.0 KiB (1 bit per row, packed) -(2)/(3) hydrate 65536 Row — allocated 2.50 MiB - ... per row 40.00 B +(2)/(3) hydrate 65536 Row — allocated 2.00 MiB + ... per row 32.01 B array flatness NOT-FLAT - ratio vs native lane bytes 2.50x - retained heap (APPROX, gc-delta) 2.75 MiB + ratio vs native lane bytes 2.00x + retained heap (APPROX, gc-delta) 2.25 MiB == time to answer the question ============================================= -(1) native one crossing, fused plan median= 15162.0 ns [min 14976.0 .. max 55083.0] n=51 -(2/3) hydrate 65536 Row objects median= 658842.0 ns [min 594094.0 .. max 1085753.0] n=51 -(2/3) scan the materialised objects median= 87213.0 ns [min 83223.0 .. max 135141.0] n=51 -(2/3) hydrate THEN scan (honest total) median= 871361.0 ns [min 787060.0 .. max 1483819.0] n=51 +(1) native one crossing, fused plan median= 18085.0 ns [min 17752.0 .. max 40869.0] n=51 +(2/3) hydrate 65536 Row objects median= 702484.0 ns [min 574460.0 .. max 4737765.0] n=51 +(2/3) scan the materialised objects median= 77463.0 ns [min 72937.0 .. max 179252.0] n=51 +(2/3) hydrate THEN scan (honest total) median= 860090.0 ns [min 773234.0 .. max 1249974.0] n=51 lab complete. diff --git a/valhalla-lab/results/valhalla-lab-javac.log b/valhalla-lab/results/valhalla-lab-javac.log index 67055fc..10bdfc3 100644 --- a/valhalla-lab/results/valhalla-lab-javac.log +++ b/valhalla-lab/results/valhalla-lab-javac.log @@ -1,3 +1,3 @@ -Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=/root/.ccr/java-truststore.p12 -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=PKCS12 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=34795 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|anthropic.com|*.anthropic.com|*.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes= -Note: Some input files use preview features of Java SE 27. +Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=/root/.ccr/java-truststore.p12 -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=PKCS12 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42017 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes= +Note: Some input files use preview features of Java SE 28. Note: Recompile with -Xlint:preview for details. diff --git a/valhalla-lab/results/valhalla-noarrayflat.txt b/valhalla-lab/results/valhalla-noarrayflat.txt index be708cb..fc9076f 100644 --- a/valhalla-lab/results/valhalla-noarrayflat.txt +++ b/valhalla-lab/results/valhalla-noarrayflat.txt @@ -1,9 +1,9 @@ Picked up JAVA_TOOL_OPTIONS: lance-graph-java :: valhalla lab platform valhalla -java.vm.version 27-jep401ea3+1-1 -java.vendor.version - -jvm args [--enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED, -XX:+UnlockDiagnosticVMOptions, -XX:-UseArrayFlattening] +java.vm.version 28-beta+16-ea +java.vendor.version Temurin-28+16-202609171655 +jvm args [--enable-preview, --enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED, -XX:+UnlockDiagnosticVMOptions, -XX:-UseArrayFlattening] == (a) SEMANTIC TRUTH — is identity observable? ============================== platform valhalla — value-record vocabulary; null-restricted non-atomic arrays; @NullRestricted fields @@ -37,32 +37,32 @@ Row 1 long + 2 int payload=16 B array=NOT-FLAT platform valhalla allocation instrument baseline 0 B N (operations per measurement) 1000000 -construct N LaneId, store into array 22.89 MiB - ... per LaneId 24.00 B -construct N LaneId, never escaping 7.26 MiB - ... per LaneId 7.61 B +construct N LaneId, store into array 15.26 MiB + ... per LaneId 16.00 B +construct N LaneId, never escaping 4.73 MiB + ... per LaneId 4.96 B LaneId[1024] flatness NOT-FLAT -allocate+fill LaneId[N] (array + elements) 26.70 MiB - ... per element 28.00 B +allocate+fill LaneId[N] (array + elements) 19.07 MiB + ... per element 20.00 B bare LaneId[N] with no elements stored 3.81 MiB ... per slot 4.00 B Descriptor kind value class with two @NullRestricted value fields Descriptor fields null-restricted true -construct N Descriptor (2 wrappers each) 39.11 MiB - ... per Descriptor 41.01 B -pass 2 wrappers through 3 call levels 9.61 MiB - ... per call 10.07 B -read 65,536 LaneId from array median= 47245.0 ns [min 46645.0 .. max 80026.0] n=51 -native runtime lance-graph native runtime: abi 0.1, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so +construct N Descriptor (2 wrappers each) 27.41 MiB + ... per Descriptor 28.74 B +pass 2 wrappers through 3 call levels 9.92 MiB + ... per call 10.40 B +read 65,536 LaneId from array median= 44895.0 ns [min 42566.0 .. max 71406.0] n=51 +native runtime lance-graph native runtime: abi 0.11, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so == FFM ADDRESSING — is the wrapper free where it touches native memory? ====== platform valhalla sum (identical across all three) 6929623 -bare long index median= 59875.0 ns [min 59085.0 .. max 132585.0] n=51 -RowRange bounds (wrapper hoisted) median= 56943.0 ns [min 56683.0 .. max 82939.0] n=51 -per-element wrapper: bytes allocated 1.50 MiB - ... per element 24.00 B -Ordinal built per element median= 49934.0 ns [min 49758.0 .. max 94648.0] n=51 +bare long index median= 48078.0 ns [min 46712.0 .. max 174025.0] n=51 +RowRange bounds (wrapper hoisted) median= 41241.0 ns [min 41035.0 .. max 62843.0] n=51 +per-element wrapper: bytes allocated 1.00 MiB + ... per element 16.00 B +Ordinal built per element median= 41905.0 ns [min 41727.0 .. max 55960.0] n=51 == THE THESIS — 65,536 entities, three representations ======================= platform valhalla @@ -72,20 +72,20 @@ answer (identical across all paths) 2173 rows, sum 499246 selectivity 3.32% == heap cost =============================================================== -(1) native — Java bytes allocated (warm) 816 B per query, for the fluent chain itself +(1) native — Java bytes allocated (warm) 712 B per query, for the fluent chain itself (1) native — Java objects per row 0 (1) native — native lane bytes 1.00 MiB (u64 id + u32 class + i32 value) (1) native — mask bytes 8.0 KiB (1 bit per row, packed) -(2)/(3) hydrate 65536 Row — allocated 2.50 MiB - ... per row 40.01 B +(2)/(3) hydrate 65536 Row — allocated 2.00 MiB + ... per row 32.01 B array flatness NOT-FLAT - ratio vs native lane bytes 2.50x - retained heap (APPROX, gc-delta) 2.75 MiB + ratio vs native lane bytes 2.00x + retained heap (APPROX, gc-delta) 2.22 MiB == time to answer the question ============================================= -(1) native one crossing, fused plan median= 16317.0 ns [min 15502.0 .. max 36120.0] n=51 -(2/3) hydrate 65536 Row objects median= 664702.0 ns [min 597322.0 .. max 1009820.0] n=51 -(2/3) scan the materialised objects median= 152176.0 ns [min 115700.0 .. max 238478.0] n=51 -(2/3) hydrate THEN scan (honest total) median= 1983973.0 ns [min 828040.0 .. max 6143362.0] n=51 +(1) native one crossing, fused plan median= 19471.0 ns [min 19124.0 .. max 48424.0] n=51 +(2/3) hydrate 65536 Row objects median= 641705.0 ns [min 545163.0 .. max 1332501.0] n=51 +(2/3) scan the materialised objects median= 79088.0 ns [min 73925.0 .. max 152945.0] n=51 +(2/3) hydrate THEN scan (honest total) median= 845990.0 ns [min 740102.0 .. max 3926698.0] n=51 lab complete. diff --git a/valhalla-lab/results/valhalla-noea.txt b/valhalla-lab/results/valhalla-noea.txt index 79574fa..ba3bd82 100644 --- a/valhalla-lab/results/valhalla-noea.txt +++ b/valhalla-lab/results/valhalla-noea.txt @@ -1,9 +1,9 @@ Picked up JAVA_TOOL_OPTIONS: lance-graph-java :: valhalla lab platform valhalla -java.vm.version 27-jep401ea3+1-1 -java.vendor.version - -jvm args [--enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED, -XX:-DoEscapeAnalysis] +java.vm.version 28-beta+16-ea +java.vendor.version Temurin-28+16-202609171655 +jvm args [--enable-preview, --enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED, -XX:-DoEscapeAnalysis] == (a) SEMANTIC TRUTH — is identity observable? ============================== platform valhalla — value-record vocabulary; null-restricted non-atomic arrays; @NullRestricted fields @@ -37,32 +37,32 @@ Row 1 long + 2 int payload=16 B array=NOT-FLAT platform valhalla allocation instrument baseline 0 B N (operations per measurement) 1000000 -construct N LaneId, store into array 3.90 MiB - ... per LaneId 4.09 B -construct N LaneId, never escaping 24.41 MiB - ... per LaneId 25.60 B +construct N LaneId, store into array 2.49 MiB + ... per LaneId 2.61 B +construct N LaneId, never escaping 16.36 MiB + ... per LaneId 17.15 B LaneId[1024] flatness FLAT -allocate+fill LaneId[N] (array + elements) 28.35 MiB - ... per element 29.73 B +allocate+fill LaneId[N] (array + elements) 20.21 MiB + ... per element 21.19 B bare LaneId[N] with no elements stored 3.81 MiB ... per slot 4.00 B Descriptor kind value class with two @NullRestricted value fields Descriptor fields null-restricted true -construct N Descriptor (2 wrappers each) 38.70 MiB - ... per Descriptor 40.58 B -pass 2 wrappers through 3 call levels 29.47 MiB - ... per call 30.90 B -read 65,536 LaneId from array median= 4954.0 ns [min 4910.0 .. max 5998.0] n=51 -native runtime lance-graph native runtime: abi 0.1, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so +construct N Descriptor (2 wrappers each) 29.87 MiB + ... per Descriptor 31.32 B +pass 2 wrappers through 3 call levels 19.96 MiB + ... per call 20.93 B +read 65,536 LaneId from array median= 3378.0 ns [min 3351.0 .. max 5135.0] n=51 +native runtime lance-graph native runtime: abi 0.11, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so == FFM ADDRESSING — is the wrapper free where it touches native memory? ====== platform valhalla sum (identical across all three) 6929623 -bare long index median= 59216.0 ns [min 59058.0 .. max 112785.0] n=51 -RowRange bounds (wrapper hoisted) median= 56995.0 ns [min 56663.0 .. max 126870.0] n=51 -per-element wrapper: bytes allocated 3.00 MiB - ... per element 48.00 B -Ordinal built per element median= 91383.0 ns [min 91100.0 .. max 120947.0] n=51 +bare long index median= 38594.0 ns [min 38535.0 .. max 108174.0] n=51 +RowRange bounds (wrapper hoisted) median= 36868.0 ns [min 36771.0 .. max 57762.0] n=51 +per-element wrapper: bytes allocated 2.00 MiB + ... per element 32.00 B +Ordinal built per element median= 68209.0 ns [min 62115.0 .. max 83104.0] n=51 == THE THESIS — 65,536 entities, three representations ======================= platform valhalla @@ -72,20 +72,20 @@ answer (identical across all paths) 2173 rows, sum 499246 selectivity 3.32% == heap cost =============================================================== -(1) native — Java bytes allocated (warm) 816 B per query, for the fluent chain itself +(1) native — Java bytes allocated (warm) 712 B per query, for the fluent chain itself (1) native — Java objects per row 0 (1) native — native lane bytes 1.00 MiB (u64 id + u32 class + i32 value) (1) native — mask bytes 8.0 KiB (1 bit per row, packed) -(2)/(3) hydrate 65536 Row — allocated 2.50 MiB - ... per row 40.00 B +(2)/(3) hydrate 65536 Row — allocated 2.00 MiB + ... per row 32.00 B array flatness NOT-FLAT - ratio vs native lane bytes 2.50x - retained heap (APPROX, gc-delta) 2.75 MiB + ratio vs native lane bytes 2.00x + retained heap (APPROX, gc-delta) 2.25 MiB == time to answer the question ============================================= -(1) native one crossing, fused plan median= 20603.0 ns [min 15179.0 .. max 59260.0] n=51 -(2/3) hydrate 65536 Row objects median= 1086237.0 ns [min 1010540.0 .. max 4481968.0] n=51 -(2/3) scan the materialised objects median= 80753.0 ns [min 75090.0 .. max 140418.0] n=51 -(2/3) hydrate THEN scan (honest total) median=25828809.0 ns [min 6057980.0 .. max 39247669.0] n=51 +(1) native one crossing, fused plan median= 15958.0 ns [min 15714.0 .. max 68146.0] n=51 +(2/3) hydrate 65536 Row objects median= 908356.0 ns [min 822247.0 .. max 1356237.0] n=51 +(2/3) scan the materialised objects median= 80056.0 ns [min 77220.0 .. max 125590.0] n=51 +(2/3) hydrate THEN scan (honest total) median=19125805.0 ns [min 18194194.0 .. max 26246871.0] n=51 lab complete. diff --git a/valhalla-lab/results/valhalla-nofieldflat.txt b/valhalla-lab/results/valhalla-nofieldflat.txt index 592d4d9..5bc7a62 100644 --- a/valhalla-lab/results/valhalla-nofieldflat.txt +++ b/valhalla-lab/results/valhalla-nofieldflat.txt @@ -1,9 +1,9 @@ Picked up JAVA_TOOL_OPTIONS: lance-graph-java :: valhalla lab platform valhalla -java.vm.version 27-jep401ea3+1-1 -java.vendor.version - -jvm args [--enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED, -XX:+UnlockDiagnosticVMOptions, -XX:-UseFieldFlattening] +java.vm.version 28-beta+16-ea +java.vendor.version Temurin-28+16-202609171655 +jvm args [--enable-preview, --enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED, -XX:+UnlockDiagnosticVMOptions, -XX:-UseFieldFlattening] == (a) SEMANTIC TRUTH — is identity observable? ============================== platform valhalla — value-record vocabulary; null-restricted non-atomic arrays; @NullRestricted fields @@ -37,32 +37,32 @@ Row 1 long + 2 int payload=16 B array=NOT-FLAT platform valhalla allocation instrument baseline 0 B N (operations per measurement) 1000000 -construct N LaneId, store into array 2.58 MiB - ... per LaneId 2.71 B -construct N LaneId, never escaping 6.68 MiB - ... per LaneId 7.00 B +construct N LaneId, store into array 1.87 MiB + ... per LaneId 1.96 B +construct N LaneId, never escaping 5.87 MiB + ... per LaneId 6.16 B LaneId[1024] flatness FLAT -allocate+fill LaneId[N] (array + elements) 6.40 MiB - ... per element 6.71 B +allocate+fill LaneId[N] (array + elements) 5.68 MiB + ... per element 5.95 B bare LaneId[N] with no elements stored 3.81 MiB ... per slot 4.00 B Descriptor kind value class with two @NullRestricted value fields Descriptor fields null-restricted true -construct N Descriptor (2 wrappers each) 76.29 MiB - ... per Descriptor 80.00 B -pass 2 wrappers through 3 call levels 11.85 MiB - ... per call 12.43 B -read 65,536 LaneId from array median= 4952.0 ns [min 4878.0 .. max 6129.0] n=51 -native runtime lance-graph native runtime: abi 0.1, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so +construct N Descriptor (2 wrappers each) 53.41 MiB + ... per Descriptor 56.00 B +pass 2 wrappers through 3 call levels 6.48 MiB + ... per call 6.79 B +read 65,536 LaneId from array median= 4350.0 ns [min 4295.0 .. max 19151.0] n=51 +native runtime lance-graph native runtime: abi 0.11, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so == FFM ADDRESSING — is the wrapper free where it touches native memory? ====== platform valhalla sum (identical across all three) 6929623 -bare long index median= 59291.0 ns [min 59072.0 .. max 127512.0] n=51 -RowRange bounds (wrapper hoisted) median= 56959.0 ns [min 56598.0 .. max 88827.0] n=51 -per-element wrapper: bytes allocated 1.50 MiB - ... per element 24.00 B -Ordinal built per element median= 53651.0 ns [min 49983.0 .. max 89971.0] n=51 +bare long index median= 46771.0 ns [min 46724.0 .. max 113186.0] n=51 +RowRange bounds (wrapper hoisted) median= 41828.0 ns [min 41267.0 .. max 79048.0] n=51 +per-element wrapper: bytes allocated 1.00 MiB + ... per element 16.00 B +Ordinal built per element median= 42301.0 ns [min 41723.0 .. max 59959.0] n=51 == THE THESIS — 65,536 entities, three representations ======================= platform valhalla @@ -72,20 +72,20 @@ answer (identical across all paths) 2173 rows, sum 499246 selectivity 3.32% == heap cost =============================================================== -(1) native — Java bytes allocated (warm) 816 B per query, for the fluent chain itself +(1) native — Java bytes allocated (warm) 680 B per query, for the fluent chain itself (1) native — Java objects per row 0 (1) native — native lane bytes 1.00 MiB (u64 id + u32 class + i32 value) (1) native — mask bytes 8.0 KiB (1 bit per row, packed) -(2)/(3) hydrate 65536 Row — allocated 2.50 MiB - ... per row 40.00 B +(2)/(3) hydrate 65536 Row — allocated 2.00 MiB + ... per row 32.00 B array flatness NOT-FLAT - ratio vs native lane bytes 2.50x - retained heap (APPROX, gc-delta) 2.75 MiB + ratio vs native lane bytes 2.00x + retained heap (APPROX, gc-delta) 2.25 MiB == time to answer the question ============================================= -(1) native one crossing, fused plan median= 13900.0 ns [min 13785.0 .. max 32163.0] n=51 -(2/3) hydrate 65536 Row objects median= 685073.0 ns [min 615948.0 .. max 5630226.0] n=51 -(2/3) scan the materialised objects median= 89883.0 ns [min 86979.0 .. max 142069.0] n=51 -(2/3) hydrate THEN scan (honest total) median= 872592.0 ns [min 804643.0 .. max 1338372.0] n=51 +(1) native one crossing, fused plan median= 19490.0 ns [min 19268.0 .. max 40047.0] n=51 +(2/3) hydrate 65536 Row objects median= 1357173.0 ns [min 583460.0 .. max 4309976.0] n=51 +(2/3) scan the materialised objects median= 82347.0 ns [min 74581.0 .. max 162329.0] n=51 +(2/3) hydrate THEN scan (honest total) median= 840043.0 ns [min 745750.0 .. max 4274093.0] n=51 lab complete. diff --git a/valhalla-lab/results/valhalla-noflat.txt b/valhalla-lab/results/valhalla-noflat.txt index 4475065..85148c0 100644 --- a/valhalla-lab/results/valhalla-noflat.txt +++ b/valhalla-lab/results/valhalla-noflat.txt @@ -1,9 +1,9 @@ Picked up JAVA_TOOL_OPTIONS: lance-graph-java :: valhalla lab platform valhalla -java.vm.version 27-jep401ea3+1-1 -java.vendor.version - -jvm args [--enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED, -XX:+UnlockDiagnosticVMOptions, -XX:-UseArrayFlattening, -XX:-UseFieldFlattening] +java.vm.version 28-beta+16-ea +java.vendor.version Temurin-28+16-202609171655 +jvm args [--enable-preview, --enable-native-access=ALL-UNNAMED, -Dstdout.encoding=UTF-8, -Dlgj.library=/home/user/lance-graph-java/target/release/liblgj_abi.so, --enable-preview, --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED, --add-exports=java.base/jdk.internal.value=ALL-UNNAMED, -XX:+UnlockDiagnosticVMOptions, -XX:-UseArrayFlattening, -XX:-UseFieldFlattening] == (a) SEMANTIC TRUTH — is identity observable? ============================== platform valhalla — value-record vocabulary; null-restricted non-atomic arrays; @NullRestricted fields @@ -39,8 +39,8 @@ allocation instrument baseline 0 B N (operations per measurement) 1000000 construct N LaneId, store into array 15.26 MiB ... per LaneId 16.00 B -construct N LaneId, never escaping 4.70 MiB - ... per LaneId 4.93 B +construct N LaneId, never escaping 5.03 MiB + ... per LaneId 5.28 B LaneId[1024] flatness NOT-FLAT allocate+fill LaneId[N] (array + elements) 19.07 MiB ... per element 20.00 B @@ -48,21 +48,21 @@ bare LaneId[N] with no elements stored 3.81 MiB ... per slot 4.00 B Descriptor kind value class with two @NullRestricted value fields Descriptor fields null-restricted true -construct N Descriptor (2 wrappers each) 53.41 MiB - ... per Descriptor 56.00 B -pass 2 wrappers through 3 call levels 9.41 MiB - ... per call 9.86 B -read 65,536 LaneId from array median= 41431.0 ns [min 41208.0 .. max 59606.0] n=51 -native runtime lance-graph native runtime: abi 0.1, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so +construct N Descriptor (2 wrappers each) 45.78 MiB + ... per Descriptor 48.00 B +pass 2 wrappers through 3 call levels 9.45 MiB + ... per call 9.91 B +read 65,536 LaneId from array median= 30317.0 ns [min 29856.0 .. max 81985.0] n=51 +native runtime lance-graph native runtime: abi 0.11, simd ndarray::simd avx512, profile release, library /home/user/lance-graph-java/target/release/liblgj_abi.so == FFM ADDRESSING — is the wrapper free where it touches native memory? ====== platform valhalla sum (identical across all three) 6929623 -bare long index median= 59150.0 ns [min 59081.0 .. max 122825.0] n=51 -RowRange bounds (wrapper hoisted) median= 56919.0 ns [min 56634.0 .. max 79142.0] n=51 +bare long index median= 42843.0 ns [min 42812.0 .. max 105187.0] n=51 +RowRange bounds (wrapper hoisted) median= 36818.0 ns [min 36618.0 .. max 53803.0] n=51 per-element wrapper: bytes allocated 1.00 MiB ... per element 16.00 B -Ordinal built per element median= 49921.0 ns [min 49739.0 .. max 69259.0] n=51 +Ordinal built per element median= 39449.0 ns [min 38264.0 .. max 71650.0] n=51 == THE THESIS — 65,536 entities, three representations ======================= platform valhalla @@ -72,20 +72,20 @@ answer (identical across all paths) 2173 rows, sum 499246 selectivity 3.32% == heap cost =============================================================== -(1) native — Java bytes allocated (warm) 816 B per query, for the fluent chain itself +(1) native — Java bytes allocated (warm) 680 B per query, for the fluent chain itself (1) native — Java objects per row 0 (1) native — native lane bytes 1.00 MiB (u64 id + u32 class + i32 value) (1) native — mask bytes 8.0 KiB (1 bit per row, packed) -(2)/(3) hydrate 65536 Row — allocated 2.00 MiB - ... per row 32.01 B +(2)/(3) hydrate 65536 Row — allocated 1.50 MiB + ... per row 24.01 B array flatness NOT-FLAT - ratio vs native lane bytes 2.00x - retained heap (APPROX, gc-delta) 2.25 MiB + ratio vs native lane bytes 1.50x + retained heap (APPROX, gc-delta) 1.75 MiB == time to answer the question ============================================= -(1) native one crossing, fused plan median= 16049.0 ns [min 15609.0 .. max 32010.0] n=51 -(2/3) hydrate 65536 Row objects median= 578187.0 ns [min 543201.0 .. max 1011453.0] n=51 -(2/3) scan the materialised objects median= 119838.0 ns [min 111905.0 .. max 211651.0] n=51 -(2/3) hydrate THEN scan (honest total) median= 1635251.0 ns [min 709125.0 .. max 6428085.0] n=51 +(1) native one crossing, fused plan median= 19267.0 ns [min 19042.0 .. max 32788.0] n=51 +(2/3) hydrate 65536 Row objects median= 693393.0 ns [min 562611.0 .. max 4519417.0] n=51 +(2/3) scan the materialised objects median= 70114.0 ns [min 67521.0 .. max 132648.0] n=51 +(2/3) hydrate THEN scan (honest total) median= 1241729.0 ns [min 1142249.0 .. max 1628911.0] n=51 lab complete. diff --git a/valhalla-lab/run.sh b/valhalla-lab/run.sh index 091c11a..afa621f 100755 --- a/valhalla-lab/run.sh +++ b/valhalla-lab/run.sh @@ -1,5 +1,16 @@ #!/usr/bin/env bash -# The A/B. One experiment source, two object models, two JDKs, one diff. +# The A/B. One experiment source, two object models, ONE JDK, one diff. +# +# RE-SCOPED 2026-09-19. Production is now the Valhalla arm (the six vocabulary types are +# `value record`s on JDK 28), so this lab no longer answers "present vs future". It answers +# a narrower and better-posed question: on ONE JDK, what does the object model alone change? +# Previously the two arms differed by object model AND by JDK (26 vs 27 EA) — two variables. +# Now the JDK is held fixed and the modifier is the only difference, so the diff is attributable. +# +# Second consequence, and it is not cosmetic: the production API classfiles are now +# preview-marked, and that marking is transitive to every consumer. The `record` arm therefore +# also runs with --enable-preview. It is a RECORD arm on a preview-enabled JVM, not a +# "preview-free" arm; the label "stable" below is kept only for path compatibility. # # Every measurement this lab reports is produced by this script. Nothing is quoted from memory. set -uo pipefail @@ -7,8 +18,8 @@ set -uo pipefail LAB="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" ROOT="$(cd "$LAB/.." && pwd)" -STABLE_JDK="${STABLE_JDK:-/opt/jdks/jdk-26.0.2}" -VALHALLA_JDK="${VALHALLA_JDK:-/opt/jdks/jdk-27}" +STABLE_JDK="${STABLE_JDK:-/opt/jdks/jdk-28+16}" # the `record` arm — same JDK, by design +VALHALLA_JDK="${VALHALLA_JDK:-/opt/jdks/jdk-28+16}" # the `value record` arm LIB_DIR="${LIB_DIR:-$ROOT/target/release}" OUT="$LAB/results" @@ -44,8 +55,10 @@ compile () { # $1=tag $2=jdk $3=vocab-root shift 3 -> extra j local api="$OUT/$tag-api" lab="$OUT/$tag-lab" rm -rf "$api" "$lab"; mkdir -p "$api" "$lab" - # the production API, compiled by THIS jdk, with no preview features anywhere - "$jdk/bin/javac" -d "$api" $(find "$ROOT/java/src/main/java" -name '*.java') \ + # The production API. It is Valhalla-shaped now, so it needs the preview flag in BOTH arms — + # that is the point of the re-scope, not a leak: the lab measures the lab vocabulary's object + # model against a production API that is value-shaped either way. + "$jdk/bin/javac" --release 28 --enable-preview -d "$api" $(find "$ROOT/java/src/main/java" -name '*.java') \ 2> "$OUT/$tag-api-javac.log" || { cat "$OUT/$tag-api-javac.log"; fail "$tag: API compile"; } "$jdk/bin/javac" "$@" -cp "$api" -d "$lab" \ @@ -55,9 +68,12 @@ compile () { # $1=tag $2=jdk $3=vocab-root shift 3 -> extra j } banner "1. compiling" -compile stable "$STABLE_JDK" "$LAB/src/stable" +# the `record` arm still needs --enable-preview: it LOADS preview-marked production classes +compile stable "$STABLE_JDK" "$LAB/src/stable" -source 28 -target 28 --enable-preview +# NOTE: no --release here. `--release` forbids --add-exports into java.base, and the lab's +# layout probes need jdk.internal.value. The JDK is 28, so the default target is already 28. compile valhalla "$VALHALLA_JDK" "$LAB/src/valhalla" \ - --enable-preview -source 27 -target 27 "${VAL_EXPORTS[@]}" + -source 28 -target 28 --enable-preview "${VAL_EXPORTS[@]}" # ── 2. run ──────────────────────────────────────────────────────────────────────────────────── run () { # $1=tag $2=jdk $3=label shift 3 -> extra jvm args @@ -65,6 +81,7 @@ run () { # $1=tag $2=jdk $3=label shift 3 -> extra jvm args local f="$OUT/$tag-$label.txt" echo "--> $tag/$label" ( JAVA_TOOL_OPTIONS= "$jdk/bin/java" \ + --enable-preview \ --enable-native-access=ALL-UNNAMED \ -Dstdout.encoding=UTF-8 \ -Dlgj.library="$LIB_DIR/liblgj_abi.so" \ diff --git a/valhalla-lab/src/valhalla/com/adaworldapi/lancegraph/lab/Platform.java b/valhalla-lab/src/valhalla/com/adaworldapi/lancegraph/lab/Platform.java index 6a53d9d..bdd7876 100644 --- a/valhalla-lab/src/valhalla/com/adaworldapi/lancegraph/lab/Platform.java +++ b/valhalla-lab/src/valhalla/com/adaworldapi/lancegraph/lab/Platform.java @@ -30,7 +30,15 @@ static boolean isValueClass(Class type) { } static String arrayFlatness(Object array) { - return ValueClass.isFlatArray(array) ? "FLAT" : "NOT-FLAT"; + // JDK 28 narrowed `ValueClass.isFlatArray` from `Object` to `Object[]` (it was `Object` + // on the 27-jep401ea3 build this lab was written against). Keeping the parameter as + // `Object` here is deliberate — callers pass arbitrary arrays, including primitive ones, + // and a primitive array is never flat in the value-class sense, so it answers without + // reaching the internal API at all. + if (!(array instanceof Object[] refs)) { + return "NOT-FLAT"; + } + return ValueClass.isFlatArray(refs) ? "FLAT" : "NOT-FLAT"; } /**