th#1803: the consumer half of the layout gate — a decoder that DECLARES hf.fp8-blockwise@1 - #662
Merged
Merged
Conversation
…ES hf.fp8-blockwise@1 tensorhub 226070fd made the tensor-layout contract enforceable at rebind, and it refused the H3 fp8 conditioner correctly: no decoder in any image declared `hf.fp8-blockwise@1`, the contract transcribed from that artifact's own header bytes. This is the declaration, and the loader that makes it true. `models/hf_fp8_blockwise.py` reads transformers' FineGrainedFP8 layout — fp8 e4m3 [out,in] weights with a `weight_scale_inv` F32 grid, one scale per 128x128 block, applied as a MULTIPLIER (verified against transformers 5.13.1's own `_dequantize_one`) — resident for the blockwise GEMM (fp8-w8a8-dynamic) or upcast-ahead at load (fp8-w8a16). The refusal is the point. `cozy.fp8-rowwise@1` is also "fp8 e4m3, dynamic activations" with a per-ROW rank-1 `weight_scale`; reading one as the other broadcasts a scale over the wrong span and yields plausible, wrong numbers with no error. So the tree is verified from HEADERS ALONE before any weight byte is read, and a rowwise tree is refused by name — PRODUCIBLE, not CONVERTIBLE (DESIGN-RULINGS §1.33): re-blocking is a re-quantization, so the remedy is a conversion-endpoint artifact, never a load-time repack. `dequantize_block_scaled` is the registry's declared `hf.fp8_blockwise.dequant@1` reference dequant — the function a §1.33 lossless conversion proves bit-exactness against. No quantization happens here and there is no quantize-if-missing fallback (Paul, 2026-08-11: quantization is ahead-of-time via conversion endpoints, never at cold boot). Proven with no mocks: a real tiny Llama quantized to the layout, written as safetensors, loaded back through the declaring loader; weights bit-identical to the reference dequant and within fp8 error of the originals, a real forward with finite logits, rowwise/transposed/dense trees refused by name, and the build derivation carrying the contract so `Satisfies` can succeed for the first time.
PaulFidika
added a commit
that referenced
this pull request
Aug 11, 2026
th#1803 PR #662 landed `hf.fp8-blockwise@1` in the transcription WITH a real decoder behind it while this branch was open. Rebased onto it; the changelog no longer claims a transcription this issue did not make.
PaulFidika
added a commit
that referenced
this pull request
Aug 11, 2026
…de can execute (#663) * pgw#1143 steps 1-3 (§1.33): a slot DECLARES the tensor layouts its code can execute The code side of the layout contract had no declaration surface at all — `api/slot.py` carried `pipeline_cls, selected_by, family, default_checkpoint, root, optional` and nothing else, so the hub's th#1803 rebind gate was keyed off the wheel's decoder census (2 distinct signatures across 64 releases) rather than off what the author says the slot needs. `Slot(layouts={component_path: (handle, ...)})` is that declaration: an ordered tuple per component path, `"*"` for the whole tree, order IS preference (§1.33 pt 2). It is published into the release manifest as `functions[].slots[].layouts` for the hub to gate against (th#1809 T1-T2). - **Absent is UNDECLARED**, a tri-state — never "accepts everything" and never "accepts nothing". An empty mapping and an empty tuple are both decoration-time errors; collapsing the tri-state is th#1580's fail-open defect wearing a new name. - **On `Slot`, not `Compile`.** `Compile`'s fields feed `contract_axes()`, a cell-key input, and §1.33 pt 5 is that conversion is upstream of compute and invisible to cell identity. A layout declaration there would re-key every cell in the fleet for the one fact that must never enter the key. - **Handles at the constructor, KEYS at decoration.** The handle vocabulary is `KNOWN_CONTRACTS` (refusal text reused verbatim from `tensor_layout_contract._validate`); the component keys are checked against the DERIVED component tree in `_spec_for_handler`, because a key that matches nothing reads as a declaration and gates nothing. - **The SDK emits HANDLES, never digests** (th#1580 A2: descriptors are Go). The hub resolves handle -> `Contract.Digest()` at manifest ingest against its own registry — the only moment one wheel and one hub are both pinned — so `KNOWN_CONTRACTS` is honestly a transcription that may be stale and is CHECKED. `hf.fp8-blockwise@1` joins it: registered hub-side since th#1803 (`226070fd`), and its absence made §1.33's own worked example (an fp8 text_encoder against a bf16 DiT) undeclarable. - **A composite `<topology>+<quant>` handle is refused, naming th#1809 T3.** The topology axis has no registry to compare against yet; storing half a pair is not exact. - **The census stays a LOWER bound.** A declared handle no `@implements_contract` decoder backs lands on the manifest as `layouts_census_unbacked` and does NOT refuse — the blockwise conditioner is decoded natively by `transformers` through `quantization_config` with zero cozy markers, and refusing would make the motivating case illegal. - **`scripts/lint_layout_declarations.py`** (new `fast gates` step) refuses a declaration the AST sweep cannot read. Discovery imports endpoint modules torch-stubbed, and pgw#1107's sweep reads them without importing at all; a computed declaration is legible to neither, which would make the published manifest the only place the demand can be read. RED on master, measured: `Slot(P, layouts=...)` -> `TypeError: unexpected keyword argument 'layouts'`; `_slot_to_manifest` emits `['components', 'name', 'pipeline_class']`; `KNOWN_CONTRACTS` holds 5 of the hub's 6 handles. * pgw#1143: the layout lint sweeps src/ only The fence's subject is a declaration that gets PUBLISHED. tests/ holds the opposite by construction — a test proving the constructor refuses a computed declaration has to write one — so sweeping it made this fence and its own negative test mutually exclusive (CI, run 31534100939). Endpoint repos point the script at their own tree, where every declaration is a real one. * pgw#1143: pin the wire the hub actually reads The declaration crosses TWO encoders before any gate sees it — msgspec TOML into endpoint.lock, then JSON into tensorhub's `manifestSlotLayoutDoc` (`map[string][]string`). A nested table inside an array-of-tables is exactly where a TOML encoder's key ordering bites, and the failure would be silent: the hub would read the slot as UNDECLARED and the gate would fall back to the census with nothing to say about it. * pgw#1143: drop the KNOWN_CONTRACTS claim — master got there first th#1803 PR #662 landed `hf.fp8-blockwise@1` in the transcription WITH a real decoder behind it while this branch was open. Rebased onto it; the changelog no longer claims a transcription this issue did not make.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tensorhub
226070fd(th#1803, PR #1046) made the tensor-layout contract enforceable at rebind — a binding whose artifact bytes no decoder in the target image declares is refused with a typed 422, no pod bought. It refused the H3 fp8 conditioner rebind correctly: nothing in any image declaredhf.fp8-blockwise@1, the contract tensorhub transcribed from that artifact's own 165,264 header bytes.This is the consumer half: the declaration, and the loader that makes it true.
What lands
models/hf_fp8_blockwise.py— transformers' FineGrainedFP8 / DeepSeek-style layout: fp8 e4m3[out,in]weights, aweight_scale_invF32 grid at one scale per 128×128 block, applied as a multiplier (verified againsttransformers==5.13.1's own_dequantize_one, not inferred from the_invsuffix).resident=Truekeeps fp8 and runs the blockwise GEMM (fp8-w8a8-dynamic);resident=Falseupcasts ahead at load (fp8-w8a16, the portable/CPU arm).@implements_contract(contract="hf.fp8-blockwise@1", …)on the loader, so the image's build-time derivation carries it intorelease_execution_lanesand the hub'sSatisfiescomparison can succeed for the first time.inspect_hf_fp8_blockwise— verification from safetensors HEADERS ALONE, before any weight byte is read.dequantize_block_scaled— the registry's declaredhf.fp8_blockwise.dequant@1reference dequant. §1.33 makes this the function a lossless conversion must prove bit-exactness against, so it exists once.Why the refusals matter more than the load
cozy.fp8-rowwise@1is also "fp8 e4m3 with dynamic activations" — same element type, same activation scheme, one English name. Its scale leaf isweight_scale, rank 1, per ROW. A decoder that reads a 128×128 reciprocal grid as a per-row multiplier broadcasts one scale across the wrong span: no exception, plausible numbers, wrong. So a rowwise tree is refused by name, and so are a transposed/mis-blocked grid, an fp8 weight with no scale, a rank-1 scale, and a config that disagrees with the bytes.The pair is PRODUCIBLE, not CONVERTIBLE (DESIGN-RULINGS §1.33): re-blocking rowwise scales is a re-quantization with new numerics, so the remedy named in the refusal is a conversion-endpoint artifact, never a load-time repack.
No quantization happens in this path (Paul, 2026-08-11, th#1803 follow-up 2): quantization is ahead-of-time via conversion endpoints, never at cold boot. There is deliberately no quantize-if-missing fallback.
Proof (no mocks, CPU, ~9s)
tests/test_hf_fp8_blockwise_th1803.pybuilds a real tiny Llama, quantizes it to the layout, writes real safetensors + the samequantization_configshape the published conditioner carries, and loads it back through the declaring loader:down_proj [128,256] → scale [1,2]);derive_execution_lanes()carrieshf.fp8-blockwise@1with this decoder.Files: one new module, one new test, one line-group in
tensor_layout_contract.py's handle list. No sibling lane touches these paths.Follow-up in this lane: the decode proof on the real 27.6 GB artifact on a pod, recorded on th#1803.