Repository navigation
hf: resolve compressed-tensors targets by checkpoint module class - #605
Conversation
There was a problem hiding this comment.
Verdict: CLEAN: Class targets use explicit module metadata, and unmatched quantized weights receive named refusals.
Checked
Duplicates: 4 new items checked (enum, classifier, two tests); no duplicate owner found. Dryer reported 0 pairs.
Complexity: crapper flagged no changed functions. The classifier adds load-time name matching; no hot path changes.
Silent failure: 5 fallback sites followed. Unparsable layer, expert and shard indices return no class. With class-only selectors, unmatched non-plain weights receive named refusals. Unprefixed names continue through classification.
Tests: static mutation review. Unconditional class matching, dropped ignore handling, removed class rejection, and missing embedding classification each make assertions fail. Migrated format, scale and header assertions remain.
Gates: no pins, thresholds or test cases removed or relaxed.
Adversarial checks: class ignores preserve the other class; identical packed shapes do not classify norms or convolutions. Exact-name and regex targets work without class lookup.
Escape catalog: no C1–C4 patterns match.
Notes
Tests were inspected, not executed in this review. The PR reports that the external GLM oracle gate and GPU inference were not run.
🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)
|
Integrated upstream Exercised after the merge:
The existing class-target docs remain accurate; this merge introduces no further public contract. New-head upstream CI is pending; the previous green/approval was for |
There was a problem hiding this comment.
Since the last review: no must-fix or non-blocking items were open.
Verdict: CLEAN: The upstream merge preserves explicit class resolution and the stored-format refusal paths.
Checked
Duplicates: 0 new PR-specific definitions since the last review; dryer reported 0 pairs. The extracted planner and new planning test already exist in base.
Scope: 58 incremental-diff paths match base exactly. The remaining two contain the existing API migration and the moved resolver argument.
Complexity: crapper flagged no changed functions. Planning remains load-time work.
Silent failure: 3 paths followed. Missing class metadata leaves packed weights refused, and mapped consumers return the reason. Unsupported F32 matrices and unquantized expert stacks return named errors before shard writing.
Tests: static mutation review. Dropping BF16 preservation at crates/model-hf/src/bin/prepare.rs:285, dropping the F32 refusal at line 279, or dropping the expert-stack refusal at line 272 fails the new planning assertions. Dropping F32 support in either output format fails the added normalization assertions. Existing class-selection tests still reject blanket matching.
Gates: no PR-specific pins, thresholds or test cases removed or relaxed.
Adversarial checks: the moved planner receives the classifier; class-only ignores preserve the other class; name selectors remain independent of metadata. A single-expert plain stack still reaches the stack refusal. Stored BF16 values still pass the finite-value check.
Escape catalog: no C1–C4 patterns match the PR-specific changes.
Notes
Tests were inspected, not executed in this review. The PR reports that GPU inference and the external GLM oracle gate were not run.
🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)
|
Exact-head upstream T0 for |
|
Merged as ba2057c on main. Main had moved, so I resolved four conflicts in Gates on the box, fixture tier, Thank you for the fix. |
Addresses the compressed-tensors class-target item of #524. Companion drive-device fix: #604; neither PR alone claims the whole issue is closed.
Fix and contract
quant_config::plannow requires an explicit module-class resolver. All production callers and fixture callers are migrated; there is no old blanket-Linear compatibility path.LinearandEmbeddingmatch only their ownModuleClass, for both group targets and ignore selectors. Unknown metadata matches no class. Exact names andre:selectors retain their name semantics.model_hf::arch, alongside the GLM/Qwen3.8 maps. It explicitly separates projections (including F32-stored projection roles), token/PLE embedding tables, norms, convolutions, vectors and unknown names. Placement Role::Attention alone is deliberately not used: it also contains norms.Verification
Embeddingtarget did not place its declared tensor. Now the packed embedding/projection cases are disjoint; the model-HF consumer test additionally exercises shared-class ignore precedence and rejects norms/conv/unknown roles despite the same 2D packed shape.cargo test -p bloomery-hf --lib: 84 passed (existing real checkpoint format/dequantization pins unchanged).cargo test -p bloomery-model-hf --lib --bin prepare: 23 library + 3 prepare tests passed; 1 existing corpus test ignored because$BLOOMERY_QWEN38_HFreal tokenizer/model folder is unavailable.Linearplaced only in_proj_a;Embedding, exactmodel.embed_tokens, andre:model\.embed_tokens$placed only the embedding. Norm and conv weights were named refusals in every scenario. The example was removed after the run.cargo fmt --all --check,cargo clippy -p bloomery-hf -p bloomery-model-hf --all-targets, GLM map integration target compile,check-comments.sh,check-arch.sh: passed. The real external GLM oracle gate and GPU inference were not run.Current upstream integration (0b89b0e)
Integrated upstream
mainat5b626c3in 0b89b0e. The sole conflict wasprepare: upstream moved tensor planning intoplan_tensorsand changed the default to preserve stored BF16, with explicit Q8 opt-in. Kept that refactor/default/refusal behavior and passed the explicitmodule_classresolver at the moved planner call; no compatibility shim.Exercised after the merge:
BLOOMERY_QWEN38_HFunavailable). This includes upstream's stored-BF16 / Q8 opt-in / F32 refusal cases.Linear,Embedding, exact name, regex, andignore: [Embedding]each selected only the expected packed projection/embedding; unmatched classes, norm and convolution stayed refused. Temporary example removed.The existing class-target docs remain accurate; this merge introduces no further public contract. Exact-head upstream T0 for
0b89b0epassed (23m52s): https://github.com/midagedev/bloomery/actions/runs/39595869472/job/119207897131 . The earlier approval/green was for105c3f7. No GPU run or new GGUF quant-format support is claimed.