Skip to content

hf: resolve compressed-tensors targets by checkpoint module class - #605

Merged
midagedev merged 2 commits into
midagedev:mainfrom
Burntgogi:fix/issue524-class-targets
Oct 21, 2026
Merged

midagedev merged 2 commits into
midagedev:mainfrom
Burntgogi:fix/issue524-class-targets

Conversation

@Burntgogi

@Burntgogi Burntgogi commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Addresses the compressed-tensors class-target item of #524. Companion drive-device fix: #604; neither PR alone claims the whole issue is closed.

Fix and contract

  • quant_config::plan now requires an explicit module-class resolver. All production callers and fixture callers are migrated; there is no old blanket-Linear compatibility path.
  • Linear and Embedding match only their own ModuleClass, for both group targets and ignore selectors. Unknown metadata matches no class. Exact names and re: selectors retain their name semantics.
  • The shared checkpoint-module classifier is owned by model_hf::arch, alongside the GLM/Qwen3.8 maps. It explicitly separates projections (including F32-stored projection roles), token/PLE embedding tables, norms, convolutions, vectors and unknown names. Placement Role::Attention alone is deliberately not used: it also contains norms.
  • Unsupported bare capitalized class-name selectors are refused by name at config parsing (including ignore selectors). Qualified module names remain names.
  • Class resolution is lazy and allocation-free per module; name-only selectors, ModelOpt and fp8 configs do not require a class lookup. Source/format/layout checks are not weakened. This does not add a new GGUF conversion or GPU quant format.
  • The existing quant-config README documents the explicit metadata contract.

Verification

  • FAIL-first packed embedding regression: the previous Embedding target did not place its declared tensor. Now the packed embedding/projection cases are disjoint; the model-HF consumer test additionally exercises shared-class ignore precedence and rejects norms/conv/unknown roles despite the same 2D packed shape.
  • cargo test -p bloomery-hf --lib: 84 passed (existing real checkpoint format/dequantization pins unchanged).
  • cargo test -p bloomery-model-hf --lib --bin prepare: 23 library + 3 prepare tests passed; 1 existing corpus test ignored because $BLOOMERY_QWEN38_HF real tokenizer/model folder is unavailable.
  • Compiled and ran a temporary executable consumer of the public planner with actual model-HF classifier: Linear placed only in_proj_a; Embedding, exact model.embed_tokens, and re:model\.embed_tokens$ placed only the embedding. Norm and conv weights were named refusals in every scenario. The example was removed after the run.
  • cargo fmt --all --check, cargo clippy -p bloomery-hf -p bloomery-model-hf --all-targets, GLM map integration target compile, check-comments.sh, check-arch.sh: passed. The real external GLM oracle gate and GPU inference were not run.
  • No LSP server is available here; the required exported-plan references query returned no server. Every discovered callsite was migrated and both libraries, bins and the shared library/integration test helper were compiled.

Current upstream integration (0b89b0e)

Integrated upstream main at 5b626c3 in 0b89b0e. The sole conflict was prepare: upstream moved tensor planning into plan_tensors and changed the default to preserve stored BF16, with explicit Q8 opt-in. Kept that refactor/default/refusal behavior and passed the explicit module_class resolver at the moved planner call; no compatibility shim.

Exercised after the merge:

  • HF library: 84 passed; model-HF library: 23 passed; prepare: 4 passed, 1 existing external-corpus test ignored (BLOOMERY_QWEN38_HF unavailable). This includes upstream's stored-BF16 / Q8 opt-in / F32 refusal cases.
  • Compiled actual API consumer with the real checkpoint classifier: Linear, Embedding, exact name, regex, and ignore: [Embedding] each selected only the expected packed projection/embedding; unmatched classes, norm and convolution stayed refused. Temporary example removed.
  • Both packages' all-target clippy, workspace fmt check, comment and architecture checks passed.

The existing class-target docs remain accurate; this merge introduces no further public contract. Exact-head upstream T0 for 0b89b0e passed (23m52s): https://github.com/midagedev/bloomery/actions/runs/39595869472/job/119207897131 . The earlier approval/green was for 105c3f7. No GPU run or new GGUF quant-format support is claimed.

@github-actions github-actions Bot added the model The CPU engine and model files: crates/model, models, hf, gguf, tokenizer, jinja label Oct 21, 2026
github-actions[bot]
github-actions Bot previously approved these changes Oct 21, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CLEAN: Class targets use explicit module metadata, and unmatched quantized weights receive named refusals.

Checked

Duplicates: 4 new items checked (enum, classifier, two tests); no duplicate owner found. Dryer reported 0 pairs.
Complexity: crapper flagged no changed functions. The classifier adds load-time name matching; no hot path changes.
Silent failure: 5 fallback sites followed. Unparsable layer, expert and shard indices return no class. With class-only selectors, unmatched non-plain weights receive named refusals. Unprefixed names continue through classification.
Tests: static mutation review. Unconditional class matching, dropped ignore handling, removed class rejection, and missing embedding classification each make assertions fail. Migrated format, scale and header assertions remain.
Gates: no pins, thresholds or test cases removed or relaxed.
Adversarial checks: class ignores preserve the other class; identical packed shapes do not classify norms or convolutions. Exact-name and regex targets work without class lookup.
Escape catalog: no C1–C4 patterns match.

Notes

Tests were inspected, not executed in this review. The PR reports that the external GLM oracle gate and GPU inference were not run.

🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)

@Burntgogi

Copy link
Copy Markdown
Contributor Author

Integrated upstream main at 5b626c3 in 0b89b0e. The sole conflict was prepare: upstream moved tensor planning into plan_tensors and changed the default to preserve stored BF16, with explicit Q8 opt-in. Kept that refactor/default/refusal behavior and passed the explicit module_class resolver at the moved planner call; no compatibility shim.

Exercised after the merge:

  • HF library: 84 passed; model-HF library: 23 passed; prepare: 4 passed, 1 existing external-corpus test ignored (BLOOMERY_QWEN38_HF unavailable). This includes upstream's stored-BF16 / Q8 opt-in / F32 refusal cases.
  • Compiled actual API consumer with the real checkpoint classifier: Linear, Embedding, exact name, regex, and ignore: [Embedding] each selected only the expected packed projection/embedding; unmatched classes, norm and convolution stayed refused. Temporary example removed.
  • Both packages' all-target clippy, workspace fmt check, comment and architecture checks passed.

The existing class-target docs remain accurate; this merge introduces no further public contract. New-head upstream CI is pending; the previous green/approval was for 105c3f7. No GPU run or new GGUF quant-format support is claimed.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the last review: no must-fix or non-blocking items were open.

Verdict: CLEAN: The upstream merge preserves explicit class resolution and the stored-format refusal paths.

Checked

Duplicates: 0 new PR-specific definitions since the last review; dryer reported 0 pairs. The extracted planner and new planning test already exist in base.
Scope: 58 incremental-diff paths match base exactly. The remaining two contain the existing API migration and the moved resolver argument.
Complexity: crapper flagged no changed functions. Planning remains load-time work.
Silent failure: 3 paths followed. Missing class metadata leaves packed weights refused, and mapped consumers return the reason. Unsupported F32 matrices and unquantized expert stacks return named errors before shard writing.
Tests: static mutation review. Dropping BF16 preservation at crates/model-hf/src/bin/prepare.rs:285, dropping the F32 refusal at line 279, or dropping the expert-stack refusal at line 272 fails the new planning assertions. Dropping F32 support in either output format fails the added normalization assertions. Existing class-selection tests still reject blanket matching.
Gates: no PR-specific pins, thresholds or test cases removed or relaxed.
Adversarial checks: the moved planner receives the classifier; class-only ignores preserve the other class; name selectors remain independent of metadata. A single-expert plain stack still reaches the stack refusal. Stored BF16 values still pass the finite-value check.
Escape catalog: no C1–C4 patterns match the PR-specific changes.

Notes

Tests were inspected, not executed in this review. The PR reports that GPU inference and the external GLM oracle gate were not run.

🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)

@Burntgogi

Copy link
Copy Markdown
Contributor Author

Exact-head upstream T0 for 0b89b0e has now passed (23m52s): https://github.com/midagedev/bloomery/actions/runs/39595869472/job/119207897131 . This is the conflict-resolved head, not the earlier 105c3f7 green. The post-merge local API/test/type/format/comment/architecture proof is in the preceding comment.

@midagedev
midagedev merged commit ba2057c into midagedev:main Oct 21, 2026
8 checks passed
@midagedev

Copy link
Copy Markdown
Owner

Merged as ba2057c on main. Main had moved, so I resolved four conflicts in crates/model-hf as unions (main's neg_exp and the TP rank's part imports beside your module_class).

Gates on the box, fixture tier, just affected origin/main (29 items): green. gate-gpu-glm5next-tp hit an out-of-memory error from another job on the same card once, and was green when run alone.

Thank you for the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model The CPU engine and model files: crates/model, models, hf, gguf, tokenizer, jinja

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants