Skip to content

fix(qwen3_5): skip GDN input-projection fusion on mixed projection dt… - #182

Open
potoior wants to merge 1 commit into
NVIDIA:mainfrom
potoior:fix/gdn-mixed-dtype-fusion
Open

fix(qwen3_5): skip GDN input-projection fusion on mixed projection dt…#182
potoior wants to merge 1 commit into
NVIDIA:mainfrom
potoior:fix/gdn-mixed-dtype-fusion

Conversation

@potoior

@potoior potoior commented Aug 17, 2026

Copy link
Copy Markdown

…ypes

Anti-compressed NVFP4 checkpoints can quantize only some GDN input projections (e.g. in_proj_qkv/in_proj_z -> NVFP4 FP8 weights) while leaving others (in_proj_b/in_proj_a) FP16. fuse_gdn_input_projections checked only the first projection's quant type and then concat weights of all four projections, so these checkpoints crashed with 'Promotion for Float8 Types is not supported, attempted to promote Float8_e4m3fn and Half'.

Verify all four projections share the same weight dtype before fusing; when they differ, log a warning and keep the unfused path (which is already supported in forward()). Pure FP16 / pure NVFP4 models still fuse.

Verified: tensorrt-edgellm-export Qwen3.8-27B-NVFP4 --skip-audio completes, with 'GDN fusion skipped' warnings logged per layer.

What does this PR do?

Type of change: ?

Overview: ?

Usage

# Add a code snippet demonstrating how to use this

🚀 Pull Request Checklist

Thank you for contributing to TensorRT Edge-LLM! Before we review your pull request, please make sure the following items are complete.
Please also refer to Contributor guidelines for general guidelines.

✅ Pre-commit Checks

  • I have installed pre-commit by running pip install pre-commit.
  • I have installed the hooks with pre-commit install.
  • I have run the hooks manually with pre-commit run --all-files and fixed any reported issues.

🧪 Tests

  • Tests have been added or updated as needed.
  • All tests are passing.

📄 Documentation

  • Updated any necessary documentation

⚙️ Compatibility

  • The change is backward compatible

Additional Information

@potoior
potoior requested a review from a team August 17, 2026 07:15
…ypes

Anti-compressed NVFP4 checkpoints can quantize only some GDN input
projections (e.g. in_proj_qkv/in_proj_z -> NVFP4 FP8 weights) while
leaving others (in_proj_b/in_proj_a) FP16. fuse_gdn_input_projections
checked only the first projection's quant type and then concat weights
of all four projections, so these checkpoints crashed with
'Promotion for Float8 Types is not supported, attempted to promote
Float8_e4m3fn and Half'.

Verify all four projections share the same weight dtype before fusing;
when they differ, log a warning and keep the unfused path (which is
already supported in forward()). Pure FP16 / pure NVFP4 models still
fuse.

Verified: tensorrt-edgellm-export Qwen3.8-27B-NVFP4 --skip-audio
completes, with 'GDN fusion skipped' warnings logged per layer.

Signed-off-by: potoior <2986485901@qq.com>
@potoior
potoior force-pushed the fix/gdn-mixed-dtype-fusion branch from a3ac7b4 to e44d305 Compare August 17, 2026 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant