Skip to content

Optimize MoE expert aggregation with ReduceSums - #264

Open
wine99 wants to merge 2 commits into
dev_backend_openvinofrom
moe-expert-reducesum
Open

Optimize MoE expert aggregation with ReduceSums#264
wine99 wants to merge 2 commits into
dev_backend_openvinofrom
moe-expert-reducesum

Conversation

@wine99

@wine99 wine99 commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

On top of the Qwen35 moe PR.

Optimizes MoE expert aggregation in the OpenVINO backend. It detects the final ADD in the structurally identified expert-view aggregation chain and translates it as a single ReduceSum over the expert dimension, instead of materializing individual expert views and chaining multiple additions.

image

@wine99
wine99 marked this pull request as ready for review July 27, 2026 06:19
@wine99
wine99 requested a review from cavusmustafa as a code owner July 27, 2026 06:19
@wine99

wine99 commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator Author

Test llama-simple -m Qwen3.6-35B-A3B-UD-Q4_K_S on Raptor Lake (i7-14700K 64GB Ubuntu24)
CPU: Almost no improvements 7.28 t/s -> 7.32 t/s (ggml cpu is 12.39 t/s, vulkan 7.53 t/s)
GPU: Cannot test due to no available GatherMatmulCompressed kernel on this platform

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant