Add challenge 109: Fused QKV Projection (Medium)

[claude]

Summary

Adds challenge 109: Fused QKV Projection — a Medium-difficulty inference kernel that fuses the three query/key/value projections into a single packed matmul and reshapes the result into the multi-head-attention layout.

• Input: hidden states x of shape [M, D] and packed weight W_qkv of shape [3D, D] (rows stacked as [W_Q; W_K; W_V])
• Output: three tensors Q, K, V, each of shape [num_heads, M, head_dim]
• Compute: qkv = x · W_qkv^T, split along the last dim, reshape to (M, num_heads, head_dim), transpose to (num_heads, M, head_dim)
• Performance test: LLaMA-2-7B-style attention projection — M = 512, num_heads = 32, head_dim = 128 (D = 4096, ~192 MB weight matrix)

Fused QKV is a canonical transformer-inference kernel: one matmul over the packed weight shares the input reads across Q/K/V and cuts launch overhead versus three separate GEMMs. The output layout requires solvers to think about non-trivial write patterns (interleaved head/token indexing) on top of a tiled matmul.

Validated end-to-end against the Tesla T4 runner with python scripts/run_challenge.py … --action submit: all functional and performance tests pass.

Test plan

• pre-commit run --all-files passes
• Example test produces the expected Q, K, V (hand-checkable identity/swap/scale weights)
• Reference reference_impl runs on all 10 functional cases locally (CPU torch)
• scripts/run_challenge.py … --action submit passes example + functional + performance tests on Tesla T4
• Directory layout, 5 required challenge.py methods, 6 starter files, and challenge.html sections all match the checklist in CLAUDE.md

🤖 Generated with Claude Code