fix(gemma4): load mixed-quant GGUF checkpoints

[Poyo-Yon]

Summary

• read embedding, dense projection, and per-layer expert GGML types from the GGUF tensor table instead of assuming Q6_K/Q4_0
• store heterogeneous routed experts in fixed-width byte slots and pass the real expert stride through every vendored MMVQ quant type
• expose all 19 MMVQ block layouts to the native GGUF dense-layer dispatch
• keep the existing uniform Q4_0 path compatible and reject unsupported mixed-type CPU/hybrid or FTW conversion paths explicitly

Fixes #188.

The fixed-stride expert-bank approach is consistent with the design explored in the closed #102.

Tested on

• FreeToken base: cac247a (v0.1.3)
• GPU: NVIDIA GeForce RTX 5060 Ti 16 GB
• Driver: 616.92
• CPU: Intel Core i7-12700F
• RAM: 64 GB
• OS: WSL2 Ubuntu 24.04, Linux 6.18.33.2
• Python 3.12.3, PyTorch 2.11.0+cu130, CUDA 13.0

Checkpoints:

• unsloth/gemma-4-26B-A4B-it-GGUF
  • gemma-4-26B-A4B-it-UD-IQ4_XS.gguf
  • gemma-4-26B-A4B-it-UD-Q3_K_XL.gguf
• mradermacher/gemma-4-26B-A4B-it-qat-q4_0-unquantized-uncensored-heretic-v2-i1-GGUF
  • gemma-4-26B-A4B-it-qat-q4_0-unquantized-uncensored-heretic-v2.i1-IQ4_XS.gguf

Exact serve command (repeated with each checkpoint path):

ft serve \
  --model <checkpoint.gguf> \
  --host 127.0.0.1 \
  --port 1919 \
  --attention-backend triton \
  --moe-backend offload \
  --moe-cache-rate 0.2 \
  --memory-ratio 0.8 \
  --max-running-requests 1 \
  --max-seq-len-override 4096 \
  --max-prefill-length 512 \
  --num-tokens 4096 \
  --kv-reserve-tokens 4096 \
  --cuda-graph-max-bs 1

All three checkpoints loaded, captured the CUDA graph, completed prefill warm-up, and served an OpenAI-compatible chat completion. The deterministic prompt 17x23 returned 391 on each checkpoint.

Focused tests:

pytest -q tests/models/test_gemma4_gguf_quant.py tests/models/test_gemma4_gguf_rope.py
6 passed, 1 skipped

pytest -q \
  tests/moe/test_offload.py::test_dummy_expert_banks_follow_the_kernel_layout \
  tests/moe/test_offload.py::test_offload_cache_rebuild_resizes_and_preserves_sources \
  tests/moe/test_offload.py::test_set_bank_sources_locked_layer_requires_cpu_layer_ids \
  tests/moe/test_offload.py::test_copy_plan_skips_locked_layers_and_keeps_fused_path
4 passed

The full tests/moe/test_offload.py result was 23 passed and 1 environment-only failure: the isolated POC environment does not have FlashInfer installed, while that existing test explicitly selects the fi attention backend.