[hsliuustc0106]

What

The ASR latency table PR #104 listed under Not done, plus the decode-loop
anatomy that says where the remaining win actually lives. No behavior change:
two weight-gated scripts + a docs section.

• scripts/build_asr_eval.py — local eval set from hf-internal-testing/librispeech_asr_dummy (LibriSpeech, CC-BY-4.0): 10 real-speech clips (1–10 s) + built 64 s/158 s concatenations with references. GPU-free; needs pyarrow.
• scripts/bench_asr_knobs.py — benches the knobs the serving path leaves at mlx-audio defaults (checkpoint quant, prefill_step_size, chunk_duration, batch_size); WER-guarded medians, peak wired memory; --mode ab interleaves two configs for comparative claims.
• docs/speech.md — "ASR performance" section with the tables below and repro commands.

Why

Issue #68's perf frame had zero measured numbers. Question answered: is the
served default (Qwen3-ASR-1.7B-4bit, stock mlx-audio knobs) leaving easy
latency on the table? No — it is the optimum of the exposed space, and
the probe data redirects effort to code (task 5), not knobs.

Numbers (M4 24 GB, mlx 0.32.3 / mlx-audio 0.5.8, GPU-serial)

Ladder — median wall over 10 short clips, mean WER, peak wired:

checkpoint	wall	mean WER	peak
0.6B-4bit	0.29 s	0.031	1.31 GiB
0.6B-8bit	0.41 s	0.027	1.59 GiB
1.7B-4bit (served)	0.63 s	0.020	2.29 GiB
1.7B-6bit	0.79 s	0.011	2.69 GiB
1.7B-8bit	1.03 s	0.012	3.09 GiB

Long-audio (158 s clip): prefill ≤1024 vs 2048 — sequential sweep said −4 %,
interleaved A/B (3 reps, 6/6 pairs) reversed it: default wins by 1–3.5 %;
prefill 8192 genuinely costs +27 %; chunking 30–120 s is WER-worse at flat
latency; 30 s chunks × batch 4 = −20 % at 5× WER (0.076) — rejected.
0.6B-4bit = 7.2 s vs 12.8 s at WER 0.029 vs 0.016 — documented as the
2.2× fast-lane option, not a default swap.

Loop anatomy (per token ≈ 16–21 ms): one-token model forward ≈ 100 % of
cadence (sampler/Python sub-ms). Inside: 28 4-bit layers ~15 ms (~75 %,
~0.76 GiB reads at ~50 % of bandwidth floor — what a compiled decode loop
closes), QuantizedEmbedding.as_linear lm_head 5.2 ms (~25 %, already at
its 4-bit floor — bf16 would read 594 MiB), KV attention +4–5 ms at 4k
context. TTFT: tower bf16 0.15 s/1.6 s (on-the-fly tower 4-bit: WER same,
slower — rejected); CPU mel ×2 ≤ 90 ms, negligible.

Gates that ran

• ruff --select E9,F clean; py_compile both scripts; unittest discover 233 OK / 25 weight-gated skips.
• Bench script smoke-run from this branch (2-clip plan mode) reproduces 0.52 s / RTF 0.097 / weights 1.49 GiB.
• Methodology: forced-mx.synchronize() brackets; medians; WER guard ±0.005; GPU-serialized. Matmul probe at PR-open: 6.36 ms/iter (2048³ bf16) — quiet. Absolute walls carry the usual thermal-state caveat (a later probe pass read ~18 % slower cadence on identical config); every comparative claim rests on interleaved A/B, per CONTRIBUTING.

Not done

• TTFT/TPOT/ITL through the HTTP server (needs a running server; frame declared in asr/capabilities.py) — the tables here are model-level.
• Non-M4 chips; zh/multilingual eval (eval set is English-only).
• The compile/prefix-cache work itself (ASR decoder-style first: /v1/audio/transcriptions on qwen3_asr — prompt-prefix cache + TPOT/ITL doctrine #68 task 5) — this PR is the evidence base for it.

Overlap

#113 touches docs/speech.md (voice-cloning + TTS perf sections); this PR
appends a new ASR section at the file end. #113 is canonical for its hunks.