I audit open-source RL / post-training frameworks, Triton kernels and interpretability libraries for silent correctness bugs: wrong gradients, mis-normalized losses, off-by-one indexing, out-of-bounds kernel reads. Every fix comes with a minimal reproduction and a test that fails before and passes after.

44 merged PRs across 15 repositories (as of 2026-11).

---

RL & post-training frameworks

• modelscope/twinkle#334: chunked cross-entropy backward wrote gradients into the logits in place, so the default accuracy metric always read ≈ 0
• modelscope/twinkle#335: Liger fused CE normalized twice in mean mode and crashed under micro-batching
• modelscope/twinkle#336: InfoNCE hard-negative upsampling offset copied the positive in as a negative
• In review: verl #8363 (SGLang silently ignores logprobs_mode), #8364, #8365, #8366; Trinity-RFT #636 (SAPO gradient), #640 (JSD advantage points away from −∇JSD), #641 (k× gradient under the verl ≥ 0.8 engine); ROLL #509 (agg_loss ignores loss_mask)

Linear-attention kernels · fla-org/flash-linear-attention

• #1620: DeltaFormer backward walked chunks from the wrong origin, so every gradient was wrong when T is not a multiple of the chunk size
• #1618: gated DeltaNet's default beta was read out of bounds under GVA
• #1627: ABC read the last partial chunk out of bounds, corrupting the recurrent cache
• #1577, #1619, #1621, #1629: Comba reference decay order, Oja GVA guard, a test that never asserted, RWKV-7 single-layer crash

Interpretability & data attribution

• TransformerLens#1953: residual-stream decomposition on post-norm models (Gemma 2/3, OLMo 2, …); relative error 0.7 → 2e-7, checked across 38 architectures
• TransformerLens#1941: Jacobian-lens ablation now projects out the whole subspace instead of one token at a time
• bergson#565: MoE expert K-FAC covariances were normalized by all tokens instead of routed tokens (influence inflated by ~1/p); #567: learning-rate history off by one step
• delphi#182, #183, #184: explainer/scorer example leakage, IntruderScorer crash, stale KV cache in SurprisalScorer
• shap#5261, shapiq#643: DeepExplainer elementwise activations; CHII Monte Carlo weights

Inference-time compute, evolutionary search, model merging

• optillm#348: DeepConf aligned with the paper (4 discrepancies); #349: top-p removed the most likely tokens instead of the tail
• openevolve#524, #528: island migration and novelty-rejected programs left in the database
• mergekit#720, #721: inverted Procrustes rotation; padding leaking into MoE gate vectors
• LLaMEA#157, #159, pytorch_optimizer#660, EMI-Group evox (5) and evomo (6)

Evaluation

• modelscope/evalscope: BIRD-SQL, TableBench, DS-1000, TAT-QA, FinQA