All pull requests

Pull requests

• Stub process-group interfaces in LayerWise optimizer unit tests

  #130·HJSang opened Oct 20, 2026Collaborator·

  ·

• [Optimizer] Add chunked Muon master weights

  #126·guapisolo opened Oct 15, 2026Collaborator·3/3·

  ·

• [Optimizer] Let optimizers choose the gradient used for the grad norm

  #125·guapisolo opened Oct 15, 2026Collaborator·2/3·

  ·

• [DDP] Bucket the non-tensor-parallel gradient all-reduce

  #124·guapisolo opened Oct 15, 2026Collaborator·1/3·

  ·

• Bound MoE dispatcher and chunked optimizer-offload memory peaks

  #122·HJSang opened Oct 15, 2026Collaborator·

  ·

• Cache the linear-offset tensor in weighted_bias_quick_geglu_impl

  #116·micahtyong opened Oct 12, 2026·

  ·

• fix: release forward-only outputs before the next microbatch

  #114·luohao opened Oct 9, 2026·

  ·
  1

• [port] fix: avoid duplicate offload of recomputed activations (upstream #7044)

  #113·HJSang opened Oct 6, 2026Collaborator·

  ·

• [port] Preserve HybridEP capacity across CUDA graph replay (upstream #6999)

  #112·HJSang opened Oct 6, 2026Collaborator·

  ·

• [port] moe(perf): Minor improvements of pre-GDR fused path (upstream #6658)

  #111·HJSang opened Oct 6, 2026Collaborator·

  ·

• [port] Add Kimi Delta Attention (KDA) variant (upstream #6556)

  #110·HJSang opened Oct 1, 2026Collaborator·

  ·
  2

• [port] Dense batch-invariant backend selection (partial NVIDIA#4871)

  #107·adrenaline21 opened Sep 28, 2026Collaborator·

  ·
  1

• Avoid Adam first-moment allocation when beta1 is zero

  #103·zianglih opened Sep 17, 2026·3 of 6 tasks·

  ·

• Fix out-of-bounds optimizer checkpoint padding

  #95·Shi-Dong opened Sep 7, 2026Collaborator·3 of 6 tasks·

  ·

• Fuse NVFP4 fake-QAT quantize-dequantize with CuTe DSL

  #88·zianglih opened Aug 24, 2026·6 tasks done·

  ·

• Fuse NVFP4 fake-QAT quantize-dequantize with CuTe DSL

  #87·ziang-and opened Aug 24, 2026·3 of 6 tasks·

  ·

• Support packed sequences in Gated Delta Net

  #85·yapdianang opened Aug 17, 2026·

  ·

• fix: restore PP process group on batched pipeline P2P ops

  #83·Laz4rz opened Aug 14, 2026·

  ·

• Make MoE router-weight placement explicit in experts

  #79·zianglih opened Aug 6, 2026·

  ·

• Extend FP32 MoE numerics to low-precision recipes and registered activations

  #77·zianglih opened Aug 5, 2026·5 of 10 tasks·

  ·

• Fix for lora dgrad flatten in _linear_dgrad_matmul

  #76·williammtan opened Aug 5, 2026·

  ·

• [WIP] Add NVFP4 fake QAT for grouped experts

  #75·zianglih opened Jul 31, 2026·6 tasks·

  ·
  1

• Add opt-in FlashInfer-fast activations for rollout parity

  #73·zianglih opened Jul 29, 2026·2 of 6 tasks·

  ·

• TOP dense parity for Qwen3 on Blackwell: flashinfer attention, batch-invariant kernel seam, non-reentrant recompute

  #70·adrenaline21 opened Jul 20, 2026Collaborator·

  ·

• fix(true-on-policy): match SGLang's row-linear k-tile reduction order

  #65·zihaow211 opened Jul 9, 2026·

  ·
  1