Repository navigation

sparse decode attention: CUDA graphs, P/D, asynchronous decode, token pipelines and preemption - #164

Open

0z5a wants to merge 36 commits into

InferMatrixLab:mainfrom

0z5a:pr157-stack-on-head

Open

sparse decode attention: CUDA graphs, P/D, asynchronous decode, token pipelines and preemption#164
0z5a wants to merge 36 commits into
InferMatrixLab:mainfrom
0z5a:pr157-stack-on-head

Commits on Oct 27, 2026
- sparse decode attention: per-token reference lifecycle for async self-speculation
  kunkunblueberry
  committed
  e2f9f73
Commits on Oct 28, 2026
Commits on Oct 29, 2026
- Merge sparse decode attention ordinary decode and synchronous speculation
  kunkunblueberry
  committed
  efe5bc7
- Batch equal-length sparse decode rows and retain scalar oracle
  
  0z5a
  authored and
  kunkunblueberry
  committed
  0aaab79
- Preserve scalar cuBLAS reduction shapes when batching reference rows
  
  0z5a
  authored and
  kunkunblueberry
  committed
  903e101
- Prepare batched views once before identical per-row bmm calls
  
  0z5a
  authored and
  kunkunblueberry
  committed
  a7859a5
- sparse decode attention: restore the dense-output contract in the batched path
  kunkunblueberry
  committed
  e4f8988
- benchmarks: dense-baseline context sweep and memory breakdown
  kunkunblueberry
  committed
  f292b76
Commits on Oct 31, 2026
Commits on Nov 1, 2026
Commits on Nov 2, 2026