Repository navigation

[Architecture] Unify stage execution with loop-level scheduling and mixed batching - #118

Merged

bjf-frz merged 32 commits into

InferMatrixLab:mainfrom

LyxWxj:pr/PREFILL-separation

Oct 21, 2026

Merged

[Architecture] Unify stage execution with loop-level scheduling and mixed batching#118
bjf-frz merged 32 commits into
InferMatrixLab:mainfrom
LyxWxj:pr/PREFILL-separation

Commits on Oct 10, 2026
- Config, Engine, Scheduler, Runner
  LyxWxj
  committed
  6e62ed2
- SplitPrefillPolicy
  LyxWxj
  committed
  61df59d
- Merge branch 'main' into pr/PREFILL-separation
  LyxWxj
  committed
  f4332cd
- Merge branch 'main' into pr/PREFILL-separation
  LyxWxj
  committed
  b9d68ed
- Fix bugs that make atomic_prefill not be passed successfully from llm to ModelRunner, refactor the prefill_boundaries calculation
  LyxWxj
  committed
  ea9240b
Commits on Oct 11, 2026
- Prefill process advance, chunked prefill snapshot
  LyxWxj
  committed
  93c1e54
- Adaption split prefill for runner side
  LyxWxj
  committed
  fa085dd
- gather/scatter prefill states in model_runner
  LyxWxj
  committed
  86d47f9
- add prefill_submissions count in model_runner
  LyxWxj
  committed
  b426cdf
- split prefill supports preemption by rewinding the resident chunk before the KV snapshot
  LyxWxj
  committed
  d33faea
- add tests for cudagraph
  LyxWxj
  committed
  d1d4f92
- PD-disag for split prefill
  LyxWxj
  committed
  904412f
Commits on Oct 12, 2026
- Implement split-prefill scheduling and PD execution
  LyxWxj
  committed
  32a5166
- optimize the split-prefill's throuput
  LyxWxj
  committed
  fcdc1da
- Merge branch 'main' into pr/PREFILL-separation
  LyxWxj
  authored
  6a4629f
- clean, compact and simplify codes
  LyxWxj
  committed
  58f6352
- Merge branch 'main' into pr/PREFILL-separation
  LyxWxj
  committed
  f8969a3
- Fix scheduling after preemption resume
  LyxWxj
  committed
  aee09ce
Commits on Oct 13, 2026
Commits on Oct 14, 2026
- Merge main into pr/PREFILL-separation
  LyxWxj
  committed
  480ae3a
- adjust prefill buffers initialization condition,adjust PrefillMetadataPoll construction parameter naming
  LyxWxj
  committed
  4ddee45
Commits on Oct 15, 2026
- refactor: unify stage execution and optimize device data movement
  bjf-frz
  committed
  09561b8
Commits on Oct 16, 2026
- docs: document stage philosophy and translate architecture guides
  bjf-frz
  committed
  94d1577
Commits on Oct 20, 2026
- refactor: complete unified stage execution and consolidate tests and docs
  bjf-frz
  committed
  c3ce155
Commits on Oct 21, 2026
- fix: enable async speculative graphs and document serving benchmarks
  bjf-frz
  committed
  73ea302
- fix: guard Python examples and stabilize graph budget regression
  bjf-frz
  committed
  e2ee604

Commits

Commits on Oct 10, 2026

Commits on Oct 11, 2026

Commits on Oct 12, 2026

Commits on Oct 13, 2026

Commits on Oct 14, 2026

Commits on Oct 15, 2026

Commits on Oct 16, 2026

Commits on Oct 20, 2026

Commits on Oct 21, 2026