All pull requests

Pull requests

• 5080deploy: opt-in on-demand experts for short prefills (FREETOKEN_MOE_SMALL_PREFILL_TOKENS, off)

  #68·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: run the PLE graph fill on its own thread, submitted before the launch (fixes #65)

  #67·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: fix 11 bugs found re-reviewing the stack

  #64·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: merge upstream #224, OpenAI-compatible logprobs

  #63·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: merge upstream #504, per-request metrics under --enable-metrics

  #62·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: merge upstream #108, GET /v1/models/{model_id}

  #61·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: merge upstream #500, single-launch prefill buffer invalidation without hidden sync

  #60·gdevenyi opened Sep 23, 2026Owner·

  ·

• 5080deploy: merge upstream #525, host-resident GDN snapshot / prefix tier for hybrid GDN models

  #59·gdevenyi opened Sep 19, 2026Owner·

  ·

• 5080deploy: merge upstream #499, host-RAM KV tier for sparse-attention models (--kv-host-pages)

  #58·gdevenyi opened Sep 19, 2026Owner·

  ·

• 5080deploy: merge upstream #222, deliver AbortMsg when the client disconnects (streaming and not)

  #57·gdevenyi opened Sep 18, 2026Owner·

  ·

• 5080deploy: merge upstream #487, hoist system messages for system-first chat templates

  #56·gdevenyi opened Sep 18, 2026Owner·

  ·

• 5080deploy: merge upstream #459, validate FTW v1 indexes eagerly

  #55·gdevenyi opened Sep 18, 2026Owner·

  ·

• 5080deploy: merge upstream #198, reserve explicit KV pages during MoE auto-sizing

  #54·gdevenyi opened Sep 18, 2026Owner·

  ·

• 5080deploy: merge upstream #340, reserve the pool's dummy page in the --moe-cache-auto KV floor

  #53·gdevenyi opened Sep 18, 2026Owner·

  ·

• 5080deploy: merge upstream #476, --default-thinking-mode server flag

  #52·gdevenyi opened Sep 18, 2026Owner·

  ·

• 5080deploy: merge upstream #118, clamp admission to the actual KV pool

  #51·gdevenyi opened Sep 18, 2026Owner·

  ·

• 5080deploy: merge upstream #414, order the batch-memcpy probe against the current stream

  #50·gdevenyi opened Sep 18, 2026Owner·

  ·

• 5080deploy: merge upstream #466, tolerate coalesced msgpack frames in the zmq pull queues

  #49·gdevenyi opened Sep 18, 2026Owner·

  ·

• 5080deploy: merge upstream #505, preserve the pending hybrid checkpoint across prefill chunks

  #48·gdevenyi opened Sep 18, 2026Owner·

  ·

• feat(server): OpenAI protocol coverage -- sampling extras, n, echo/suffix, usage details, tokenize/detokenize/metrics

  #47·gdevenyi opened Sep 15, 2026Owner·

  ·

• fix(tokenizer): accept the OpenAI developer role (map to system for templates without it)

  #46·gdevenyi opened Sep 15, 2026Owner·

  ·

• feat(qwen4_exp): load-time per-tensor FP8 dense projections (W8A8 via _scaled_mm)

  #45·gdevenyi opened Sep 15, 2026Owner·

  ·

• test(qwen4_exp): compare chunked prefill to one-shot at bf16 tolerance

  #44·gdevenyi opened Sep 15, 2026Owner·

  ·

• feat(qwen4_exp): tensor parallelism for Qwen3.8-Flash-Next (offload backend)

  #43·gdevenyi opened Sep 15, 2026Owner·

  ·

• feat(kvcache): 8-bit DSV4 window/compressed KV behind --kv-cache-dtype

  #42·gdevenyi opened Sep 15, 2026Owner·

  ·