pegainfer-project

Popular repositories Loading

1. pegainfer pegainfer Public

   Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

   Rust 719 115

2. kern kern Public

   A model-agnostic GPU runtime for shipping models as verified programs.

   Rust 89 7

3. pega-omni pega-omni Public

   OpenAI-compatible speech serving in Rust: streaming TTS and full-duplex voice over GPT-Live. 128 concurrent PersonaPlex sessions on one GPU.

   Rust 44 2

4. website website Public

   Documentation website for openinfer — Astro Starlight on Cloudflare Workers

   MDX 4 1

5. pegastore pegastore Public

   Placement-aware immutable large-object cache for AI workloads: multi-slot values across GPU / DRAM / SSD, shared across nodes.

   Rust 4

6. kern-k3-pp-prefill kern-k3-pp-prefill Public

   Kimi-K3 prefill as pipeline-parallel stages for kern: manifests, kernel sources, build recipes

   Cuda 4