You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please avoid creating issues regarding the following questions, as they might be closed without a response.
Kept up to date by the maintainers. Check here before opening an issue. See also the Roadmap and CONTRIBUTING.md.
Requirements
Q: What hardware and OS are supported?
x86_64 only, with an NVIDIA GPU from Ampere (RTX 30 series) up and driver r580+ (CUDA 13). Windows and Linux through the Desktop app; Linux through pip. macOS, AMD, DGX Spark (aarch64) are on the Roadmap.
Q: How much RAM do I need?
For MoE models the experts live in host RAM, so you need roughly the size of the expert weights in free RAM. For example Qwen/Qwen3.6-35B-A3B (bf16) needs about 70 GB; the NVFP4 checkpoint of the same model needs far less.
Q: Which models are supported?
See models.md. Other checkpoints of the same architectures usually work too. Support for GGUF quantization types and architectures is on the Roadmap.
Troubleshooting
Q: On Linux a model fails to start with fatal error: Python.h: No such file or directory.
Triton compiles a small C helper the first time it runs, so it needs a C compiler and the Python development headers. The Python that ships with a Linux distribution may come without the headers. Install them and start the model again; no reinstall is needed: sudo apt install gcc python3-dev (Fedora: sudo dnf install gcc python3-devel).
Note
Please avoid creating issues regarding the following questions, as they might be closed without a response.
Kept up to date by the maintainers. Check here before opening an issue. See also the Roadmap and CONTRIBUTING.md.
Requirements
Q: What hardware and OS are supported?
x86_64 only, with an NVIDIA GPU from Ampere (RTX 30 series) up and driver r580+ (CUDA 13). Windows and Linux through the Desktop app; Linux through
pip. macOS, AMD, DGX Spark (aarch64) are on the Roadmap.Q: How much RAM do I need?
For MoE models the experts live in host RAM, so you need roughly the size of the expert weights in free RAM. For example
Qwen/Qwen3.6-35B-A3B(bf16) needs about 70 GB; the NVFP4 checkpoint of the same model needs far less.Q: Which models are supported?
See models.md. Other checkpoints of the same architectures usually work too. Support for GGUF quantization types and architectures is on the Roadmap.
Troubleshooting
Q: On Linux a model fails to start with
fatal error: Python.h: No such file or directory.Triton compiles a small C helper the first time it runs, so it needs a C compiler and the Python development headers. The Python that ships with a Linux distribution may come without the headers. Install them and start the model again; no reinstall is needed:
sudo apt install gcc python3-dev(Fedora:sudo dnf install gcc python3-devel).