Skip to content

feat: add Ollama GGUF runtime support - #17

Open
FenjuFu wants to merge 1 commit into
iflytek:mainfrom
FenjuFu:feat/ollama-gguf
Open

feat: add Ollama GGUF runtime support#17
FenjuFu wants to merge 1 commit into
iflytek:mainfrom
FenjuFu:feat/ollama-gguf

Conversation

@FenjuFu

@FenjuFu FenjuFu commented Aug 12, 2026

Copy link
Copy Markdown
Member

Summary

  • add a dependency-free GGUF v2/v3 metadata parser that never loads tensor weights and recognizes Q4_0, Q4_K_M, Q5_K_M, Q6_K, and Q8_0
  • add local/remote Ollama REST support for server discovery, model listing, progress-aware pulls, loading, and deterministic generation
  • add a CLI for GGUF inspection/discovery/import and Ollama-managed models
  • keep local GGUF, Ollama-managed storage, and existing Safetensors training checkpoints explicitly separate
  • document the workflow in English and Chinese and add a Python 3.10-3.12 CI matrix

Repository scope

Domux currently has no application UI or model-management service. This PR provides a reusable CLI/library entry point instead of introducing an unrelated UI framework.

Validation

  • python -m unittest discover -s tests -v — 6 passed
  • python -m compileall -q domux_runtime tests
  • Ruff check and format — passed
  • git diff --check — passed

Partially addresses #8 (criteria 1, 3, 5, 6). The UI upload path (criterion 2) and the in-place weight-loader memory optimization (criterion 4) are intentionally out of scope here — see the scope note above and the follow-up comment on #8.

Signed-off-by: FenjuFu <92919259+FenjuFu@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant