Repository navigation
Supported Model in README #60
|
Can you please update the README, specifying which model do you support? All LLMs from the hugging face Or specific architecture? Extended from this issue: #30 By the way, this is great library for quantizing model. |
Answered by
mobicham
Apr 23, 2024
Replies: 1 comment
|
Hi, I think it's quite clear. |
0 replies
Answer selected by
mobicham
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment

Hi, I think it's quite clear.
If a Hugging Face model is not on the list (llama, Mistral, Mixtral, Phi), use auto mode: https://github.com/mobiusml/hqq/?tab=readme-ov-file#auto-mode-1
Auto-mode basically tries to figure out how to quantize stuff automatically, but it's not guaranteed to work.