llama.cpp
Possible solution to allow K-quants on models with n_vocab!=32000
#2148
Merged
Merged
Possible solution to allow K-quants on models with n_vocab!=32000 #2148
LostRuins
merged
3 commits into
ggml-org:master
from
LostRuins:kquant_vocab_fix
This allows LLAMA models that were previously incompatible with K qua…
18541688
ggerganov
approved these changes
on 2023-07-09
Fix indentation
048dca98
As an alternative, to avoid failing on Metal due to lack of Q8_0 supp…
fd9a2fdf
Login to write a write a comment.
Login via GitHub