IQ4_NL: 4-bit non-linear quants with blocks of 32 #5590
ikawrakow
merged
6 commits into
master
from
ik/iq4_nl_no_superblock
iq4_nl: squash commits for easier rebase
9b0d3a85
iq4_nl: Fix after merging with master
1d900212
iq4_nl: another fix after merging with master
e7b999c3
Use IQ4_NL instead of Q4_K when using k-quants is not possible
3fc45558
Fix typo that makes several tests fail
b376bbb2
It was the ggml_vdotq thing missed inside the brackets
daacf6ca
ggerganov
approved these changes
on 2024-02-20
Login to write a write a comment.
Login via GitHub