PR #4897 2-bit quantizations

2-bit quantizations #4897

ikawrakow merged 5 commits into master from ik/quantize-iq2

imatrix: load

e9372e40

imatrix: WIP

8da2b25b

imatrix: Add Q2_K quantization

75f4cbf2

imatrix: also guard against Q2_K_S quantization without importance ma…

d5598f7e

imatrix: guard even more against low-bit quantization misuse

f342143e

ggerganov commented on 2024-01-12

ggerganov approved these changes on 2024-01-13

ikawrakow merged 147b17ac into master 2 years ago

ikawrakow deleted the ik/quantize-iq2 branch 2 years ago

Reviewers

ggerganov

Assignees

No one assigned

Labels

None yet

Milestone

No milestone