gq : attempt at n-bit quantization
8f45628f
gq : add amax based method 3
b0a46fd2
gq : progress on method 2
da2de942
gq : method 4 (AVX2)
aa5506c0
gq : method 4 (ARM)
1fc11de6
gq : method 4 (AVX2 attempt) + method 5 (no min)
dae323c3
gq : method 5 (ARM)
349e9173
gpt-2 : model conversion for Q4_0 quantization
ff4c6532
ggml : Q4_0 quantization support (ggml_get_rows())
21514b72
gpt-2 : loading Q4_0 quantized model
ff54fda8
ggml : q4_0 quantization support
2219c11b
ggml : q4_1 quantization support (seems to work for bigger models)
5bdfce2a
gpt-2 : add gpt-2-quantize tool for quantizing f32 GPT-2 models
b0cab890
ggml : 4-bit quantization works (only scalar for now)
2f118888
gq : add method 6 (ARM)
b82c27fe
ggml : vectorized mad q4_0 (ARM)
2e75d8f1
ggml : vectorized quantize_row_q4_0 (ARM)
b0c22a48
ggml : simplify mad q4_0 (ARM)
3c757a46
ggml : minor indentations
e3ad879f
gpt-j : support for 4-bit quantized model inference
8abcab49
ggml : GGML_ASSERT() instead of assert() where appropriate
c21972cb
gpt : avoid ggml_transpose on model tensors (new models!)
4a56c5b9
gpt-2 : minor
904605c4
gpt-j : fix conversion for FP16 models (such as GPT-JT-6B)
441a38f4
ggml : add ggml_compute_forward_rope_f16()
53368280
gpt : fix memory usage computation
99af48e2
ggml : fix ggml_is_contiguous() to take into account blck size
6aae09ea
whisper : add whisper-qunatize tool
d0ac5ebf
whisper : add support for quantized models
37d427d4
whisper : mem usage based on model format type
e904a582
gpt : seems not worth to use FP16 for KV cache
63a8f622
gpt : support quantisation of f16 models files
519ce47e
ggml : fixes for rpi4
9881c2bc
whisper : add Q4_1 model sizes
a85bc0fd
ggml : add WASM SIMD for Q4_0
c4f14039
utils : print quantization histograms
331a8621
ggml : sync all changes from llama.cpp and whisper.cpp
154fcc3a
ggml : finalize the Q4_1 quantization for ARM_NEON
724c45d5
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub