ggml-org/ggml

Pull Requests Commits

ggml : finalize the Q4_1 quantization for ARM_NEON

ggerganov committed 3 years ago

Verified 724c45d5

ggml : sync all changes from llama.cpp and whisper.cpp

ggerganov committed 3 years ago

Verified 154fcc3a

utils : print quantization histograms

ggerganov committed 3 years ago

Verified 331a8621

ggml : add WASM SIMD for Q4_0

ggerganov committed 3 years ago

Verified c4f14039

whisper : add Q4_1 model sizes

ggerganov committed 3 years ago

Verified a85bc0fd

ggml : fixes for rpi4

ggerganov committed 3 years ago

Verified 9881c2bc

gpt : support quantisation of f16 models files

ggerganov committed 3 years ago

Verified 519ce47e

gpt : seems not worth to use FP16 for KV cache

ggerganov committed 3 years ago

Verified 63a8f622

whisper : mem usage based on model format type

ggerganov committed 3 years ago

Verified e904a582

whisper : add support for quantized models

ggerganov committed 3 years ago

Verified 37d427d4

whisper : add whisper-qunatize tool

ggerganov committed 3 years ago

Verified d0ac5ebf

ggml : fix ggml_is_contiguous() to take into account blck size

ggerganov committed 3 years ago

Verified 6aae09ea

gpt : fix memory usage computation

ggerganov committed 3 years ago

Verified 99af48e2

ggml : add ggml_compute_forward_rope_f16()

ggerganov committed 3 years ago

Verified 53368280

gpt-j : fix conversion for FP16 models (such as GPT-JT-6B)

ggerganov committed 3 years ago

Verified 441a38f4

ggerganov committed 3 years ago

Verified 904605c4

gpt : avoid ggml_transpose on model tensors (new models!)

ggerganov committed 3 years ago

Verified 4a56c5b9

ggml : GGML_ASSERT() instead of assert() where appropriate

ggerganov committed 3 years ago

Verified c21972cb

gpt-j : support for 4-bit quantized model inference

ggerganov committed 3 years ago

Verified 8abcab49

ggml : minor indentations

ggerganov committed 3 years ago

Verified e3ad879f

ggml : simplify mad q4_0 (ARM)

ggerganov committed 3 years ago

Verified 3c757a46

ggml : vectorized quantize_row_q4_0 (ARM)

ggerganov committed 3 years ago

Verified b0c22a48

ggml : vectorized mad q4_0 (ARM)

ggerganov committed 3 years ago

Verified 2e75d8f1

gq : add method 6 (ARM)

ggerganov committed 3 years ago

Verified b82c27fe

ggml : 4-bit quantization works (only scalar for now)

ggerganov committed 3 years ago

Verified 2f118888

gpt-2 : add gpt-2-quantize tool for quantizing f32 GPT-2 models

ggerganov committed 3 years ago

Verified b0cab890

ggml : q4_1 quantization support (seems to work for bigger models)

ggerganov committed 3 years ago

Verified 5bdfce2a

ggml : q4_0 quantization support

ggerganov committed 3 years ago

Verified 2219c11b

gpt-2 : loading Q4_0 quantized model

ggerganov committed 3 years ago

Verified ff54fda8

ggml : Q4_0 quantization support (ggml_get_rows())

ggerganov committed 3 years ago

Verified 21514b72

Older