llama.cpp
11d4e099 - iq3_s: PPL improvement

Commit

1 year ago

iq3_s: PPL improvement E.g., for a context of 4096 LLaMA-v2-7B goes to 5.1340 from 5.1653.

References

#5829 - IQ3_S improvements

Author

Kawrakow

Kawrakow

Parents

Files2

ggml-cuda.cu
ggml-quants.c