ggerganov/llama.cpp

Pull Requests Commits

iq3_s_mult_shuffle: use lookup table on Metal

Iwan Kawrakow committed 1 year ago

31cecc87

iq3_s_mult_shuffle: use lookup table on CUDA

Iwan Kawrakow committed 1 year ago

93034df7

iq3_s_mult_shuffle: use new multiplier and cleanup

Iwan Kawrakow committed 1 year ago

6d15da1e

iq3_s_mult: remove SLOW_MULT option

Iwan Kawrakow committed 1 year ago

b1d753be

iq3_s_mult_shuffle: works on ARM_NEON and Metal

Iwan Kawrakow committed 1 year ago

a6a263b9

iq3_s_mult_shuffle: mult + shuffle based codebook

Iwan Kawrakow committed 1 year ago

b5874822

iq3_s_mult: scalar dot product

Iwan Kawrakow committed 1 year ago

b48bf8b4

iq3_s_mult: also CUDA

Iwan Kawrakow committed 1 year ago

f2c2bd6b

iq3_s_mult: back to blocks of 32

Iwan Kawrakow committed 1 year ago

e5e72562

iq3_s_mult: play with blocks of 16

Iwan Kawrakow committed 1 year ago

f4cb4eac

iq3_s_mult: another alternative multiplier

Iwan Kawrakow committed 1 year ago

dbe98dfe

iq3s_mult: quantization tuning

Iwan Kawrakow committed 1 year ago

8b713a98

iq3s_mult: ARM and Metal

Iwan Kawrakow committed 1 year ago

5b9c8785

iq3_s_mult: ifdef'd slow / fast versions

Iwan Kawrakow committed 1 year ago

b6402fa7

iq3_s_mult: alternative multiplier / bit twidling

Iwan Kawrakow committed 1 year ago

726aed30

iq3_s_mult: quantization tuning

Iwan Kawrakow committed 1 year ago

fe3c20b2

iq3_s_mult: Metal works - slower than lookup

Iwan Kawrakow committed 1 year ago

3000e0ac

iq3_s_mult: ARM_NEON works - 13 t/s

Iwan Kawrakow committed 1 year ago

bf90920f

Iwan Kawrakow committed 1 year ago

0fe9cd48

Iwan Kawrakow committed 1 year ago

e43e81a5

iq3_s_multiplier: CUDA and AVX2 works

Iwan Kawrakow committed 1 year ago

160aceca

Iwan Kawrakow committed 1 year ago

4c21c826

iq3_s(multiplier): use SIMD also in dequantize

Iwan Kawrakow committed 1 year ago

1cc7cb2b

Trying IQ3_S without a lookup table

Iwan Kawrakow committed 1 year ago

9c752ff0

Attempt to fix android build (#5752)

ikawrakow committed 1 year ago

Verified cb49e0f8

IQ4_XS: a 4.25 bpw quantization (#5747)

ikawrakow committed 1 year ago

Verified 0becb22a

cuda : replace remaining shfl_xor with calls to warp_reduce functions (#5744)

Engininja2 committed 1 year ago

Verified c24a2a6e

ggml-quants : fix avx2 iq1_s vec_dot when compiled with gcc (#5742)

Engininja2 committed 1 year ago

Verified 1f30b7a9

llama : fix defrag bugs + add parameter (#5735)

ggerganov committed 1 year ago

Verified 9d533a77

Makefile: use variables for cublas (#5689)

lindeer committed 1 year ago

Verified cbbd1efa

Older