ggerganov/llama.cpp

Pull Requests Commits

iq1_s: turn off SIMD implementation for QK_K = 64 (it does not work)

Iwan Kawrakow committed 2 years ago

f0cbb6dd

Q2_K: fixed bug in imatrix quantization for QK_K = 64

Iwan Kawrakow committed 2 years ago

47d52b2b

Make CUDA compile with QK_K = 64

Iwan Kawrakow committed 2 years ago

2540a290

QK_K = 64 tests pass on ARM_NEON and Metal

Iwan Kawrakow committed 2 years ago

de64e061

iq2_xs: attempt to fix AVX dot product for QK_K = 64

Iwan Kawrakow committed 2 years ago

28e6146c

WIP: make i-quants work for QK_K = 64

Iwan Kawrakow committed 2 years ago

13ba37f1

IQ4_XS: a 4.25 bpw quantization (#5747)

ikawrakow committed 2 years ago

Verified 0becb22a

cuda : replace remaining shfl_xor with calls to warp_reduce functions (#5744)

Engininja2 committed 2 years ago

Verified c24a2a6e

ggml-quants : fix avx2 iq1_s vec_dot when compiled with gcc (#5742)

Engininja2 committed 2 years ago

Verified 1f30b7a9

llama : fix defrag bugs + add parameter (#5735)

ggerganov committed 2 years ago

Verified 9d533a77

Makefile: use variables for cublas (#5689)

lindeer committed 2 years ago

Verified cbbd1efa

fix server hangs on empty prompt (#5733)

ngxson committed 2 years ago

Verified b11a93df

Adding IQ2_S and IQ2_M to complete coverage of the 2-3 bit quantization range (#5721)

ikawrakow committed 2 years ago

Verified a33e6a0d

CUDA: fix DEBUG_CUDA_MALLOC (#5729)

JohannesGaessler committed 2 years ago

Verified 47bb7b48

readme : update ui list (#5731)

guinmoon committed 2 years ago

Verified c4d7f817

[SYCL] Add support for soft_max ALiBi (#5639)

AidanBeltonS committed 2 years ago

Verified e849078c

unicode : reuse iterator (#5726)

ggerganov committed 2 years ago

Verified 67fd3313

server: CI fix trailing space (#5728)

phymbert committed 2 years ago

Verified 4804215c

server: CI tests reduce build matrix (#5725)

phymbert committed 2 years ago

Verified 8a533f0d

llama : fix Gemma rope type (#5691)

ggerganov committed 2 years ago

Verified 269de86b

flake.lock: Update

github-actions[bot] committed 2 years ago

c3937339

server: tests - slow inference causes timeout on the CI (#5715)

phymbert committed 2 years ago

Verified e3965cf3

server: docs - refresh and tease a little bit more the http server (#5718)

phymbert committed 2 years ago

Verified 8b350356

llama : refactor k-shift implementation + KV defragmentation (#5691)

ggerganov committed 2 years ago

Verified bf08e006

server : fix crash when system prompt is bigger than batch size (#5714)

compilade committed 2 years ago

Verified f7625019

ggml-quants : provide ggml_vqtbl1q_u8 for 64bit compatibility (#5711)

rgryta committed 2 years ago

Verified abbabc5e

make : fix nvcc version is empty (#5713)

kwin1412 committed 2 years ago

Verified f1a98c52

readme : add Msty to UI list (#5618)

ashokgelal committed 2 years ago

Verified 7d548a18

server: logs - unified format and --log-format option (#5700)

phymbert committed 2 years ago

Verified 930b1780

server: concurrency fix + monitoring - add /metrics prometheus compatible endpoint (#5708)

phymbert committed 2 years ago

Verified d52d7819

Older