ggerganov/llama.cpp

Pull Requests Commits

build : add compile option to force use of MMQ kernels

ggerganov committed 1 year ago

Verified 49af767f

cuda : add CUDA_USE_TENSOR_CORES and GGML_CUDA_FORCE_MMQ macros

ggerganov committed 1 year ago

Verified a4e15a36

cuda : remove duplicated cuBLAS GEMM code

ggerganov committed 1 year ago

Verified 4c6744b5

cuda : fine-tune >= VOLTA params + use MMQ only for small batches

ggerganov committed 1 year ago

Verified a3c28439

cuda : add F32 sgemm branch

ggerganov committed 1 year ago

Verified 16b60dd7

cuda : new cublas gemm branch for multi-batch quantized src0

ggerganov committed 1 year ago

Verified 52af7826

cuda : prints wip

ggerganov committed 1 year ago

Verified 59d1232e

batched-bench : print params at start

ggerganov committed 1 year ago

Verified 6961c4bd

log : disable pid in log filenames

ggerganov committed 1 year ago

Verified cc448774

server : add parameter -tb N, --threads-batch N (#3584) (#3768)

cebtenzzre committed 1 year ago

Verified ad939626

server : do not block system prompt update (#3767)

ggerganov committed 1 year ago

Verified 1717521c

sync : ggml (conv ops + cuda MSVC fixes) (#3765)

ggerganov committed 1 year ago

Verified b2f7e04b

cmake : add missed dependencies (#3763)

kingsidelee committed 1 year ago

Verified abd21fc9

cuda : add batched cuBLAS GEMM for faster attention (#3749)

ggerganov committed 1 year ago

Verified 2b4ea35e

Add more tokenizer tests (#3742)

Galunid committed 1 year ago

Verified daab3d7f

metal : handle ggml_scale for n%4 != 0 (close #3754)

ggerganov committed 1 year ago

Verified 469c9add

Revert "make : add optional CUDA_NATIVE_ARCH (#2482)"

ggerganov committed 1 year ago

Verified e3932593

issues : separate bug and enhancement template + no default title (#3748)

monatis committed 1 year ago

Verified 9d029564

Update special token handling in conversion scripts for gpt2 derived tokenizers (#3746)

Galunid committed 1 year ago

Verified 69a67350

llama : remove token functions with `context` args in favor of `model` (#3720)

MarcusDunn committed 1 year ago

Verified 5be6c803

Fix baichuan convert script not detecing model (#3739)

Galunid committed 1 year ago

Verified 6336701c

make : add optional CUDA_NATIVE_ARCH (#2482)

awhill19 committed 1 year ago

Verified 96981f37

server : parallel decoding and multimodal (#3677)

ggerganov committed 1 year ago

Verified 438c2ca8

Add test for MPT tokenization (#3728)

goerch committed 1 year ago

Verified 9e70cc03

readme : remove unsupported node.js library (#3703)

ianscrivener committed 1 year ago

Verified 5a42a5f8

llama : validate special token ids are in range when loading GGUF model (#3635)

KerfuffleV2 committed 1 year ago

Verified a5e7dbd6

main : escape prompt for cfg_negative_prompt and consecutive inputs in main with interactive (#3623)

vvhg1 committed 1 year ago

Verified d3956aea

batched : add len CLI argument

ggerganov committed 1 year ago

Verified 22c69a27

CLBlast: Add outer loops over src0 for broadcasting in mulmat

shibe2 committed 1 year ago

465219b9

sampling : refactor init to use llama_sampling_params (#3696)

ggerganov committed 1 year ago

Verified d1031cf4

Older