ggerganov/llama.cpp

Pull Requests Commits

cuda : add TODO for calling cublas from kernel + using mem pool

ggerganov committed 2 years ago

Verified d798a17c

cuda : reduce mallocs in cublasGemmBatchedEx branch

ggerganov committed 2 years ago

Verified 27c34c01

cuda : add cublasGemmStridedBatchedEx for non-broadcasted cases

ggerganov committed 2 years ago

Verified 3d297c1a

cuda : add ROCm / hipBLAS cublasGemmBatchedEx define

ggerganov committed 2 years ago

Verified d4156690

Apply suggestions from code review

KerfuffleV2 committed 2 years ago

Verified 878aa4f2

cuda : batched cuBLAS GEMMs for src0 F16 and src1 F32 (attention ops)

ggerganov committed 2 years ago

Verified c13fcfbf

cuda : minor indentation

ggerganov committed 2 years ago

Verified 84d4ca0e

ggml : skip nops in compute_forward

ggerganov committed 2 years ago

Verified 8d8d54f8

batched : add NGL arg

ggerganov committed 2 years ago

Verified 6a30bf3e

cmake : add helper for faster CUDA builds

ggerganov committed 2 years ago

Verified 8fb1be64

make : add optional CUDA_NATIVE_ARCH (#2482)

awhill19 committed 2 years ago

Verified 96981f37

server : parallel decoding and multimodal (#3677)

ggerganov committed 2 years ago

Verified 438c2ca8

Add test for MPT tokenization (#3728)

goerch committed 2 years ago

Verified 9e70cc03

readme : remove unsupported node.js library (#3703)

ianscrivener committed 2 years ago

Verified 5a42a5f8

llama : validate special token ids are in range when loading GGUF model (#3635)

KerfuffleV2 committed 2 years ago

Verified a5e7dbd6

main : escape prompt for cfg_negative_prompt and consecutive inputs in main with interactive (#3623)

vvhg1 committed 2 years ago

Verified d3956aea

batched : add len CLI argument

ggerganov committed 2 years ago

Verified 22c69a27

CLBlast: Add outer loops over src0 for broadcasting in mulmat

shibe2 committed 2 years ago

465219b9

sampling : refactor init to use llama_sampling_params (#3696)

ggerganov committed 2 years ago

Verified d1031cf4

gguf : support big endian platform (#3552)

chenqiny committed 2 years ago

Verified 8cf19d60

server : fix uninitialized sampling context (close #3685)

ggerganov committed 2 years ago

Verified a0edf73b

ggml : fix rope + llama minor optimizations (#3560)

GermanAizek committed 2 years ago

Verified f439e506

convert : restore compat with old Falcon models (#3680)

cebtenzzre committed 2 years ago

Verified e78f3ef2

multimodal : add BakLLaVA conversion support (#3682)

monatis committed 2 years ago

Verified f3b25e40

llava : avoid segfault in case of non-existent mmproj file (#3674)

monatis committed 2 years ago

Verified 60abea97

readme : update hot topics

ggerganov committed 2 years ago

Verified 004797f6

speculative : bug fixes

ggerganov committed 2 years ago

Verified 4e82b2ea

speculative : add tree-based sampling example (#3624)

ggerganov committed 2 years ago

Verified 0e89203b

metal : implement q5_0 and q5_1 kernels (#3648)

jhen0409 committed 2 years ago

Verified c67fe68e

opencl : fix element-wise multiplication (#3656)

shibe2 committed 2 years ago

Verified 1117d066

Older