ggerganov/llama.cpp

Pull Requests Commits

CUDA: use only 1 thread if fully offloaded (#2915)

JohannesGaessler committed 1 year ago

Verified 8185710a

readme : update hot topics

ggerganov committed 1 year ago

Verified 7eb41179

parallel : add disabled experimental batch chunking in powers of two

ggerganov committed 1 year ago

Verified b2debf65

llama : allow gguf RoPE keys to be overridden with defaults (#3240)

cebtenzzre committed 1 year ago

Verified a5661d7e

parallel : fix cases where the input prompts can overflow the batch

ggerganov committed 1 year ago

Verified ded9b43c

benchmark-matmult : do not use integer abs() on a float (#3277)

cebtenzzre committed 1 year ago

Verified 65c2c1c5

parallel : fix bug (extra BOS) + smaller token_prev array

ggerganov committed 1 year ago

ee1d670c

flake : Restore default package's buildInputs (#3262)

tpdns90321 committed 1 year ago

Verified 80834dae

ggml : revert change to ggml_cpy, add ggml_cont_Nd instead (#3275)

slaren committed 1 year ago

Verified 1be2b8c1

CI: FreeBSD fix (#3258)

alonfaraj committed 1 year ago

Verified a40f2b65

train : make KQ_pos memory buffer permanent via dummy scale op

ggerganov committed 1 year ago

Verified 2f3a46fc

llama : disable MPI for now

ggerganov committed 1 year ago

Verified 54206962

ggml-cuda : add rope f16, restore performance with parallel decoding (#3272)

slaren committed 1 year ago

Verified e04dc519

simple : improve comments + free batch

ggerganov committed 1 year ago

Verified db0fc2da

simple : add parallel decoding support

ggerganov committed 1 year ago

Verified b377bf22

llama : improve llama_batch API + simplify parallel example

ggerganov committed 1 year ago

Verified addae65f

examples : fix benchmark-matmult (#1554)

ggerganov committed 1 year ago

Verified d119c04c

parallel : rename hot-plug to continuous-batching

ggerganov committed 1 year ago

Verified a1327c71

llama : fix n_kv to never become 0

ggerganov committed 1 year ago

Verified e1067efb

parallel : minor

ggerganov committed 1 year ago

Verified 7b7472ee

parallel : print misses on each request

ggerganov committed 1 year ago

6028879f

parallel : count cache misses

ggerganov committed 1 year ago

eed3fd42

parallel : remove question with short answers

ggerganov committed 1 year ago

Verified 8a9aca37

parallel : process system prompt once + configurable paramters + llama API

ggerganov committed 1 year ago

Verified 4b5f3cd6

parallel : remove new line from prompt

ggerganov committed 1 year ago

82e20e9b

llama : silence errors KV cache errors

ggerganov committed 1 year ago

Verified d37081ae

parallel : fix sequence termination criteria

ggerganov committed 1 year ago

Verified 16090a5d

parallel : try smaller batches when the KV cache is fragmented

ggerganov committed 1 year ago

Verified 806d397c

llama : fix cell_max logic + rename functions

ggerganov committed 1 year ago

Verified ddad2277

parallel : various improvements

ggerganov committed 1 year ago

Verified 36714e16

Newer Older