ggerganov/llama.cpp

Pull Requests Commits
simple : fixes
slaren committed 1 year ago
72e7ef4e
simple : add README.md
ggerganov committed 1 year ago
Verified 88451600
llama : llama.h formatting + comments
ggerganov committed 1 year ago
Verified 5a3369d8
parallel : add disabled experimental batch chunking in powers of two
ggerganov committed 1 year ago
Verified b2debf65
parallel : fix cases where the input prompts can overflow the batch
ggerganov committed 1 year ago
Verified ded9b43c
parallel : fix bug (extra BOS) + smaller token_prev array
ggerganov committed 1 year ago
ee1d670c
ggml : revert change to ggml_cpy, add ggml_cont_Nd instead (#3275)
slaren committed 1 year ago
Verified 1be2b8c1
train : make KQ_pos memory buffer permanent via dummy scale op
ggerganov committed 1 year ago
Verified 2f3a46fc
llama : disable MPI for now
ggerganov committed 1 year ago
Verified 54206962
ggml-cuda : add rope f16, restore performance with parallel decoding (#3272)
slaren committed 1 year ago
Verified e04dc519
simple : improve comments + free batch
ggerganov committed 1 year ago
Verified db0fc2da
simple : add parallel decoding support
ggerganov committed 1 year ago
Verified b377bf22
llama : improve llama_batch API + simplify parallel example
ggerganov committed 1 year ago
Verified addae65f
parallel : rename hot-plug to continuous-batching
ggerganov committed 1 year ago
Verified a1327c71
llama : fix n_kv to never become 0
ggerganov committed 1 year ago
Verified e1067efb
parallel : minor
ggerganov committed 1 year ago
Verified 7b7472ee
parallel : print misses on each request
ggerganov committed 1 year ago
6028879f
parallel : count cache misses
ggerganov committed 1 year ago
eed3fd42
parallel : remove question with short answers
ggerganov committed 1 year ago
Verified 8a9aca37
parallel : process system prompt once + configurable paramters + llama API
ggerganov committed 1 year ago
Verified 4b5f3cd6
parallel : remove new line from prompt
ggerganov committed 1 year ago
82e20e9b
llama : silence errors KV cache errors
ggerganov committed 1 year ago
Verified d37081ae
parallel : fix sequence termination criteria
ggerganov committed 1 year ago
Verified 16090a5d
parallel : try smaller batches when the KV cache is fragmented
ggerganov committed 1 year ago
Verified 806d397c
llama : fix cell_max logic + rename functions
ggerganov committed 1 year ago
Verified ddad2277
parallel : various improvements
ggerganov committed 1 year ago
Verified 36714e16
simple : fix token counting
ggerganov committed 1 year ago
Verified 467e3079
make : add parallel to build + fix static functions in llama.cpp
ggerganov committed 1 year ago
Verified 25bd2540
ggml-cuda : update rope implementation for parallel decoding (#3254)
slaren committed 1 year ago
Verified 7e2b9974
llama : fix worst case graph build
ggerganov committed 1 year ago
Verified daf4c6d3
Terms Of Use Privacy Policy Refund Policy Impressum