ggml-org/llama.cpp

Pull Requests Commits

llama : extend batch API to select which logits to output

ggerganov committed 2 years ago

Verified fa0e6778

fixes : speculative KV cache + llama worst-case graph

ggerganov committed 2 years ago

Verified 897caccd

parallel : disable hot-plug to avoid cache fragmentation

ggerganov committed 2 years ago

Verified 466b5138

parallel : example for serving multiple users in parallel

ggerganov committed 2 years ago

Verified 0161372b

speculative : fix KV cache management

ggerganov committed 2 years ago

Verified 1f17ea63

llama : apply K-cache roping for Falcon and Baichuan

ggerganov committed 2 years ago

Verified 7c1bdd0e

llama : add llama_kv_cache_shift_seq + no more context swaps

ggerganov committed 2 years ago

Verified 0cbf3bfe

metal : disable concurrency optimization

ggerganov committed 2 years ago

Verified 86c90e34

llama : more robust cell_max heuristic + wip shift

ggerganov committed 2 years ago

Verified f015b266

llama : extend llama_kv_cache API

ggerganov committed 2 years ago

Verified 4d76d762

llama : add cell_max heuristic for more efficient kv_cache

ggerganov committed 2 years ago

Verified 6952a460

llama : add new llama_decode() API that works with llama_batch

ggerganov committed 2 years ago

Verified 9f42e754

Merge branch 'master' into custom-attention-mask

ggerganov committed 2 years ago

Verified 58bb5110

llama : unified KV cache + batch inference API

ggerganov committed 2 years ago

Verified d29e7693

ci : switch cudatoolkit install on windows to networked (#3236)

Green-Sky committed 2 years ago

Verified 7ddf1855

CUDA: fix peer access logic (#3231)

JohannesGaessler committed 2 years ago

Verified ee66942d

metal : add rope_f16 kernel + optimize cpy kernels

ggerganov committed 2 years ago

Verified fad56936

ggml : ggml_rope now takes a vector with positions instead of n_past

ggerganov committed 2 years ago

Verified 1fb033fd

llama : replace ggml_diag_mask_inf with ggml_add (custom -inf mask)

ggerganov committed 2 years ago

Verified 3b4bab6a

tests : verify that RoPE is "additive"

ggerganov committed 2 years ago

Verified c5df72e8

CUDA: enable peer access between devices (#2470)

JohannesGaessler committed 2 years ago

Verified 111163e2

llama.cpp : show model size and BPW on load (#3223)

slaren committed 2 years ago

Verified 8b428c9b

CUDA: fix scratch malloced on non-main device (#3220)

JohannesGaessler committed 2 years ago

Verified 578d8c8f

Enable BUILD_SHARED_LIBS=ON on all Windows builds (#3215)

IsaacDynamo committed 2 years ago

Verified b541b4f0

Enable build with CUDA 11.0 (make) (#3132)

Microflame committed 2 years ago

Verified 5dbc2b32

Fixing the last deviations from sentencepiece indicated by test-tokenizer-1 (#3170)

goerch committed 2 years ago

Verified b08e75ba

examples : add compiler version and target to build info (#2998)

cebtenzzre committed 2 years ago

Verified e6616cf0

check C++ code with -Wmissing-declarations (#3184)

cebtenzzre committed 2 years ago

Verified 3aefaab9

fix build numbers by setting fetch-depth=0 (#3197)

cebtenzzre committed 2 years ago

Verified 69eb67e2

llama : add support for StarCoder model architectures (#3187)

wsxiaoys committed 2 years ago

Verified 4fe09dfe

Newer Older