ggml-org/llama.cpp

Pull Requests Commits

scripts : working curl pipe

ggerganov committed 2 years ago

Verified dab42893

scripts : rename to server-llm.sh

ggerganov committed 2 years ago

Verified f3947e1e

scripts : add deploy-server.sh

ggerganov committed 2 years ago

Verified 2f719c87

ggml : move FP16 <-> FP32 code to ggml-impl.h (#3861)

ggerganov committed 2 years ago

Verified 207b5190

Extend llama_kv_cache_seq_rm to allow matching any sequence (#3843)

KerfuffleV2 committed 2 years ago

Verified 6e08281e

make : remove unnecessary dependency on build-info.h (#3842)

cebtenzzre committed 2 years ago

Verified 2046eb43

llama : fix kv shift bug (#3835)

ggerganov committed 2 years ago

Verified 71a09da3

ggml : quantization refactoring (#3833)

ggerganov committed 2 years ago

Verified d69d777c

flake : update flake.lock for newer transformers version + provide extra dev shell (#3797)

Green-Sky committed 2 years ago

Verified ff3bad83

metal : try cwd for ggml-metal.metal if bundle lookup fails (#3793)

akx committed 2 years ago

Verified 82a6646e

issues : change label from bug to bug-unconfirmed (#3748)

ggerganov committed 2 years ago

Verified ba231e8a

convert : ignore tokens if their IDs are within [0, vocab_size) (#3831)

ggerganov committed 2 years ago

Verified 8a2f2fea

llama : allow quantizing k-quants to fall back when tensor size incompatible (#3747)

KerfuffleV2 committed 2 years ago

Verified bd6d9e20

llama : add option for greedy sampling with probs (#3813)

ggerganov committed 2 years ago

Verified ee1a0ec9

common : print that one line of the syntax help *also* to standard output (#3823)

HenkPoley committed 2 years ago

Verified 17746110

starcoder : add GPU offloading (#3827)

ggerganov committed 2 years ago

Verified fdee152e

speculative : ensure draft and target model vocab matches (#3812)

KerfuffleV2 committed 2 years ago

Verified 41aee4df

llama : correctly report GGUFv3 format (#3818)

cebtenzzre committed 2 years ago

Verified 6d459cbf

simple : fix batch handling (#3803)

tterrasson committed 2 years ago

Verified c8d6a1f3

cuda : improve text-generation and batched decoding performance (#3776)

ggerganov committed 2 years ago

Verified 2f9ec7e2

server : do not release slot on image input (#3798)

ggerganov committed 2 years ago

Verified 34b2a5e1

batched-bench : print params at start

ggerganov committed 2 years ago

Verified 6961c4bd

log : disable pid in log filenames

ggerganov committed 2 years ago

Verified cc448774

server : add parameter -tb N, --threads-batch N (#3584) (#3768)

cebtenzzre committed 2 years ago

Verified ad939626

server : do not block system prompt update (#3767)

ggerganov committed 2 years ago

Verified 1717521c

sync : ggml (conv ops + cuda MSVC fixes) (#3765)

ggerganov committed 2 years ago

Verified b2f7e04b

cmake : add missed dependencies (#3763)

kingsidelee committed 2 years ago

Verified abd21fc9

cuda : add batched cuBLAS GEMM for faster attention (#3749)

ggerganov committed 2 years ago

Verified 2b4ea35e

Add more tokenizer tests (#3742)

Galunid committed 2 years ago

Verified daab3d7f

metal : handle ggml_scale for n%4 != 0 (close #3754)

ggerganov committed 2 years ago

Verified 469c9add

Older