ggerganov/llama.cpp

Pull Requests Commits

readme : add API change notice

ggerganov committed 1 year ago

Verified fc5f3346

Merge branch 'master' into gg/per-layer-kv

ggerganov committed 1 year ago

Verified 680a99e7

train : fix #4227 (double free in examples/train-text-from-scratch/train-text-from-scratch.cpp) (#4351)

casavaca committed 1 year ago

Verified 81bc9214

server : recognize cache_prompt parameter in OAI API (#4347)

ggerganov committed 1 year ago

Verified 05cd6e50

llama : support quantum K cache (#4312)

ggerganov committed 1 year ago

Verified 1a1a1c38

common : fix compile warning

ggerganov committed 1 year ago

Verified caa92492

speculative : support `--color` (#4343)

stduhpf committed 1 year ago

Verified da5eaef1

grammar : pre-computed pieces + reserve mem + less string copies (#4330)

MarcusDunn committed 1 year ago

Verified 5f6e0c0d

llama : allow overriding GGUF metadata when loading model (#4092)

KerfuffleV2 committed 1 year ago

Verified 5aa365d8

sampling : custom samplers order (#4285)

MaggotHATE committed 1 year ago

Verified 52c8bc3c

swift : revert compiler checks for swift package (#4332)

kchro3 committed 1 year ago

Verified e4b76bbe

simple : update error message for KV cache check (#4324)

danbev committed 1 year ago

Verified 23b5e12e

swift : fix concatenation method to avoid invalid UTF8 stringfication (#4325)

ensan-hcl committed 1 year ago

Verified d208995c

swift : fix prompt tokenization logic (#4321)

ensan-hcl committed 1 year ago

Verified 5c9f90cb

grammar-parser : fix typo (#4318)

eltociear committed 1 year ago

Verified 4fa44e84

llama : update session save/load

ggerganov committed 1 year ago

Verified 66aaac98

common : add command-line arg to disable KV cache offloading

ggerganov committed 1 year ago

Verified e262947d

llama : remove mirrors, perform Device -> Host when partial offload

ggerganov committed 1 year ago

Verified c80b8a2b

llama : keep the KV related layers on the device

ggerganov committed 1 year ago

Verified c44bc1ee

llama : enable offload debug temporarily

ggerganov committed 1 year ago

Verified 1fa91a48

llama : offload for rest of the model arches

ggerganov committed 1 year ago

Verified 3d3e6bd0

llama : offload K shift tensors

ggerganov committed 1 year ago

Verified f3dbfb9f

llama : offload KV cache per-layer

ggerganov committed 1 year ago

Verified 986b3da7

Merge branch 'master' into per-layer-kv

ggerganov committed 1 year ago

Verified c294c78e

ggml : reuse ggml_get_n_tasks() in ggml_graph_plan() (#4308)

ggerganov committed 1 year ago

Verified fbbc4282

ggml : fix soft max out-of-bounds access (#4307)

ggerganov committed 1 year ago

Verified adf3de4f

server : fix OpenAI API `stop` field to be optional (#4299)

Mardak committed 1 year ago

Verified 33e171d1

py : add grammar to oai like api (#4294)

neph1 committed 1 year ago

Verified 6949b50d

llama : pad KV cache size (#4280)

ggerganov committed 1 year ago

Verified d7b800b8

llama : avoid using "optional" keyword (#4283)

ggerganov committed 1 year ago

Verified 5a7d3125

Older