ggml-org/llama.cpp

Pull Requests Commits

llama : remove memory_f16 and kv_f16 flags

ggerganov committed 2 years ago

Verified af99c6fb

cuda : add comment

ggerganov committed 2 years ago

Verified 4adb1d69

metal : use mm kernel only for quantum KV cache

ggerganov committed 2 years ago

Verified dd86df82

llama-bench : support type_k/type_v

slaren committed 2 years ago

903167a7

cuda : add F32 -> Q4_0 and F32 -> Q4_1 copy kernels

ggerganov committed 2 years ago

Verified b2acedeb

ggerganov committed 2 years ago

Verified e8457c90

metal : add F32 -> Q4_1 copy kernel

ggerganov committed 2 years ago

Verified 6b58ae98

metal : add F32 -> Q4_0 copy kernel

ggerganov committed 2 years ago

Verified 9d69ecc0

llama : fix build

ggerganov committed 2 years ago

Verified 7864a2cd

llama : pass KV cache type through API

ggerganov committed 2 years ago

Verified 3ce30e07

cuda : use mmv kernel for quantum cache ops

ggerganov committed 2 years ago

Verified b881f630

cuda : add F32 -> Q8_0 copy kernel

ggerganov committed 2 years ago

Verified a1bf6c09

metal : add F32 -> Q8_0 copy kernel

ggerganov committed 2 years ago

Verified bcfebf24

llama : support quantum K cache (wip)

ggerganov committed 2 years ago

Verified d04ee928

llama : update session save/load

ggerganov committed 2 years ago

Verified 66aaac98

common : add command-line arg to disable KV cache offloading

ggerganov committed 2 years ago

Verified e262947d

llama : remove mirrors, perform Device -> Host when partial offload

ggerganov committed 2 years ago

Verified c80b8a2b

llama : keep the KV related layers on the device

ggerganov committed 2 years ago

Verified c44bc1ee

llama : enable offload debug temporarily

ggerganov committed 2 years ago

Verified 1fa91a48

llama : offload for rest of the model arches

ggerganov committed 2 years ago

Verified 3d3e6bd0

llama : offload K shift tensors

ggerganov committed 2 years ago

Verified f3dbfb9f

llama : offload KV cache per-layer

ggerganov committed 2 years ago

Verified 986b3da7

Merge branch 'master' into per-layer-kv

ggerganov committed 2 years ago

Verified c294c78e

ggml : reuse ggml_get_n_tasks() in ggml_graph_plan() (#4308)

ggerganov committed 2 years ago

Verified fbbc4282

ggml : fix soft max out-of-bounds access (#4307)

ggerganov committed 2 years ago

Verified adf3de4f

server : fix OpenAI API `stop` field to be optional (#4299)

Mardak committed 2 years ago

Verified 33e171d1

py : add grammar to oai like api (#4294)

neph1 committed 2 years ago

Verified 6949b50d

llama : pad KV cache size (#4280)

ggerganov committed 2 years ago

Verified d7b800b8

llama : avoid using "optional" keyword (#4283)

ggerganov committed 2 years ago

Verified 5a7d3125

llama : support optional tensors (#4283)

ggerganov committed 2 years ago

Verified d5a1cbde

Older