ggerganov/llama.cpp

Pull Requests Commits

metal : implement .get_alloc_size for the rest of the buffer types

ggerganov committed 5 hours ago

Verified 3bc8fd46

metal : add comments

ggerganov committed 6 hours ago

Verified 25e82c96

cont : add functions for the extra tensor sizes

ggerganov committed 6 hours ago

Verified 6752ba02

cont : improve, comments

ggerganov committed 6 hours ago

Verified 93685771

cont : use ggml_backend_buft_get_alloc_size

ggerganov committed 22 hours ago

Verified 400e0379

metal : take into account the actual allocated memory of the tensor

ggerganov committed 23 hours ago

Verified d29fb057

metal : remove mem pool implementation

ggerganov committed 1 day ago

Verified 59483466

metal : remove mem pool usage

ggerganov committed 1 day ago

Verified 29f59037

metal : refactor kernel loading (#15964)

ggerganov committed 1 day ago

Verified 55758b00

metal : allow ops to run concurrently (#15929)

ggerganov committed 1 day ago

Verified f161463a

metal : fix memory leaks (#15962)

ggerganov committed 1 day ago

Verified 84d7b2fc

ggml-zdnn: fix #15414, activate FP16 and BF16 acceleration and incorrect zTensor free (#15839)

taronaeo committed 1 day ago

Verified 40be5115

Add docker protocol support for llama-server model loading (#15790)

ericcurtin committed 1 day ago

Verified 4bf55492

context : remove redundant explicit casting to the same type (#15948)

haiyuewa committed 1 day ago

Verified f4e664f8

server : adjust prompt similarity thold + add logs (#15913)

ggerganov committed 2 days ago

Verified f088b6a8

Vulkan iGPU device selection overhaul and PCI ID API support (#15947)

0cc4m committed 2 days ago

Verified 304ac569

vulkan: Make device memory check more portable (#15939)

mbaudier committed 2 days ago

Verified 6c88ad8f

Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910)

NeoZhangJianyu committed 2 days ago

Verified 704d90c9

ggml-backend : add GGML_BACKEND_DEVICE_TYPE_IGPU device type (#15797)

slaren committed 2 days ago

Verified 360d6533

CUDA: larger SRAM reads for tile FA, AMD FP16 dot (#15927)

JohannesGaessler committed 2 days ago

Verified 0e6ff004

nitpick : correct MB to MiB (#15934)

ddh0 committed 2 days ago

Verified df082f56

ggml-cpu : add check for ARM MATMUL_INT8/i8mm support (#15922)

danbev committed 3 days ago

Verified 24a6734d

kleidiai: fix GGML_ASSERT(*cur_backend_id != -1) failed (#15614)

chaxu01 committed 3 days ago

Verified 2b3efea9

CANN: Disable acl_graph for prefill stage (#15933)

hipudding committed 3 days ago

Verified c0389dba

CUDA: Add `fastdiv` to `k_bin_bcast*`, giving 1-3% E2E performance (#15872)

ORippler committed 3 days ago

Verified 00681dfc

llama : support T5 models with unequal number of encoder-decoder layers (#15909)

DamonFool committed 3 days ago

Verified 4f658855

graph : support non-contiguous Q in build_attn_mha (#15908)

CISC committed 3 days ago

Verified 6ab397e1

ggml-cpu : fix padding in ggml_timestep_embedding (#15917)

danbev committed 3 days ago

Verified 9de447d9

metal : make the backend async (#15906)

ggerganov committed 3 days ago

Verified 0f0a3c28

ci : add caching for ROCm installation in release workflow (#15924)

danbev committed 4 days ago

Verified 33daece8

Older