ggml-org/llama.cpp

Pull Requests Commits

metal : fix memory leaks due to missing autoreleasepools (#27758)

nikwen committed 3 days ago

Verified 915dc6d3

llama : add --n-cpu-ffn option (#26622)

John-194 committed 3 days ago

Verified c5fc7e34

qwen4exp: give the PLE conv history its own mirrored recurrent row

danielhanchen committed 4 days ago

bf539602

kv-cache: check the mirrored slot layout on a whole-context restore too

danielhanchen committed 4 days ago

d1fe507f

kv-cache: clear the cache once when restoring a whole context

danielhanchen committed 4 days ago

f43070aa

qwen4exp: keep the indexer cache in step across server slots

danielhanchen committed 4 days ago

8b3ed0a4

models : support nanbeige4.2-3B (#27730)

zqlcode committed 4 days ago

Verified d7a20741

qwen4exp: double the Q split granularity for tensor parallelism

danielhanchen committed 4 days ago

6c5afc86

llama: give the qwen4exp indexer cache its own tensor names

danielhanchen committed 4 days ago

fbe17732

convert: fix the qwen4exp lint and type-check failures

danielhanchen committed 4 days ago

213df585

hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchronous backend (#26501)

max-krasnyansky committed 4 days ago

Verified 192067b7

qwen4exp: include llama-impl.h explicitly for llama_mul_mat_hadamard

danielhanchen committed 4 days ago

0b19188e

llama: fix the qwen4exp PLE history seq_rm(-1) iterator invalidation and the fatal-warning build

danielhanchen committed 4 days ago

24ea62df

llama: segment the qwen4exp fused QKV for tensor split

danielhanchen committed 4 days ago

5674c73a

quantize: dequantize and quantize large tensors in row bands

danielhanchen committed 4 days ago

ef9fa1ba

qwen4exp: drop an unused variable that breaks -Werror builds

danielhanchen committed 4 days ago

6a69a0c1

llama: give qwen4exp a large-graph node budget

danielhanchen committed 4 days ago

c52ed2a0

qwen4exp: support a quantized KV cache in the QSA attention path

danielhanchen committed 4 days ago

0ac4b180

qwen4exp: tidy comments and simplify image token read

danielhanchen committed 4 days ago

d4a943f9

llama: make the qwen4exp PLE n-gram history per context and serialise it

danielhanchen committed 4 days ago

d22d2be2

llama: save and restore the qwen4exp indexer KV cache

danielhanchen committed 4 days ago

cfbdc0a5

llama: add token ID tracking to KV cell (#27762)

ngxson committed 4 days ago

Verified 925e1179

ui: Move Settings and MCP Servers routes to dialog-based views (#27744)

allozaur committed 4 days ago

Verified 539f2452

ui: Update Dialog component styling (#27743)

allozaur committed 4 days ago

Verified 0379a19f

vulkan: warptiles currently assume warp sizes <= 64, clamp to work around larger warps (#27726)

0cc4m committed 4 days ago

Verified 5e6a37cb

llama: hold the qwen4exp indexer cache in a new llama_memory_hybrid_idx

danielhanchen committed 4 days ago

035e2273

llama: keep the qwen4exp top-k attention mask arch-local

danielhanchen committed 4 days ago

7fee670a

Implemented vulkan cross_entropy_loss and cross_entropy_loss_back (#27216)

PranavUttarkar committed 4 days ago

Verified bf942164

rpc : implement event and async backend APIs (#18626)

rgerganov committed 4 days ago

Verified d0132a68

qwen4exp: support a non-unified KV cache in QSA

danielhanchen committed 4 days ago

bea3b12d

Newer Older