ggml-org/llama.cpp

Pull Requests Commits

remove duplicated init calls

0cc4m committed 6 days ago

82403fdc

cleanup unused variable

0cc4m committed 6 days ago

5329e5b1

precompute name->buft map, map GPU host types to CPU buft

0cc4m committed 6 days ago

739e9a61

move model memory estimation to subprocess

0cc4m committed 6 days ago

5fa4f2df

extract duplicated check into helper function

0cc4m committed 6 days ago

a17db48c

replace device memory map with buft memory map. Use llama_get_memory_breakdown

0cc4m committed 6 days ago

cc834a42

load directly from downloaded state

0cc4m committed 6 days ago

1fe77758

handle models that need to be downloaded before estimation

0cc4m committed 6 days ago

0c664db3

cont : clean-up

ggerganov committed 6 days ago

28b19990

also strip models memory margin from child processes

0cc4m committed 6 days ago

3db9f096

improve variable naming, fix style

0cc4m committed 6 days ago

ab31e3ed

improve memory_per_device map naming

0cc4m committed 6 days ago

9ce1c50c

fix model count exceeded check

0cc4m committed 6 days ago

e7fd20ab

move llama_context_device_memory function to llama-ext.h

0cc4m committed 6 days ago

4e6c4822

add server memory debug logging

0cc4m committed 6 days ago

4caabb47

use memory margin instead of total size limit, apply to each device separately

0cc4m committed 6 days ago

bfd8ca34

only set model memory_mb if not previously calculated

0cc4m committed 6 days ago

f59618aa

use no_alloc to get memory requirements for model load

0cc4m committed 6 days ago

003a9c5e

estimate with to-be-loaded model size included

0cc4m committed 6 days ago

37db40e4

server: add --models-memory-max parameter to allow dynamically unloading models when they exceed a memory size threshold

0cc4m committed 6 days ago

75917152

llama : do not skip iGPU when only RPC devices are present (#23868)

rgerganov committed 6 days ago

Verified 1738129b

server: in SSE mode, send HTTP headers when slot starts (#23884)

ngxson committed 6 days ago

Verified 0821c5fc

ggml-webgpu: Check earlier for WebGPU required features (#23879)

reeselevine committed 6 days ago

Verified 151f3a98

ggml-webgpu: add q4_0/q8_0 SET_ROWS (#23760)

reeselevine committed 6 days ago

Verified b22da258

server-bench : add speed-bench for speculative decoding benchmarking (#23869)

ruixiang63 committed 6 days ago

Verified 689a9a47

app: add llama update self updater (#23865)

ServeurpersoCom committed 6 days ago

Verified 5a46b46a

ui: handle audio/vnd.wave as audio WAV file (#23754)

ValdikSS committed 6 days ago

Verified 22d66b56

vocab : support tokenizer for LFM2.5-8B-A1B (#23826)

tdakhran committed 6 days ago

Verified 2084434e

graph : ensure DS32 kq_mask_lid is F32 (#23864)

CISC committed 6 days ago

Verified 764f1e64

server: remove obsolete scripts (#23870)

ngxson committed 6 days ago

Verified b5f52280

Older