ggerganov/llama.cpp

Pull Requests Commits

cleanup unused variable

0cc4m committed 5 days ago

6adf9643

precompute name->buft map, map GPU host types to CPU buft

0cc4m committed 5 days ago

32eff6ed

move model memory estimation to subprocess

0cc4m committed 5 days ago

4adfcb22

extract duplicated check into helper function

0cc4m committed 5 days ago

14d6d3b2

replace device memory map with buft memory map. Use llama_get_memory_breakdown

0cc4m committed 5 days ago

59ef4bfd

load directly from downloaded state

0cc4m committed 5 days ago

57544a6b

handle models that need to be downloaded before estimation

0cc4m committed 5 days ago

749682ae

cont : clean-up

ggerganov committed 5 days ago

5b04dbd4

also strip models memory margin from child processes

0cc4m committed 5 days ago

0467fede

improve variable naming, fix style

0cc4m committed 5 days ago

ae6a070b

improve memory_per_device map naming

0cc4m committed 5 days ago

8771ef48

fix model count exceeded check

0cc4m committed 5 days ago

5e07cfb9

move llama_context_device_memory function to llama-ext.h

0cc4m committed 5 days ago

c7707af3

add server memory debug logging

0cc4m committed 5 days ago

827b445e

use memory margin instead of total size limit, apply to each device separately

0cc4m committed 5 days ago

d13bc052

only set model memory_mb if not previously calculated

0cc4m committed 5 days ago

16600eab

use no_alloc to get memory requirements for model load

0cc4m committed 5 days ago

494503a0

estimate with to-be-loaded model size included

0cc4m committed 5 days ago

998c76a2

server: add --models-memory-max parameter to allow dynamically unloading models when they exceed a memory size threshold

0cc4m committed 5 days ago

747e008e

server : free draft/MTP resources on sleep to fix VRAM leak (#23461)

am17an committed 5 days ago

Verified 52fb93a2

server: re-inject subcommand when router spawns children under unified binary (#23442)

ServeurpersoCom committed 5 days ago

Verified c9021714

app : add batched-bench, fit-params, quantize & perplexity (#23459)

angt committed 5 days ago

Verified 1d7ab2b9

mtp: use inp_out_ids for skipping logit computation (#23433)

am17an committed 5 days ago

Verified 12e5d990

vocab : add Carbon-3B (HybridDNATokenizer) support (#23410)

kashif committed 5 days ago

Verified 7ea23ddf

doc: fix spec mtp typo (#23435)

ruixiang63 committed 5 days ago

Verified 2fc8d185

ui: Improve Git Hooks for UI development (#23403)

allozaur committed 5 days ago

Verified 5e932a1c

ggml : Check the right iface method before using the fallback 2d get (#23306)

TheBlueMatt committed 5 days ago

Verified 2754ce1b

llama-graph: fix null-buffer crash in llm_graph_input_attn_kv_iswa for SWA-only models (#23131)

ssfdre38 committed 5 days ago

Verified eeeaf618

hexagon: ssm-conv fix for large prompts (#23307)

tboinovski1 committed 5 days ago

Verified 0be84685

app : show version (#23426)

angt committed 5 days ago

Verified ce02093f

Older