ggml-org/llama.cpp

Pull Requests Commits

remove duplicated init calls

0cc4m committed 9 days ago

5a746223

cleanup unused variable

0cc4m committed 9 days ago

79210e30

precompute name->buft map, map GPU host types to CPU buft

0cc4m committed 9 days ago

84c4214b

move model memory estimation to subprocess

0cc4m committed 9 days ago

dbc5f7ec

extract duplicated check into helper function

0cc4m committed 9 days ago

384a495a

replace device memory map with buft memory map. Use llama_get_memory_breakdown

0cc4m committed 9 days ago

997491a6

cont : clean-up

ggerganov committed 9 days ago

a35afd50

also strip models memory margin from child processes

0cc4m committed 9 days ago

3046b885

improve variable naming, fix style

0cc4m committed 9 days ago

216aaf1a

improve memory_per_device map naming

0cc4m committed 9 days ago

ff41b3db

fix model count exceeded check

0cc4m committed 9 days ago

0e2f08a5

move llama_context_device_memory function to llama-ext.h

0cc4m committed 9 days ago

669948ce

add server memory debug logging

0cc4m committed 9 days ago

09d8eb95

use memory margin instead of total size limit, apply to each device separately

0cc4m committed 9 days ago

c749b688

only set model memory_mb if not previously calculated

0cc4m committed 9 days ago

4ed48154

use no_alloc to get memory requirements for model load

0cc4m committed 9 days ago

6178b875

estimate with to-be-loaded model size included

0cc4m committed 9 days ago

340c8671

server: add --models-memory-max parameter to allow dynamically unloading models when they exceed a memory size threshold

0cc4m committed 9 days ago

f38c4f94

server: fix non-bound n_discard value (ctx shifting) (#24786)

ngxson committed 9 days ago

Verified 159d093a

ggerganov committed 9 days ago

5fd2dc2c

ggml : bump version to 0.15.2 (ggml/1548)

ggerganov committed 9 days ago

1868af13

pi : remove docs from system prompt (#24791)

ggerganov committed 9 days ago

Verified 5bd21b85

server : consolidate slot selection into get_available_slot (#24755)

ggerganov committed 9 days ago

Verified 80452d65

ggml-cpu: support K tails in power10 Q8/Q4 MMA matmul (#24753)

shalinib-ibm committed 9 days ago

Verified 8141e730

mtmd: add batching support for internvl (#24775)

ngxson committed 9 days ago

Verified db52540f

Ggml/cuda col2im 1d (#24417)

ServeurpersoCom committed 9 days ago

Verified 3a3edc9a

server: add "X-Accel-Buffering": "no" header to streaming endpoints (#24774)

regunakyle committed 9 days ago

Verified 40f3aafc

mtmd: add batching for mtmd-cli, add video tests (#24778)

ngxson committed 9 days ago

Verified a6b3260a

cmake : fix ui build with read-only source (#24752)

o7si committed 9 days ago

Verified 32eddaf2

mtmd: refactor llava-uhd overview image handling (always use ov_img_first) (#24769)

ngxson committed 9 days ago

Verified 060ce1bf

Older