ggml-org/llama.cpp

Pull Requests Commits

also strip models memory margin from child processes

0cc4m committed 1 day ago

3c53be14

improve variable naming, fix style

0cc4m committed 1 day ago

a8d71b1b

improve memory_per_device map naming

0cc4m committed 1 day ago

f7cb12b7

fix model count exceeded check

0cc4m committed 1 day ago

2629c0a9

move llama_context_device_memory function to llama-ext.h

0cc4m committed 1 day ago

be7bd6b5

add server memory debug logging

0cc4m committed 1 day ago

57639cf0

use memory margin instead of total size limit, apply to each device separately

0cc4m committed 1 day ago

1bef0108

only set model memory_mb if not previously calculated

0cc4m committed 1 day ago

cbd9798f

use no_alloc to get memory requirements for model load

0cc4m committed 1 day ago

428772f3

estimate with to-be-loaded model size included

0cc4m committed 1 day ago

ae2094a3

server: add --models-memory-max parameter to allow dynamically unloading models when they exceed a memory size threshold

0cc4m committed 1 day ago

38cf3d32

webui: MCP Diagnostics improvements (#21803)

allozaur committed 1 day ago

Verified 227ed28e

Remove extra conditional check on debug mode. (#21798)

yomaytk committed 1 day ago

Verified bafae276

sycl: disable Q1_0 in backend and cleanup unused variables (#21807)

qnixsynapse committed 1 day ago

Verified 873c8256

mtmd: fix crash when sending image under 2x2 pixels (#21711)

mzsergiu committed 1 day ago

Verified 82764d8f

mtmd: qwen3 audio support (qwen3-omni and qwen3-asr) (#19441)

ngxson committed 1 day ago

Verified 21a49330

convert : force f16 or f32 on step3-vl conv weights (#21646)

CISC committed 1 day ago

Verified 1e9d771e

mtmd: add gemma 4 test (vision + audio) [no ci] (#21806)

ngxson committed 1 day ago

Verified aa4695c5

mtmd: add Gemma 4 audio conformer encoder support (#21421)

stephencox-ict committed 1 day ago

Verified 547765a9

fix: Proper messages rendering for "Show raw output" (#21672)

allozaur committed 2 days ago

Verified 9e209c5a

docs: add guide on how to add multimodal support (#21778)

ngxson committed 2 days ago

Verified 6313acbe

CUDA: skip compilation of superfluous FA kernels (#21768)

JohannesGaessler committed 2 days ago

Verified ff5ef827

mtmd : add MERaLiON-2 multimodal audio support (#21756)

SiruiHe committed 2 days ago

Verified 073bb2c2

opencl: add basic support for q5_k (#21593)

shaofeiqi committed 3 days ago

Verified af1127d3

TP: fix Qwen 3 Next data split (#21732)

JohannesGaessler committed 3 days ago

Verified 865ff06b

ggml : fix a few instances of missing GGML_TYPE_Q1_0 cases (#21716)

CISC committed 3 days ago

Verified 2b2cd57d

py : Bump typer to latest to fix huggingface_hub issue (#21701)

bartowski1182 committed 3 days ago

Verified 660386f6

CUDA: also store node->src ne/nb for graph equality (#21736)

am17an committed 3 days ago

Verified a29e4c0b

fix: Fix broken structured output when using $refs in json_schema (#21699)

Galunid committed 3 days ago

Verified b136b62c

hexagon: add support for linux on snapdragon (#21707)

tboinovski1 committed 3 days ago

Verified 81069a80

Older