ggml-org/llama.cpp

Pull Requests Commits

load directly from downloaded state

0cc4m committed 11 days ago

da1f1688

handle models that need to be downloaded before estimation

0cc4m committed 11 days ago

884901f0

cont : clean-up

ggerganov committed 11 days ago

01dd3934

also strip models memory margin from child processes

0cc4m committed 11 days ago

972813c2

improve variable naming, fix style

0cc4m committed 11 days ago

b440ee05

improve memory_per_device map naming

0cc4m committed 11 days ago

f24011f2

fix model count exceeded check

0cc4m committed 11 days ago

f4a384b4

move llama_context_device_memory function to llama-ext.h

0cc4m committed 11 days ago

f750bae2

add server memory debug logging

0cc4m committed 11 days ago

527c91ac

use memory margin instead of total size limit, apply to each device separately

0cc4m committed 11 days ago

3c815b36

only set model memory_mb if not previously calculated

0cc4m committed 11 days ago

18163c41

use no_alloc to get memory requirements for model load

0cc4m committed 11 days ago

af28cd24

estimate with to-be-loaded model size included

0cc4m committed 11 days ago

e6468c17

server: add --models-memory-max parameter to allow dynamically unloading models when they exceed a memory size threshold

0cc4m committed 11 days ago

0a019ed8

vulkan: Support asymmetric FA in coopmat2 path (#21753)

jeffbolznv committed 11 days ago

Verified 05e141a6

ggml-webgpu: add the upscale shader (#22419)

Constannnnnt committed 12 days ago

Verified aab68217

ggml-webgpu: Improve performance of mat-vec and mat-mat for MUL_MAT_ID (#22464)

yomaytk committed 12 days ago

Verified a95a11e5

Update llama-mmap to use ftello/fseeko (#22497)

reeselevine committed 12 days ago

Verified 5cbfb180

common : check for null getpwuid in hf-cache (#22550)

angt committed 12 days ago

Verified beb42fff

vulkan: add get/set tensor 2d functions (#22514)

0cc4m committed 12 days ago

Verified 660b1b4b

spec: fix argument typo (#22552)

barnjamin committed 12 days ago

Verified c20c4451

ci : bump ty to 0.0.33 (#22535)

CISC committed 12 days ago

Verified 6118c043

vendor : update cpp-httplib to 0.43.2 (#22548)

angt committed 12 days ago

Verified 5f0ab726

CUDA: fix tile FA kernel on Pascal (#22541)

JohannesGaessler committed 13 days ago

Verified e82aaf25

scripts : add wc2wt.sh - create worktree from current HEAD (#22513)

ggerganov committed 13 days ago

Verified 27aef3dd

add fast matmul iquants (#22504)

SharmaRithik committed 13 days ago

Verified 45155597

spec : fix draft model checkpoints (#22521)

ggerganov committed 13 days ago

Verified 80afa33a

spec : fix vocab compat checks in spec example (#22426)

petersid2022 committed 13 days ago

Verified b42c7fa5

common : do not pass prompt tokens to reasoning budget sampler (#22488)

aldehir committed 13 days ago

Verified d7759923

hexagon: make vmem and buffer-size configurable (#22487)

max-krasnyansky committed 13 days ago

Verified 41a63be2

Older