ggml-org/llama.cpp

Pull Requests Commits

ci : run the server-speculative steps with bash

ServeurpersoCom committed 2 days ago

613584a4

docs: record speculative Spark validation

nv-vankit committed 4 days ago

6a68bac8

server: refine speculative test calibration

nv-vankit committed 4 days ago

5dfb96d8

server: use consolidated speculative models

nv-vankit committed 4 days ago

aa90207b

server: address speculative test review

nv-vankit committed 7 days ago

7a3d1fbf

ci: run speculative tests on DGX Spark

nv-vankit committed 7 days ago

52166ab6

ci: avoid GHA cache for speculative models

nv-vankit committed 8 days ago

d7dec8d8

server: use compact models for speculative decoding CI

nv-vankit committed 8 days ago

c0a7cd20

server : functionality test to make sure every time when we add new spec dec method the existing ones are not broken.

nv-vankit committed 14 days ago

00bfee9f

vendor: update BoringSSL to 0.20261020.0 (#30889)

cabelo committed 16 days ago

Verified f8490953

llama : optimize vocabulary loading (#30679)

mykhailen committed 16 days ago

Verified 824eaecf

llama : place d2t by its set_rows use so it can stay on the GPU (#30912)

am17an committed 16 days ago

Verified eb182d85

metal : fix memory leaks and cleanup logic in GPU capture path (#30890)

nikwen committed 16 days ago

Verified ea9e3446

hexagon: neon-vectorized and threaded repack (tiling) (#30901)

max-krasnyansky committed 16 days ago

Verified 0efe8e4c

opencl: add dp4a GEMM variants to vector load activation (#30753)

lhez committed 16 days ago

Verified b531e4c7

sycl : fix fa crash for Query head dimension mismatch (neq0 != HSK / D=2048/4096) (#30286)

arthw committed 16 days ago

Verified 74b115ca

sycl: collapse duplicate need_check launch bodies in mmq (#30739)

Titaniumtown committed 16 days ago

Verified 96caa800

ggml-cpu : compute f32 gelu_quick and geglu_quick without the fp16 table (#30737)

boshjerns committed 16 days ago

Verified 479a5277

llama : scale moe down projection per expert token (#30861)

ggerganov committed 16 days ago

Verified 8baabd53

Sycl iq3 crash fix (#30604)

anantshri committed 16 days ago

Verified 493253fe

opencl: fix rope program build error with newer Clang (#30893)

bashbaug committed 16 days ago

Verified 339f3242

server : fix usage.completion_tokens undercount with n > 1 (#29486)

unbug committed 16 days ago

Verified 9a3b91c6

server : add POST /tts endpoint for TTS models (#26603)

ngxson committed 16 days ago

Verified 5c0b49a2

server: add realtime transcription API (#30636)

ngxson committed 16 days ago

Verified 1242be66

refactor : move LFM2 shortconv and FFN blocks into a shared LFM base (#30874)

tdakhran committed 16 days ago

Verified 84e66d03

model : support Nemotron-3(.5)-Embed (#30877)

tokinasin committed 16 days ago

Verified 52ffa027

qwen: reshape output to 2d instead of 3d (#30823)

am17an committed 16 days ago

Verified 839a572a

server: Support logprobs with speculative decoding (#27196)

pnb committed 16 days ago

Verified ead9f320

hexagon: fix VTCM size query on images that need layout out-params (#30655)

tdakhran committed 16 days ago

Verified b684b498

mtmd : lfm2 tiles and layout match transformers (#30261)

ykhrustalev committed 16 days ago

Verified 4a45de4f

Older