sync : ggml #4034
ggerganov
merged
104 commits into
master
from
sync-ggml-26-09-04
metal : null-check buffer alloc to fix OOM crash (llama/25371)
be12d39a
rpc: support apple RDMA as an RPC transport (llama/26421)
e820c280
kleidiai: Rework KleidiAI Build System/Integration (llama/26077)
482956e7
ggml-meta: propagate buffer usage and call init on the new tensors (l…
8df657a2
ggml-metal: add chunked SSD MMA for Mamba-2 prefill optimization (lla…
8c0adb05
cuda: unblock mmq for MoE on sm_60 (llama/26264)
9d8e6b91
rpc : implement event and async backend APIs (llama/18626)
82f5f85e
Implemented vulkan cross_entropy_loss and cross_entropy_loss_back (ll…
0a026972
vulkan: warptiles currently assume warp sizes <= 64, clamp to work ar…
5271734e
hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchro…
3fea10db
metal : fix memory leaks due to missing autoreleasepools (llama/27758)
a5db1d66
Feature: Added LIGHTNING_INDEXER support for Deepseek V4 ops on Vulka…
55ab1e51
opencl: add bin kernels `kernel_gemm_moe_q4_0_q8_1_dp4a_bin`, `kernel…
85299715
hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast …
a0614d9e
ggml-hexagon: add HTP unary ops for ABS and LOG (llama/27786)
b6571e4a
metal : add fa-vec tunings for M4 Pro (llama/27824)
ff38b98e
metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (llama/27863)
530e3f48
sycl: bind the f16 KV cache in place for the oneDNN SDPA path (llama/…
97d0da26
sycl: use TILE for quantized KV decode on BMG (llama/26689)
fa4d244c
OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU,…
7f78e1b4
metal : add fa-vec tunings for M4 (llama/27875)
0a150873
Vulkan: add hoisting support for row IDs and expert count in shaders …
ba99c098
ggml : fix conv_transpose_2d for multiple batches (llama/26132)
caea96f6
vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize…
4c38040f
vulkan: Change mul_mat_id to pad K rather than N (llama/27925)
d501a0a3
metal : add fa-vec tunings for M1 Max (llama/27932)
590fe189
vulkan: combine duplicated fastdiv functions, rename the one optimizi…
e6447520
sycl: make --fit respect --fit-target better (llama/27629)
325c8d16
metal : add remaining fa-vec tunings for M4 Pro (llama/27915)
308fa4f8
metal : assert shared memory padding (llama/27951)
285f1ffd
opencl: use a better matmul path on two Adreno GPU generations (llama…
2a11026c
metal : add fa-vec tunings for M2 (llama/27940)
b33bbc5c
ggml: allow passing alloc dependencies in graph_optimize (llama/27301)
c969c68b
metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (llam…
c68f2055
sycl: split long rows in TOP_K instead of one work-group per row (lla…
3d4e0e98
hexagon: support for device discovery and create sessions on demand (…
3ad8b9b2
rpc : fix pre-rdma macOS versions (llama/27815)
5e494599
metal : Add fa-vec tuning for M3 Pro (llama/27963)
b66593ef
metal: add fa-vec tunings for M3 Ultra (llama/27999)
1e0f3825
rpc: fix apple rdma error spew on teardown (llama/27908)
43acf3d6
ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (llam…
4b2243a6
hip : optimize Q2_0 dot-product path for gfx1201 (llama/26753)
e5c9e3e3
hip: tune rdna 3 mmq config (llama/26284)
35d9e223
CUDA: use the fast mm_ids_helper path for any n_expert_used (llama/27…
e900a732
ggml: add SWIGLU_CLAMP (llama/27930)
e9583f07
ggml : fix ggml_backend_buft_get_alloc_size() guard (llama/28038)
749683d3
rpc: avoid serializing buffers from other servers (llama/26500)
4089fa62
metal : add remaining Q4_1/Q5_0/Q5_1 fa-vec tunings for M2 (llama/28017)
e5c96ca4
hexagon: fix CPY fence bug (llama/28033)
01ebd225
vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next (lla…
db00b019
ggml : add MUL_MAT to the list of ops that may need additional memory…
96dddd87
vulkan: tune mat-vec rows for batched inference on Strix Halo (llama/…
6ce7b899
sycl : Enhance to get the free memory of Intel GPU (llama/27968)
76a51e82
CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-r…
b0f4bc02
metal : add fa-vec tunings for M1 (llama/28078)
7614a4c1
ROCm: add radix TOP_K for long rows (llama/27466)
c1be45b8
webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_…
c648b9a4
opencl: tune the quant paths for Intel Xe-LP GPUs to improve its TG a…
088c603e
metal : add top-k radix implementation (llama/28073)
c6934d0f
AVX2: Speed up large batch size prompt processing of IQ models (llama…
2f608ab4
metal : add concat support for quantized types (llama/28116)
dbc40efc
CUDA: XOR swizzle flash attn K,V smem fp16 tiles (llama/25635)
f22bb2ea
metal : add fa-vec tunings for M1 Ultra (llama/28088)
8e54c659
metal: enable Metal 4.0 tensor API on M5+/A19+ (llama/27461)
5032008b
sycl : support limit max alloc memory within 2GB for host-pinned memo…
4f3a2a4b
metal : add fa-vec tuning for M2 Max (llama/28015)
870db2af
metal : fix more leaks due to missing autoreleasepools (llama/27883)
a245a8f4
metal : add fa-vec tunings for A18 Pro (MacBook Neo) (llama/28152)
8cca1a36
metal : add fa-vec tuning for M2 Pro (llama/28122)
5f07f856
sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 12…
408faaaf
Revert "sycl : add Kronecker product FWHT support for sizes 384, 640,…
f162a194
cuda: fuse MoE weighted expert reduction (llama/25952)
2c486783
metal : add metallib build support for xcframework (llama/28163)
fcc2feee
hexagon: add missing FARF logs for cpy/get_rows/set_rows/gdn ops (lla…
c94921f8
opencl: fix out‐of‐bound reads in the Adreno image kernels (#27632)
35133c94
ggml-cpu : conditionally add SpacemiT IME kernel sources (llama/27961)
d57ae982
vulkan : only request VK_KHR_shader_bfloat16 extension if supported (…
a9e58612
vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec …
1c7d35e1
hexagon: MUL_MAT and MUL_MAT_ID fusion and fixes (llama/28202)
dc70853e
ggml: avoid KleidiAI buffer type init on dispatch (llama/27891)
c2b40075
CUDA + ggml: add sparse-fa for DSV4/GLM (llama/27970)
519df618
ggml-cuda : remove unused vars (llama/28235)
4d343d7c
metal : fix memory query under low-memory conditions (llama/27701)
3a1c7d6b
metal : add fa-vec tunings for M3 (llama/28236)
1bdda1e3
ggml-hexagon: add F16 support for unary ops (llama/28228)
37f0f443
finetune: fix no KV cache (llama/27199)
e5605697
sycl: reduce redundant work in Q4_K multi-column MMVQ (llama/27062)
a704770e
sycl : enhance the api to support peer-to-peer copy (llama/27550)
f24a3860
vulkan: fix FA dequant path engagement (llama/28190)
47d348a2
CUDA: Allow concurrent streams per split for multi-GPU (llama/28198)
25350b57
metal : fix glu dispatch with ne00 = 1 (llama/28306)
d55d345e
metal : add sparse FA (llama/28098)
4dd48dde
tune MMVQ to MMQ crossover for SM87 (llama/28285)
0a4a95c8
opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizatio…
d784add7
SYCL: Refactor GGML_SYCL_ENABLE_MKL_FA to global var (llama/26863)
36f170e5
sycl: fuse rms_norm+mul+add and add+add residual chains (llama/27610)
d1e0e649
ggml-cpu(s390x) : fix q5_1 uninitialized v_acc (llama/28332)
e1bbe405
ggml : remove GGML_CUDA_PEER_MAX_BATCH_SIZE (llama/28177)
f32e6fa0
ggml : don't crash when backend search path can't be read (llama/28271)
1b37bead
ggml : rename and make private ggml_op_alloc_size_may_expand() (ggml/0)
e2389eb9
ggml : remove GGML_CUDA_PEER_MAX_BATCH_SIZE (llama/28177)
f32e6fa0
sync : ggml
52a939a2
danbev
approved these changes
on 2026-09-04
Login to write a write a comment.
Login via GitHub