llama.cpp
opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations (speculative decoding/MTP)
#26477
Merged

opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations (speculative decoding/MTP) #26477

wanghqc
github-actions github-actions added ggml
github-actions github-actions added OpenCL
wanghqc wanghqc force pushed from fa2eacc9 to eb0607ee 107 days ago
wanghqc wanghqc force pushed from eb0607ee to 127316f3 102 days ago
wanghqc wanghqc force pushed from 127316f3 96 days ago
wanghqc wanghqc force pushed to 8b0fb4b0 88 days ago
lhez lhez force pushed from 8b0fb4b0 77 days ago
wanghqc opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations
af7dffa8
wanghqc opencl: guard q4_K/q6_K tiled_ns convert-kernel registration for non-…
b517cd33
wanghqc opencl: gate q4_K MUL_MAT+GLU fusion dispatch to Adreno
aaf2e90e
wanghqc opencl: require the noshuffle weight layout in the q4_K GLU fusion gate
11384baf
wanghqc opencl: do not take the vectorized f16 mrow GEMV path on an unaligned…
c6d0997b
wanghqc opencl: pass the new get_scale_min_k4 stride argument at the row-majo…
f6dcd645
wanghqc opencl: enable the q4_K split-K decode GEMV only where it is measured…
c7ca886d
wanghqc opencl: record the X1-85 split-K datapoint (neutral, exclusion confir…
d8d712b3
wanghqc opencl: restrict the tiled lm_head/embed GEMV default to X2E/A8X
3e83f77b
wanghqc opencl: fix q4_K variant kernels to read the transposed scales layout
b7386ebf
wanghqc opencl: keep the flat-GEMV large-m escape opt-in
6dbdc9b7
wanghqc opencl: guard the o4 GEMV store against the rounded-up dispatch tail
10289e3b
wanghqc opencl: restore the tiled q4_K/q6_K layout on tensor read-back
ad3cef32
wanghqc opencl: split-K for the q8_0 decode GEMV at small M
9ceaa97a
wanghqc opencl: keep the q6_K noshuffle correctness escape ahead of the opt-i…
7833efff
lhez lhez force pushed to 7833efff 77 days ago
lhez lhez marked this pull request as ready for review 77 days ago
lhez lhez requested a review 77 days ago
lhez
lhez approved these changes on 2026-09-03
lhez lhez requested a review from max-krasnyansky max-krasnyansky 77 days ago
max-krasnyansky
max-krasnyansky approved these changes on 2026-09-03
lhez lhez merged 95ef7fc1 into master 76 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone