llama.cpp
ggml-cpu: aarm64: q5_K repack gemm and gemv (and generic) implementations (i8mm)
#18860
Merged
Merged
ggml-cpu: aarm64: q5_K repack gemm and gemv (and generic) implementations (i8mm) #18860
ggerganov
merged
14 commits into
ggml-org:master
from
Alcpz:Alcpz/arm_q5_K_repack
Boilerplate for q5_Kx8 REPACK on ARM and fallback
9b2129b0
Implements make_block_q5_Kx8 by extending make_block_q4_Kx8
7d944e99
q5_K repack gemm and gemv generics
5ea06c3a
Gemm and Gemv ARM implementations (i8mm)
f5341c60
Improved qh manipulation looking at non-repack vec_dot implementation
a8e2fdbd
Full unroll
960689d2
Apply Q5_K Gemv vand and vshl optimizations to gemm. Improve comments.
1d8c0bd8
Fix wrong fallback definitions of Q5_K
f9582a66
Fixed comments. Reverted unnecessary formatting
794e9ecd
Fixed typo in generic definitions
d65e2eae
Switching AND + Shift with Shift Insert. Better op interleaving.
a6e22819
Vectorize + unroll the block scales
339734dc
Apply gemm optimizations to gemv
365555de
Improve bias calculation
69b24778
ggerganov
approved these changes
on 2026-01-23
Login to write a write a comment.
Login via GitHub