llama.cpp
b54752df - ggml-cpu: add x86 VNNI Q2_0 dot product (#26348)

Commit
20 hours ago
ggml-cpu: add x86 VNNI Q2_0 dot product (#26348) * ggml-cpu: use VNNI for Q2_0 dot products Keep the upstream AVX2 unpacking, accumulation, and fallback. Use VNNI for the integer dot product on supported x86 CPUs. Assisted-by: Codex * ggml-cpu: use VBMI for Q2_0 unpacking Use multishift to unpack Q2_0 weights on CPUs with AVX-512 VBMI and VL. Retain the existing shift-and-mask unpacking on other CPUs. Assisted-by: Codex * Update ggml/src/ggml-cpu/arch/x86/quants.c Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Author
Parents
Loading