llama.cpp
ggml: aarch64: implement mmla kernels for q8_0_q8_0, q4_0_q8_0 and q4_1_q8_1 quantized gemm
#4966

Merged

ggml: aarch64: implement mmla kernels for q8_0_q8_0, q4_0_q8_0 and q4_1_q8_1 quantized gemm #4966

ggerganov merged 5 commits into ggml-org:master from snadampal:smmla_aarch64

snadampal force pushed from d924089b to 2790ddae 1 year ago

snadampal changed the title ~~ggml: aarch64: implement mmla kernel for q8_0_q8_0 quantized gemm~~ ggml: aarch64: implement mmla kernels for q8_0_q8_0 and q4_0_q8_0 quantized gemm 1 year ago

AGSaidi commented on 2024-01-16

snadampal force pushed from 2790ddae to 72cad332 1 year ago

snadampal changed the title ~~ggml: aarch64: implement mmla kernels for q8_0_q8_0 and q4_0_q8_0 quantized gemm~~ ggml: aarch64: implement mmla kernels for q8_0_q8_0, q4_0_q8_0 and q4_1_q8_1 quantized gemm 1 year ago

cebtenzzre commented on 2024-01-16

snadampal force pushed from 72cad332 to 9859c5b2 1 year ago

ggerganov added performance

snadampal force pushed from c5c91408 to f434e522 1 year ago

snadampal force pushed from f434e522 to 99b811df 1 year ago

cebtenzzre commented on 2024-01-23

snadampal force pushed from 99b811df to d228130a 1 year ago

snadampal force pushed from d228130a to 9eaba38c 1 year ago

ggerganov added high priority

ggerganov requested a review from

ggerganov 1 year ago

ggerganov commented on 2024-01-27

snadampal force pushed from 9eaba38c to d0b014f7 1 year ago

ggerganov commented on 2024-02-02

snadampal force pushed from d0b014f7 to ff677758 1 year ago

snadampal force pushed from ff677758 to 4c840fd3 1 year ago

ggerganov added need feedback

ggerganov commented on 2024-02-06

ggml: aarch64: implement smmla kernel for q8_0_q8_0 quantized gemm

52489546

ggml: aarch64: implement smmla kernel for q4_0_q8_0 quantized gemm

ba668572

ggml: aarch64: implement smmla kernel for q4_1_q8_1 quantized gemm

9cd5b8de

ggml: update unit tests for the new vec_dot interface

bca726f0

llama.cpp: add MATMUL_INT8 capability to system_info

d8f132d1

snadampal force pushed from 4c840fd3 to d8f132d1 1 year ago

ggerganov approved these changes on 2024-02-11

ggerganov merged a07d0fee into master 1 year ago

Reviewers

ggerganov

cebtenzzre

wtarreau

AGSaidi

Assignees

No one assigned

Labels

performance high priority need feedback

Milestone

No milestone

llama.cpp ggml: aarch64: implement mmla kernels for q8_0_q8_0, q4_0_q8_0 and q4_1_q8_1 quantized gemm #4966 Merged

ggml: aarch64: implement mmla kernels for q8_0_q8_0, q4_0_q8_0 and q4_1_q8_1 quantized gemm #4966

llama.cpp
ggml: aarch64: implement mmla kernels for q8_0_q8_0, q4_0_q8_0 and q4_1_q8_1 quantized gemm
#4966

Merged