llama.cpp
797de63b - hexagon: add support for Q1_0 and Q2_0 (#30122)

Commit
6 days ago
hexagon: add support for Q1_0 and Q2_0 (#30122) * hexagon: add support for q1_0 quant * hexagon: enable Q1_0 weights in the 2D HMX MUL_MAT path * hex-q1: cleanup gates, generalize tiling and remove in-loop divs * hex-tiling: document tiled formats in developer.md and update htp-tensor.h * hex-q2: add support for Q2_0 * hex-q2: remove stale asserts and speacial cases * hex-mm: fix MUL_MAT_NX assumption about square tiling, Q1/Q2 have non-square tiles * hex-q1/2: minor optimizations (unrolls/un-unrolls/...) * hex-q1: optimize Q1_0 layout to reduce unpack/vrmpy overhead * hex-q1/q2_0: further optimize vec_dots --------- Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
Author
Parents
Loading