llama.cpp
960dffab - hexagon: MUL_MAT and MUL_MAT_ID fusion and fixes (#28202)

Commit
33 days ago
hexagon: MUL_MAT and MUL_MAT_ID fusion and fixes (#28202) * hex-mm: fuse QKV and FFN matmuls that land on HMX * hex-mm: remove hardcoded ne[1] < 32K restriction * hex-get-rows: explicitly reject repacked Q8_0 just in case somebody decided to add an override * hex-mm: correct overhead sizing to make sure we dont exceed vtcm budget for large dims * hex-mm: fuse MUL_MAT_ID into MUL_MAT_ID_NX (2x,3x,...) where possible * hex-fusion: update opbatch and opqueue sizing to acount for new fusion and reduce overhead for trace buffer alloc * hex-bufs: sort buffers while finalizing opbatch, helps avoid va space fragmentation * hex-bufs: add simple va defrag to make sure we dont abort just because the va space is fragmented * hex-mm: replaced more scalar divs with fastdiv and minor cleanup * hex-mm: tighten up supported fusion checks to exactly match supported kernels
Parents
Loading