llama.cpp
Add Hexagon Matrix Extensions (HMX) for Hexagon NPU backend
#20693
Merged
Merged
Add Hexagon Matrix Extensions (HMX) for Hexagon NPU backend #20693
max-krasnyansky
merged
55 commits into
ggml-org:master
from
aizip:feat/hexagon-hmx
chraac
commented
on 2026-03-18
chraac
commented
on 2026-03-18
migrate(vtcm): unify VTCM management for HMX merge
cca1cb87
migrate(repack): replace x4x2 with HMX tile-permuted super-block format
7e641b5e
migrate(dma): add dma_queue_push_1d() convenience wrapper for HMX ops
5863fcf5
migrate(hmx): reorganize HMX files into htp/hmx/ and simplify HMX loc…
2f628d1c
migrate(hmx-infra): consolidate HMX infrastructure into htp_context
5426a6e0
migrate(flash-attn): remove HTP_EXP2_TABLE_COPIES, use single exp2 table
56bbeda2
migrate(dsp-main): add HMX priority dispatch in packet_callback
1fa952b0
migrate(cmake-dsp): add HMX source files and -mhmx for v73+ skels
6e53fb50
migrate(hmx-ops): fix compile errors in HMX ops for ggml struct compa…
ae98cd24
hmx: set Q/O element type to fp16 for flash attention
334bf1d8
hexagon: unify HMX weight format to x4x2, add IQ4_NL and DSP-side fal…
e3a11942
Enhance HMX debugging capabilities with new tile dumping functions
a476c368
OK for small mat mul
f21a276e
hexagon: fix UDMA roiwidth 16-bit overflow in HMX matmul DMA transfers
d1b5384a
hexagon: remove HMX RMS norm fallback and re-enable matmul pipeline
f7b4c499
hexagon: guard all HMX matmul DMA transfers against UDMA 16-bit field…
ee5d6c3c
hexagon: multithread activation/output transfer and add HMX matmul fa…
4106932c
[todo]: dynamic alloc vtcm, cause prefill regression.
28df959f
hexagon: constrain HMX mxmem tile load region to avoid VTCM bank boun…
76c53abd
hexagon: split unaligned-M HMX matmul into HMX+HVX phases
56c4d3e7
hexagon: batch-4 Q4_0 dequantize fast path and remove debug traces
73efe1b5
hexagon: abort on DSP error and fix HMX-to-HVX fallback quantize flag
e23b3edb
hexagon: support batch matmul. This fix perplexity issue
c65f912d
hexagon: reuse weights in fp16 batch matmul
5313184b
hexagon: remove unused HMX flash attention operations and precomputat…
feb4a3fb
hexagon: remove unused HVX math helpers, debug infrastructure, and st…
06ab76cb
hexagon: fix HMX not enabled due to missing force_hvx parameter in IDL
689f9804
hexagon: remove the unnecessary changes not related to HMX
77a166d2
hexagon: bypass HMX by default
bb12f235
hexagon: add upstream repo link to htp-ops-lib ported file headers
90e4573c
hexagon: restore host buffer support
9b633954
hexagon: add HMX=1 option for the adb scripts
485921f2
hex-hmx: improve DMA pipelining
8a6534b9
hex-hmx: further improvements to dma pipelining
b4a726cf
hex-hmx: minor cleanup
aae158c9
hex-hmx: move hmx lock out of inner loops/calls
1c961a7f
hex-hmx: remove unnecessary state and wrappers
7c3e785c
hex-hmx: remove hmx dir and unify f32 to f16 conversions
6ecb04c5
hex-hmx: further unify hvx conversions
3478b673
hex-hmx: revert f16 converter to the original for now
95ca33f4
hex-hmx: minor cleanup for f16 to f32 converter
8611ead7
hex-mm: replace incorrect fp16-to-fp32 hmx converter and reformated r…
f3a25148
hex-dma: move chanied dma push into hex-dma.h header and update hmx-mm
19b68ab5
hex-mm: use hex_is_aligned instead of a duplicated hmx_is_aligned
3645fa3f
hex-mm: use hvx_vec_splat_f16 in the hmx code
03dfccf7
hex-mm: use VLEN and HTP types in hmx-code
fb937da8
hex-mm: remove duplicate QK and defs
3a1dc30d
hexagon: pre-shuffle quants before vlut16
f3741a3c
hexagon: enabel HMX by default
fbeefeab
hex-mm: code indent fixes for hmx-matmul
1f0c4e6f
hexagon: update hex-utils to include align/smin/etc helpers and use t…
07e428bb
hex-mm: more formatting fixes
769d3cb1
hex-mm: minor naming updates in hmx code
cd0bb004
hex-mm: remove leftover from rebase conflict
b120f771
max-krasnyansky
commented
on 2026-03-19
Fix the incorrect indents
2b0619f5
max-krasnyansky
approved these changes
on 2026-03-19
Reviewers
Assignees
No one assigned
Labels
script
ggml
Hexagon
Milestone
No milestone
Login to write a write a comment.
Login via GitHub