llama.cpp
hexagon: support for multi-device model split (aka row-split)
#28589
Merged
Merged
hexagon: support for multi-device model split (aka row-split) #28589
max-krasnyansky
merged
68 commits into
ggml-org:master
from
qualcomm:hexagon-mdev
hex-row-split: add support for multi-device row spliting
71fcfedc
hex-mdev: add work splitting to fused kernels
0abdcee9
hex-mdev: use mdev_ prefix for all multi-device state
e13454cf
hex-mdev: make device configuration more expressive to support device…
dd241755
hex-mdev: fix mdev session init
f7a6d7f6
hex-mdev: fused nx (2x,3x) matmuls must update row counts for each w/o
89aa7359
hex-mdev: fix MUL_MAT work partitioning bugs introduced by mdev
fbd86417
hex-cont: fix crashes with new tests due to wrong striding
cd3f00ae
hex-mdev: move fences after l2flushes
20c1f986
hex-cont: fix work splitting for mnpu -- align chunks to cachelines
35c9f7b6
hex-mdev: fix CPY tests with multi-dev
9bdfc495
hex-mmid: fix work partitioning with mnpu
60244bb8
hex-mm: fix test failures with mdev
428e33f5
hex-binary: fix work partitioning for mdev
165a86ca
hex-argsort: fix mdev partitioning
5ddb1ad1
hex-mdev: fix work partitioning and general updates for all simple ops
01acec12
hex-fa: fix mdev work splitting issues
1ad87b40
hex-mdev: fixing more failing ops test
d2995e3d
hex-mdev: update the rest of the ops
a48a66f3
hex-mdev: refactor all mdev splitting logic to be contained within if…
165fbc81
hex-mdev: fix macros
26b767a5
hex-mdev: simplify session flush logic
446c7d95
hex-sync: fix recursion in session flush
ca234602
hex-mdev: factor out fence buffer and allocator
b0c236a8
hex-fence: make fence allocation more robust with reserved slots for …
7c742bbb
hex-mdev: keep all mdev state in htp_mdev_group
c5ea84ea
hex-mdev: further cleanup mdev group handling at the host
78cd3617
hex-mdev: update group idx in the opbatch before serializing
e4df2c69
hex-batch: remove separate op_pending and use batch_req/rsp_seq
007b1df5
hex-async: workaround another missing tensor_init in ggml-meta
7c22ca99
hex-fence: cleanup and robustify fences and error handling in multi-d…
207076ec
hex-ar: improve ALLREDUCE error handling
3c10d6fe
hex-async: robust error handling for op_cpy_fence
930917a3
hex-async: use seq0 from allreduce context to allocate fence_seq
2dbb1000
hex-mdev: fix remaining issues with fence and barrier clearing in CPY…
2dea460c
hex-misc: realign macros and fix misplaces trace events
03a5f17d
hex-misc: align macros
bd6cc5c7
hex-mdev: fix unclone buffer re-entrancy
500641d0
hex-glu: fix mdev partitioning logic
b08a823a
hex-mdev: make buffer uncloning/cleanup work with tensor-split scenarios
295f70c1
hex-mdev: tighten up the can_split check in act-ops
132aab63
hex-mdev: factor out common bits of the partitioning logic
647612f5
hex-mm: minor realignment of the macros
94eea9fe
hex-bufs: fix incorrectly placed assert for MAX_BUFS
ca94ef22
hex-pad: tighten up gating checks for PAD
fa1f7dfe
hex-kparams: make sure all kernels properly use kparams->n_threads
0e458476
hex-docs: update user and developer docs with new features and detail…
01ee3911
hex-scripts: update run script to properly parse dev groups
8c983ace
hex-misc: formatting
08054ec2
hex-sess: minor cleanup for session init
e40f24bf
hex-ar: fix vtcm size calc in allreduce kparams
90050320
hex-scripts: fix flake8 warnings
d57cb39a
hex-rope: update ROPE to support mdev work split
d896ff8a
hex-ops: remove redunant checks and minor reformat
97d0ae1d
hex-dev-guide: update dev-guide to avoid redundant null checks
ecc59fc8
hex-async: improve event_wait, event_sync and fence implementations
11638724
hex-async: remove synchronous flush from event_sync
3c3ca80e
hex-async: symplify fence recovery protocol and make sync more robust
eb2023c0
hex-async: futher simplify error recovery for fences
b752c7cc
hex-err: return status instead of just -1
b70b163a
hex-async: print all seq nums in hex
1966ad1c
hex-async: make sure fences flush dirty ranges
a76241b8
hex-async: add dirty ranges merging to reduce fence flushes
d1194be7
hex-async: properly sync before freeing the event
2833f168
hex-async: make sure fence owner session is not overriden
f733de46
hex-async: more fence write order more robust
250e3c96
hex-async: make sure not to fuse ALLREDUCE+ADD if their dsts overlap
2a8cb531
hex-fusion: cleanup redundant checks
9b9bdf1e
lhez
approved these changes
on 2026-09-11
jhen0409
approved these changes
on 2026-09-12
Login to write a write a comment.
Login via GitHub