llama.cpp
13500544 - glm5-next : split MLA heads in pairs for -sm tensor

Commit
8 days ago
glm5-next : split MLA heads in pairs for -sm tensor The MLA cache has a single KV head, so the heads on a device are its GQA ratio. The HIP tile FA kernel for D=512 needs an even GQA ratio; with 3 devices the 16 heads of the test model were split 5/5/6 and it aborted.
Author
Committer
Parents
Loading