llama.cpp
5c755617 - metal : single-pass flash attention sparse index compaction

Commit
71 days ago
metal : single-pass flash attention sparse index compaction The idx kernel previously read the mask row twice: once to count the finite entries (for the prefix scan) and again to recover their positions. Since the kernel is memory-bound, this doubled the mask traffic. Keep the finite positions in a per-thread register array during the count pass and write them out directly, avoiding the second mask read. A dense mask with more than NLOCAL finite entries in a slice falls back to re-reading the mask to write the remaining positions. Assisted-by: pi:llama.cpp/DeepSeek-v4-0731
Author
Committer
Parents
Loading