llama.cpp
713499e6 - webgpu: support per-head flash attention masks

Commit
1 hour ago
webgpu: support per-head flash attention masks The flash attention shaders indexed the mask without its head dimension, so masks with ne[2] > 1 gave wrong results. The mask offset and the vec block states now include the head, broadcast as head % mask->ne[2] like the other backends. The params buffer grows to 256 bytes to fit the two new mask fields. The shared memory estimate dropped kv_shmem when only one of K and V is read directly, while the shaders keep it unless both are, which made the K f16 / V q8_0 cases exceed the workgroup storage limit. The vec kv tile now takes both flags: the memory estimate drops kv_shmem only when both are direct, and the tile still divides the KV padding when either is. get_alloc_size and supports_op use the same rule as the pipelines, and supports_op accounts for the register tile fallback.
Committer
Parents
Loading