ggml-metal: FA tensor path: gate to the (256, 256) shape
the tensor kernel is only faster than the vec kernel for head size
256/256 (measured +3%..+13% across GQA ratios 8:8 .. 64:8 on M5 Max,
nq = kv = 4096). all the other (dk, dv) pairs lose and now fall back
to the vec kernel:
64/64: -18%
128/128: -12% MHA, -64% GQA 32:8
192/128: -3%
192/192: -53%
320/256: -20% GQA
512/512: -35%..-65%
576/512: -60%..-70%
the large-dk losses are register spills (the f32 Q tile (dk, 8) costs
dk/4 registers per thread). the f16 Q attempt to fix them is blocked
by an MPP driver/compiler bug: with f16 QK operands the coop
destination (P) tile element map changes from (2q x 8kv) to (4q x 4kv)
per thread and the hardware P -> f16 right input conversion of
get_right_input_cooperative_tensor does not match the new layout
(reproduced standalone: the PV output contains the correct values at
the wrong positions; the same class of bug breaks the coop element
map when the K operand is strided). the smem P workaround (write P
to threadgroup memory, PV from a tensor_inline right input, transR =
true with the (k, n) k-contiguous tile) is correct (4809/4809
test-backend-ops) but 2-3x slower than the coop register path.
correctness: 4809/4809 test-backend-ops FLASH_ATTN_EXT.
