metal: add a tensor API conv_2d kernel and split long reductions
The conv_2d matmul kernel runs on simdgroup matrices only, and gives
each threadgroup the whole reduction of its output tile. On M5 this
leaves the tensor cores unused, and convolutions with few output
positions and a long reduction (IC * KW * KH in the thousands) run a
handful of threadgroups on an otherwise idle GPU.
Add kernel_conv_2d_mm_tensor for devices with the tensor API: like
kernel_mul_mm, it gathers the patches of 64 output positions into
threadgroup memory as the left operand and reads the kernel rows
from device memory as the right operand. Devices without the tensor
API keep the simdgroup kernel.
Both kernels can now split the reduction until there are enough
threadgroups: each split writes its partial sums to a buffer placed
after the output, as flash attention does, and a small kernel adds
them in a fixed order, so the result stays deterministic. CONV_2D is
marked as an op whose allocation may exceed its output size.
test-backend-ops -o CONV_2D passes, with and without
GGML_METAL_TENSOR_DISABLE.
CONV_2D, F16 kernel, us/run on an M5 Max:
[19,19,256,16] k [4,4,256,4096] 19008 -> 5235
[16,16,128,8] k [3,3,128,512] 333 -> 124
[1536,1536,32,1] k [3,3,32,64] 12086 -> 6512
[33,1,1280,1] k [3,1,1280,1280] s2 659 -> 33
[2054,1,768,1] k [7,1,768,768] 3161 -> 699