transformers
055f2fff - Use 32-row flex attention blocks for head_dim > 256

Commit
28 days ago
Use 32-row flex attention blocks for head_dim > 256 The default flex block sizes fault at head_dim 512 on Blackwell (misaligned address), which breaks attn_implementation="flex_attention" for Gemma 4, and are 1.7x slower than 32-row blocks on Hopper.
Author
Parents
Loading