onnxruntime
15920d08 - [CUDA] Extend paged XQA decode to head size 256 (#32229)

Commit
83 days ago
[CUDA] Extend paged XQA decode to head size 256 (#32229) Extend the quantized paged-KV XQA decode path to `head_size=256` for FP16 and BF16 activations with INT8 or FP8 KV caches. This enables XQA for models such as Qwen3.8 whose full-attention layers use 24 query heads, 4 KV heads, and head size 256. - Add FP16/BF16 paged XQA instantiations for head size 256 with INT8 and FP8 KV caches. - Add H256 loader declarations, launch dispatch, and dynamic shared-memory queries. - Extend PagedAttention's XQA eligibility check to head size 256. - Add an operator test for the Qwen group-size-6 INT8 geometry that verifies numerical output and confirms `SdpaKernel=XQA` dispatch. The existing runtime shared-memory check remains authoritative. If an H256 kernel does not fit the device's opt-in shared-memory limit, `PagedAttention` uses the existing fallback path.
Author
Parents
Loading