onnxruntime
747b4b02 - Add CUDA implementation of GatherBlockQuantized operator (#25575)

Commit
1 year ago
Add CUDA implementation of GatherBlockQuantized operator (#25575) ### Description This PR implements GatherBlockQuantized operator for CUDA EP with 4 bit and 8 bit data support. ### Motivation and Context GatherBlockQuantified operator is essential for MOE model's expert selection, especially when the model has been statically quantized. --------- Co-authored-by: Xiaoyan Hu <xiaoh@microsoft.com>
Author
Parents
Loading