onnxruntime
95c4fc68 - [CUDA] Add TensorRT fused attention fp16 v2 kernels (#12814)

Commit
3 years ago
[CUDA] Add TensorRT fused attention fp16 v2 kernels (#12814) * Add TensorRT fused attention fp16 kernels * drop sm 72; seq 512 for sm75; and head_size 32 kernels * Add env variable ORT_DISABLE_FUSED_ATTENTION * exclude files in hipify * update AttentionPastState_dynamic test threshold * fix --use_mask_index in benchmark
Author
Parents
Loading