onnxruntime
[CUDA] Add TensorRT fused attention fp16 v2 kernels
#12814

Merged

Commits

Add TensorRT fused attention fp16 kernels

tianleiwu committed 4 years ago
fix cpplint warning

tianleiwu committed 4 years ago
dump half tensor

tianleiwu committed 4 years ago
update test thresholds

tianleiwu committed 4 years ago
drop sm 72; fix lint warnings

tianleiwu committed 4 years ago
sm86=>sm80 fall back; remove sm_72

tianleiwu committed 4 years ago
fix TRT kernel input format; exclude sm75_s512_d64

tianleiwu committed 4 years ago
rewrite AddBiasTransposeTrt

tianleiwu committed 4 years ago
remove seq 512 for sm75; and head_size 32 kernels

tianleiwu committed 4 years ago
Add env variable ORT_DISABLE_FUSED_ATTENTION

tianleiwu committed 4 years ago
exclude files in hipify

tianleiwu committed 4 years ago
fix cpplint; AttentionPastState_dynamic threshold

tianleiwu committed 4 years ago
format

tianleiwu committed 4 years ago
fix --use_mask_index in benchmark

tianleiwu committed 4 years ago
Merge branch 'main' into tlwu/trt_fused_attention

tianleiwu committed 4 years ago