[pytorch] use cublas lt interface for bias fusion (#71200)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/71200
To quantify how much cublas lt interface can help param bench (https://github.com/facebookresearch/param/) linear perf
On V100 GPU
for b in 512 1024; do for i in {1..5}; param_bench/train/compute/pt/pytorch_linear.py --device gpu --dtype=float16 --hidden-size 1024 --batch-size ${b}; done; done
Before this commit
batch size 512: median 21.4 TF/s (20.7, 20.6, 21.8, 21.6, 21.4)
batch size 1024: median 40.1 TF/s (39.4, 39.3, 40.2, 40.4, 40.1)
After this commit
batch size 512: median 23.5 TF/s (23.2, 23.5, 23.8, 23.9, 23.6 ) 9.8% speedup
batch size 1024: median 41.6 TF/s (42.7, 41.6, 40.4, 41.3, 41.9 ) 3.7% speedup
Reviewed By: jasonjk-park, ngimel
Differential Revision: D32053748
fbshipit-source-id: accf787c8727a2f8fb16fae92de461367ac10442
(cherry picked from commit 254532ac451859982da07648431ccbea12e21397)
