Conservative-ish persistent RNN heuristics for compute capability 8.0+ (#43165)
Summary:
Based on https://github.com/pytorch/pytorch/pull/43165#issuecomment-697033663 and tests by Vasily Volkov ([persistentRNN-speedup.xlsx](https://github.com/pytorch/pytorch/files/5298001/persistentRNN-speedup.xlsx)). See comments in code.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43165
Reviewed By: zhangguanheng66, mruberry
Differential Revision: D23991756
Pulled By: ngimel
fbshipit-source-id: 4c2c14c9002be2fec76fb21ba55b7dab79497510
