Derivatives of relu (#63027) (#63089)
Summary:
Optimization of relu and leaky_relu derivatives for reduction of VRAM needed for the backward-passes
Fixes https://github.com/pytorch/pytorch/issues/63027
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63089
Reviewed By: iramazanli
Differential Revision: D30582049
Pulled By: albanD
fbshipit-source-id: a9481fe8c10cbfe2db485e28ce80cabfef501eb8
