Add constant folding to ONNX graph during export (Resubmission) (#18698)
Summary:
Rewritten version of https://github.com/pytorch/pytorch/pull/17771 using graph C++ APIs.
This PR adds the ability to do constant folding on ONNX graphs during PT->ONNX export. This is done mainly to optimize the graph and make it leaner. The two attached snapshots show a multiple-node LSTM model before and after constant folding.
A couple of notes:
1. Constant folding is by default turned off for now. The goal is to turn it on by default once we have validated it through all the tests.
2. Support for folding in nested blocks is not in place, but will be added in the future, if needed.
**Original Model:**
**Constant-folded model:**
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18698
Differential Revision: D14889768
Pulled By: houseroad
fbshipit-source-id: b6616b1011de9668f7c4317c880cb8ad4c7b631a
