Reland "AOTAutograd: Go down inference path if no outputs require grad (#111011)" (#111347)
Re-land of https://github.com/pytorch/pytorch/pull/111011.
The original PR ended up having a bad interaction with code that tried to run `torch.compile` under `with torch.inference_mode`, which caused some internal tests to fail.
The issue was that:
(1) AOTInductor invokes the pattern matcher passes in inductor
(2) The pattern matcher registers some code with [training_graph](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/fx_passes/pad_mm.py#L461)
(3) The `training_graph` function expects to be able to set the global autograd state to `requires_grad`, and always get out a join graph (assertion [here](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/pattern_matcher.py#L1196)).
(4) However, when inference_mode is activated, and you try to run AOTAutograd, AOTAutograd will witness that all outputs to the traced function will not require grad, and (now correctly) think that we are tracing an inference graph, which fails the above assert.
After talking to Bin, it sounds like these training-only patterns aren't necessary when we know we are compiling an inference graph (which should always be the case if you're running torch.compile with inference_mode). So I updated the pattern matcher to ignore any pattern matches using `training_graph`, when inference_mode is enabled.
This reverts commit cf6b1cdf6ac74d375b0787bd8f9463cb3a53b0e5.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/111347
Approved by: https://github.com/Chillee
