Dynamo+LTC: handle inplace ops (#75359)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75359
For some models in torchbench (e.g. pyhpc_isoneutral_mixing), dynamo will generate Fx graphs that has side effects . Those graphs may
- return an empty tuple
- change tensors passed in as forward method arguments in-place
This makes the Dynamo+LTC integration fail since we extract compiled graph based on the lazy tensors returned from the forward method. From an empty tuple, we extract nothing.
To solve this problem, we extract compile graph from `union(argument lazy tensors, returned lazy tensors)` instead. The inplace mutations applied to argument lazy tensors will be captured this way.
Test Plan:
```
pytest test/lazy/test_extract_compiled_graph.py
```
```
LTC_TS_CUDA=1 gpui time python torchbench.py --speedup-ltc -dcuda --nvfuser --randomize-input --only pyhpc_isoneutral_mixing
```
Reviewed By: ZolotukhinM
Differential Revision: D35478799
Pulled By: shunting314
fbshipit-source-id: 8116768fc50fe7630e481e6039319ddf5c6a9416
(cherry picked from commit 2e6531d2c80c35ae99c11d49ca01dcdb7fc032f2)
