pytorch
c39f1c14 - Allow DTensor to trigger collecives before inplace ops (#97787)

Commit
1 year ago
Allow DTensor to trigger collecives before inplace ops (#97787) Mainly two fixes: 1. `make_fx` seems trace through DeviceMesh operations. This commit removes that from the DTensor expanded graph 2. During DTensor expansion, autograd complains about inplace changes on leaf node. This commit wraps entire DTensor expansion code with `torch.no_grad()` Pull Request resolved: https://github.com/pytorch/pytorch/pull/97787 Approved by: https://github.com/wanchaol
Author
Committer
Parents
Loading