pytorch
910d9224 - [spmd compile api] use fake tensor for DTensor propagation (#98789)

Commit
1 year ago
[spmd compile api] use fake tensor for DTensor propagation (#98789) Summary: When using real tensors for DTensor propagation, functionalized _fuse_adam causes a memory spike of size(params + optim_state), which causes OOM on memory constrained environments. Test Plan: Tested manually. Differential Revision: D44845043 Pull Request resolved: https://github.com/pytorch/pytorch/pull/98789 Approved by: https://github.com/mrshenli
Author
Committer
Parents
Loading