pytorch
7f50ff16 - [FSDP] Test `use_orig_params=True`, `no_sync()`, mixed precision (#91193)

Commit
2 years ago
[FSDP] Test `use_orig_params=True`, `no_sync()`, mixed precision (#91193) This makes some minor fixes to ensure that `use_orig_params=True`, `no_sync()`, and mixed precision work together for `FULL_SHARD`, `SHARD_GRAD_OP`, and `NO_SHARD`. The added unit test only checks that dtypes are correct since for FP16, it is hard to test for numeric parity against a baseline. Pull Request resolved: https://github.com/pytorch/pytorch/pull/91193 Approved by: https://github.com/zhaojuanmao
Author
Andrew Gu
Committer
Parents
Loading