pytorch
1a729618 - [FSDP][optim_state_dict] Make the new optimizer allgather fusion work with fine-tuning models (#110540)

Commit
1 year ago
[FSDP][optim_state_dict] Make the new optimizer allgather fusion work with fine-tuning models (#110540) With use_orig_params=True, it is possible that some parameters with the same FlatParameter are in the optimizer while others parameters are frozen. This PR makes the allgather fusion logic support the case. Differential Revision: [D49922028](https://our.internmc.facebook.com/intern/diff/D49922028/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/110540 Approved by: https://github.com/awgu, https://github.com/rohan-varma
Author
Committer
Parents
Loading