pytorch
9c45f47b - [FSDP] Save `_fsdp_states` on root (#95343)

Commit
1 year ago
[FSDP] Save `_fsdp_states` on root (#95343) This saves an attribute `_fsdp_states: Optional[_FSDPState]`. For root, it is populated with all `_FSDPState`s in the root's tree. For non-root, it is `None`. This is used to avoid doing the tree traversal during `_root_pre_forward()` when `forward_prefetch=True`. Differential Revision: [D43536895](https://our.internmc.facebook.com/intern/diff/D43536895) Pull Request resolved: https://github.com/pytorch/pytorch/pull/95343 Approved by: https://github.com/fegin
Author
Committer
Parents
Loading