pytorch
a5d47201 - [FSDP] Option to summon on rank 0 only (#73903)

Comment changes are shownComment changes are hidden
Commit
2 years ago
[FSDP] Option to summon on rank 0 only (#73903) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/73903 Add an option to summon full parameter on rank0_only. Note that as documented, when using this flag, writes are only guaranteed to persist for rank 0. ghstack-source-id: 151311233 Test Plan: CI Reviewed By: zhaojuanmao Differential Revision: D34706449 fbshipit-source-id: 034d76cdae6916a225bd6fcee092db48359f8a0b (cherry picked from commit 9cef01d2c5cbeda4ea02d22d7dd6ef17fb4e7250)
Author
Committer
Parents
  • test/distributed/fsdp
    • File
      test_fsdp_summon_full_params.py
  • torch/distributed/fsdp
    • File
      fully_sharded_data_parallel.py