[BE][DDP] enable rebuilt bucket when find_unused_parameters=True (#73276)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73276
There are two major cases when find_unused_parameters=True:
1. grad ready order does not change over iterations, in this case, enable rebuilt bucket after first iteration can potentially improve performance
2. grad ready order changes over iterations, in this case, use static bucket order or dynamic bucket order in the first iteration does not matter much, as order changes per iteration
ghstack-source-id: 149820812
Test Plan: unit tests
Reviewed By: rohan-varma
Differential Revision: D34410523
fbshipit-source-id: 73284c3629ff2696de76681f070b74ad2bb01f1b
(cherry picked from commit fa3a54bdd659669b776439190039ad889cf3371f)
