pytorch

078dcc4e - [wip] Move smallest bucket to end after rebuild buckets (#62279)

[wip] Move smallest bucket to end after rebuild buckets (#62279) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/62279 Before rebuild buckets, `kDefaultFirstBucketBytes` is actually misleading because we reverse the parameter indices when initialize reducer so it is actually the size of the last bucket. Currently rebuild buckets sets this to be the first bucket size, but seeing if keeping it as last can help perf. This is currently experimental only and don't plan to land it unless experiments show a clear win. ghstack-source-id: 135966897 Test Plan: CI Reviewed By: SciPioneer Differential Revision: D29927931 fbshipit-source-id: 55b949986fa2c3bade6fcb4bf5b513461bf0f490