Change reorder_dimensions behavior to favor output writting sequence (#28615)
Summary:
reorder_dimensions() currently iterate all the operands when determining the dimension order in the TensorIterator. It tries to move a dimension to front if any operand has a dimension whose stride is bigger than this dimension.
reorder_dimensions() do respect the case that stride has zero value. I did not see a reason why reorder_dimensions() need to keep probing each operand under regular cases.
Changed behavior a little bit.
Since operands is ordered by outputs tensor first followed by input tensor. I would favor the writing of outputs is as sequential as possible. This could make the copy between tensors with different memory format faster.
Pls correct me if this change is wrong, thanks.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/28615
Reviewed By: VitalyFedyunin
Differential Revision: D18122474
Pulled By: glaringlee
fbshipit-source-id: f36467489fe6c6514b14ce9dcc439628d5d5ad0e
