pytorch

7030f2c6 - Implement tensor.align_to(names), torch.align_tensors(*tensors) (#23804)

Implement tensor.align_to(names), torch.align_tensors(*tensors) (#23804) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/23804 `output = tensor.align_to(names)` returns a view of `tensor` such that `output.names = names`. Dimensions with the same names in `tensor` and `output` have the same sizes; dimensions with new names have size 1. The following must be true for this operation to succeed: 1) tensor.names must be a subsequence (not necessarily contiguous) of `names` 2) Aligning tensor.names to names must not change the absolute position from the right of any unnamed dimension. In practice, these constraints mean that aligning cannot transpose names. Some examples: - Tensor[C].align_to(C) -> Tensor[C] - Tensor[N].align_to([N, C]) -> Tensor[N, C] - Tensor[H, W].align_to([N, H, W, C]) -> Tensor[N, H, W, C] - Tensor[None].align_to([N, None]) -> Tensor[N, None] - Tensor[N].align_to([N, None None]) -> Tensor[N, None, None] Examples of error cases: - Tensor[W, H].align_to([N, H, W, C]) -> Error (not a subsequence) - Tensor[None, H].align_to([None, H, W]) -> Error (would change the absolute position from the right of a None dimension) `torch.align_tensors(*tensors)` aligns the named dimensions of each tensor according to the alignment rules so that they can be used in an operation. More concretely, it aligns each tensor to the longest names among the names of the tensors in `tensors`. This allows users to emulate "broadcasting by names", which is one of the things named tensors tries to enable. Here is an example: ``` imgs: Tensor[N, C, H, W] scale: Tensor[N] // Doesn't work because we do broadcasting by alignment by default imgs * scale // Does work imgs, scale = torch.align_tensors(imgs, scale) imas * scale ``` Future: - Consider allowing broadcasting by names by default. Test Plan: - The diff looks pretty large but more than half of it is testing. - new tests [namedtensor ci] Differential Revision: D16657927 Pulled By: zou3519 fbshipit-source-id: e2f958bf5146c8ee3b694aba57d21b08e928a4e6