pytorch

e154b366 - Standardized clamp kernels to Numpy-like implementation (#43288)

Standardized clamp kernels to Numpy-like implementation (#43288) Summary: **BC-breaking note** For ease of exposition let a_min be the value of the "min" argument to clamp, and a_max be the value of the "max" argument to clamp. This PR changes the behavior of torch.clamp to always compute min(max(a, a_min), a_max). torch.clamp currently computes this in its vectorized CPU specializations: https://github.com/pytorch/pytorch/blob/78b95b6204809822def6dd1b06d03cf002cd30c5/aten/src/ATen/cpu/vec256/vec256_double.h#L304 but in other places it clamps differently: https://github.com/pytorch/pytorch/blob/78b95b6204809822def6dd1b06d03cf002cd30c5/aten/src/ATen/cpu/vec256/vec256_base.h#L624 https://github.com/pytorch/pytorch/blob/78b95b6204809822def6dd1b06d03cf002cd30c5/aten/src/ATen/native/cuda/UnaryOpsKernel.cu#L160 These implementations are the same when a_min < a_max, but divergent when a_min > a_max. This divergence is easily triggered: ``` t = torch.arange(200).to(torch.float) torch.clamp(t, 4, 2)[0] : tensor(2.) torch.clamp(t.cuda(), 4, 2)[0] : tensor(4., device='cuda:0') torch.clamp(torch.tensor(0), 4, 2) : tensor(4) ``` This PR makes the behavior consistent with NumPy's clip. C++'s std::clamp's behavior is undefined when a_min > a_max, but Clang's std::clamp will return 10 in this case (although the program, per the above comment, is in error). Python has no standard clamp implementation. **PR Summary** Fixes discrepancy between AVX, CUDA, and base vector implementation for clamp, such that all implementations are consistent and use min(max_vec, max(min_vec, x) formula, thus making it equivalent to numpy.clip in all implementations. The same fix as in https://github.com/pytorch/pytorch/issues/32587 but isolated to the kernel change only, so that the internal team can benchmark. Pull Request resolved: https://github.com/pytorch/pytorch/pull/43288 Reviewed By: colesbury Differential Revision: D24079453 Pulled By: mruberry fbshipit-source-id: 67f30d2f2c86bbd3e87080b32f00e8fb131a53f7