pytorch

a90389f2 - Port cuda sigmoid to Aten(CUDA) (#26643)

Port cuda sigmoid to Aten(CUDA) (#26643) Summary: VitalyFedyunin, this PR port cuda sigmoid to Aten: https://github.com/pytorch/pytorch/issues/24624; TH/THC sigmoid code can 't be removed because the sigmoid_backward in THNN/THCUNN rely on it. I will port sigmoid_backward to Aten next step, incuding CPU and CUDA, which will remove the sigmoid code in TH/THC . Test script: ``` import timeit device = "cuda" for n, t in [(10, 100000),(1000, 10000)]: print('a.sigmoid() (a.numel() == {}) for {} times'.format(n, t)) for dtype in ('torch.float', 'torch.double', 'torch.half'): print(f'device: {device}, dtype: {dtype}, {t} times', end='\t\t') print(timeit.timeit(f'a.sigmoid()\nif "{device}" == "cuda": torch.cuda.synchronize()', setup=f'import torch; a = torch.ones({n}, device="{device}", dtype={dtype})', number=t)) ``` Device: **Tesla P40** Before: ``` a.sigmoid() (a.numel() == 10) for 100000 times device: cuda, dtype: torch.float, 100000 times 1.2853778750286438 device: cuda, dtype: torch.double, 100000 times 1.2787265420192853 device: cuda, dtype: torch.half, 100000 times 1.2610833930084482 a.sigmoid() (a.numel() == 1000) for 10000 times device: cuda, dtype: torch.float, 10000 times 0.1274153349804692 device: cuda, dtype: torch.double, 10000 times 0.13953313598176464 device: cuda, dtype: torch.half, 10000 times 0.1265286349807866 ``` After: ``` a.sigmoid() (a.numel() == 10) for 100000 times device: cuda, dtype: torch.float, 100000 times 1.275270765996538 device: cuda, dtype: torch.double, 100000 times 1.285128042974975 device: cuda, dtype: torch.half, 100000 times 1.2761492819990963 a.sigmoid() (a.numel() == 1000) for 10000 times device: cuda, dtype: torch.float, 10000 times 0.12851508799940348 device: cuda, dtype: torch.double, 10000 times 0.13738596899202093 device: cuda, dtype: torch.half, 10000 times 0.12715664599090815 ``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/26643 Differential Revision: D17666550 Pulled By: VitalyFedyunin fbshipit-source-id: 376479d94d0649c171fd0b2557699bbdd050fec3