pytorch
f1f13a35 - Fix GELU-related docstring formatting (#102845)

Commit
1 year ago
Fix GELU-related docstring formatting (#102845) The docstring about GELU seems formatted incorrectly. The original docstring about GELU is rendered as below: $$ \text{GELU}(x) = 0.5 * x * (1 + \text{Tanh}(\sqrt(2 / \pi) * (x + 0.044715 * x^3))) $$ where the square root of which part is confusing. I double-checked the formula, which should be: $$ \text{GELU}(x) = 0.5 * x * (1 + \text{Tanh}(\sqrt{2 / \pi} * (x + 0.044715 * x^3))) $$ where round brackets in resource code should be brace brackets. > _formula in [original paper](https://arxiv.org/abs/1606.08415)_ > ![Snipaste_2023-06-03_00-43-49](https://github.com/pytorch/pytorch/assets/39690782/22511c4e-2f20-4a16-9bda-4c182a360160) Pull Request resolved: https://github.com/pytorch/pytorch/pull/102845 Approved by: https://github.com/mikaylagawarecki
Committer
Parents
Loading