Add GLU variants (#47)
* feat: add glu variant activations
* fix: rm extraneous parentheses
* feat: rm bias to support jit
* fix: replace negative dim with explicit dim
* fix: use `x.ndim` for generic dim handling
* docs: add note on version for posterity
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* docs: specify jit in `x.ndim` comment
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* test: add simple tests to check activations
* fix: use `torch.testing` for tensor checks
* test: use seed-controlled random batch inputs
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
