Expose GLU activations as arguments #69
jaketae
merged
15 commits into
main
from
activation-args
feat: expose glu activations as argument
3ba7882d
thomasw21
commented
on 2021-08-18
chore: rename activations -> glu_activations
f004a5aa
refactor: use lookup dict instead of `getattr()`
c064e2e0
thomasw21
commented
on 2021-08-18
refactor: mv lookup dict to `glu_activations.py`
2944fe43
Merge branch 'main' into activation-args
7d04752e
chore: rm unnecessary default arg
3729ebc2
Merge branch 'activation-args' of github.com:bigscience-workshop/Mega…
04e397e1
stas00
requested changes
on 2021-08-22
test: add bf16 test; gelu in `test_training_all()`
b3c6bbe3
stas00
commented
on 2021-08-22
stas00
commented
on 2021-08-22
Update megatron/testing_utils.py
ff248f0a
refactor: use `require_torch_bf16` decorator
0056ab7d
chore: comment out bf16 test
4097af6f
consistent style
dc61f89e
fix look up table
6798e819
stas00
approved these changes
on 2021-08-22
better grouping
ba8f4041
stas00
commented
on 2021-08-22
fix: replace hard coded options with `GLU_ACTIVATIONS`
0f36f705
Login to write a write a comment.
Login via GitHub