PR #139 adding scalenorm, attention_init_method and relu^2

adding scalenorm, attention_init_method and relu^2 #139

huu4ontocord wants to merge 1 commit into bigscience-workshop:main from huu4ontocord:scalenorm_relu2

adding scalenorm, attention_init_method which uses the normal init wi…

ee876652

huu4ontocord requested a review from

stas00 4 years ago

huu4ontocord requested a review from

thomasw21 4 years ago

huu4ontocord requested a review from

jaketae 4 years ago

thomasw21 commented on 2021-10-18

jaketae commented on 2021-10-19

Reviewers

jaketae

thomasw21

stas00

Assignees

No one assigned

Labels

None yet

Milestone

No milestone