PR #17420 Add Gated-SiLU to T5

Add Gated-SiLU to T5 #17420

patrickvonplaten merged 12 commits into huggingface:main from DanielHesslow:main

patrickvonplaten commented on 2022-05-25

patrickvonplaten commented on 2022-05-26

patrickvonplaten requested a review from

patil-suraj 3 years ago

patrickvonplaten commented on 2022-05-31

patrickvonplaten requested a review from

sgugger 3 years ago

patrickvonplaten approved these changes on 2022-05-31

sgugger approved these changes on 2022-05-31

Add gated-silu to t5 architecture to support UL2

a5bdfcac

Fix error message

b9340089

formatting

3c213c25

formatting again

e7fc734c

refactor

62143366

fix classnames in _init_weights

b583b23e

remove is_gated

a5df0eea

add test

a2bc0b37

fix test

e032d400

Try without the test?

b01264c8

Add back the test.

710eb1ae

Improve error message.

fe29654d

DanielHesslow force pushed to fe29654d 3 years ago

patrickvonplaten approved these changes on 2022-06-03

patrickvonplaten merged 607acd4f into main 3 years ago

Reviewers

patrickvonplaten

sgugger

patil-suraj

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers Add Gated-SiLU to T5 #17420 Merged

Add Gated-SiLU to T5 #17420

transformers
Add Gated-SiLU to T5
#17420

Merged