PR #17437 OPT - Fix Softmax NaN in half precision mode

OPT - Fix Softmax NaN in half precision mode #17437

younesbelkada merged 12 commits into huggingface:main from younesbelkada:opt-fix-softmax

younesbelkada requested a review from

stas00 3 years ago

younesbelkada requested a review from

ArthurZucker 3 years ago

younesbelkada requested a review from

patrickvonplaten 3 years ago

younesbelkada requested a review from

ydshieh 3 years ago

ydshieh approved these changes on 2022-05-27

stephenroller approved these changes on 2022-05-27

patrickvonplaten requested a review from

patil-suraj 3 years ago

patil-suraj approved these changes on 2022-06-15

ArthurZucker approved these changes on 2022-06-15

patrickvonplaten approved these changes on 2022-06-23

stas00 commented on 2022-06-27

younesbelkada force pushed 3 years ago

attempt to fix softmax

b673beaf

fix copied from

7bc7bbef

fix fp16 issue

16cb85a5

forward contrib credits

de4d086c

final fix fp16

78d7a11c

make style

42c156ac

fix nan in softmax fp16

874377c9

small nit

b5c307ba

final fixes

22bcc84b

younesbelkada force pushed to 22bcc84b 3 years ago

make style

c2909c56

small changes

4c1fb44e

make style

8b992a6e

stas00 commented on 2022-06-29

stas00 approved these changes on 2022-06-29

younesbelkada merged d444edb3 into main 3 years ago

patrickvonplaten deleted the opt-fix-softmax branch 3 years ago

Reviewers

ArthurZucker

ydshieh

stas00

patrickvonplaten

patil-suraj

stephenroller

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers OPT - Fix Softmax NaN in half precision mode #17437 Merged

OPT - Fix Softmax NaN in half precision mode #17437

transformers
OPT - Fix Softmax NaN in half precision mode
#17437

Merged