DeepSpeed
Fix the bias-add and add the layer-norm-eps parameter
#791

Merged

Fix the bias-add and add the layer-norm-eps parameter #791

tjruwase merged 3 commits into master from transformer/fix-bias-add

fix the bias-add precision and indexing and also adding the layer-nor…

6308ff70

RezaYazdaniAminabadi requested a review from

arashashari 4 years ago

RezaYazdaniAminabadi requested a review from

awan-10 4 years ago

RezaYazdaniAminabadi requested a review from

cli99 4 years ago

RezaYazdaniAminabadi requested a review from

conglongli 4 years ago

RezaYazdaniAminabadi requested a review from

eltonzheng 4 years ago

RezaYazdaniAminabadi requested a review from

jeffra 4 years ago

RezaYazdaniAminabadi requested a review from

minjiaz 4 years ago

RezaYazdaniAminabadi requested a review from

niumanar 4 years ago

RezaYazdaniAminabadi requested a review from

samyam 4 years ago

RezaYazdaniAminabadi requested a review from

ShadenSmith 4 years ago

RezaYazdaniAminabadi requested a review from

tjruwase 4 years ago

tjruwase approved these changes on 2021-02-24

add ACC_HALF config

3b8af456

use defined to check if ACC_Half is defined

9c9f80f1

tjruwase merged e2dfcadf into master 4 years ago

mrwyattii deleted the transformer/fix-bias-add branch 2 years ago

Reviewers

tjruwase

arashashari

awan-10

cli99

conglongli

eltonzheng

jeffra

minjiaz

niumanar

samyam

ShadenSmith

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed Fix the bias-add and add the layer-norm-eps parameter #791 Merged

Fix the bias-add and add the layer-norm-eps parameter #791

DeepSpeed
Fix the bias-add and add the layer-norm-eps parameter
#791

Merged