DeepSpeed
Fix the residual add mp scaling for GPTNeoX
#2310
Merged
Overview
Commits
2
Changes
View On GitHub

Fix the residual add mp scaling for GPTNeoX #2310

arashb merged 2 commits into master from arashb/fix-res-add
arashb
arashb arashb requested a review from jeffra jeffra 3 years ago
arashb arashb requested a review from samyam samyam 3 years ago
arashb arashb requested a review from tjruwase tjruwase 3 years ago
arashb arashb requested a review from ShadenSmith ShadenSmith 3 years ago
arashb arashb requested a review from conglongli conglongli 3 years ago
arashb arashb requested a review from awan-10 awan-10 3 years ago
arashb arashb requested a review from cli99 cli99 3 years ago
arashb arashb requested a review from eltonzheng eltonzheng 3 years ago
arashb arashb requested a review from minjiaz minjiaz 3 years ago
arashb arashb requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 3 years ago
arashb arashb requested a review from duli2012 duli2012 3 years ago
arashb arashb requested a review from mrwyattii mrwyattii 3 years ago
arashb arashb requested a review from yaozhewei yaozhewei 3 years ago
arashb arashb requested a review from xiaoxiawu-microsoft xiaoxiawu-microsoft 3 years ago
arashb arashb requested a review from samadejacobs samadejacobs 3 years ago
arashb arashb requested a review from cmikeh2 cmikeh2 3 years ago
arashb arashb marked this pull request as draft 3 years ago
arashb Fix the residual add mp scaling for GPTJ and GPTNeoX
978e058b
arashb arashb force pushed from 0dbce863 to 978e058b 3 years ago
arashb arashb changed the title Fix the residual add mp scaling for GPTJ and GPTNeoX Fix the residual add mp scaling for GPTNeoX 3 years ago
arashb arashb marked this pull request as ready for review 3 years ago
arashb Merge branch 'master' into arashb/fix-res-add
07a623d0
arashb arashb force pushed from cdd078c8 to 07a623d0 3 years ago
RezaYazdaniAminabadi
RezaYazdaniAminabadi approved these changes on 2022-09-12
awan-10
awan-10 approved these changes on 2022-09-12
arashb arashb merged efa8aded into master 3 years ago
mrwyattii mrwyattii deleted the arashb/fix-res-add branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum