DeepSpeed
Correctness fix PP+ZeRO for gradient accumulation
#1264
Merged
Overview
Commits
4
Changes
View On GitHub

Correctness fix PP+ZeRO for gradient accumulation #1264

jeffra merged 4 commits into master from jeffra/pp-zero-gas-fix
jeffra
jeffra pass GAS boundary state from PP -> ZeRO
089087bf
jeffra jeffra requested a review from awan-10 awan-10 4 years ago
jeffra jeffra requested a review from cli99 cli99 4 years ago
jeffra jeffra requested a review from conglongli conglongli 4 years ago
jeffra jeffra requested a review from eltonzheng eltonzheng 4 years ago
jeffra jeffra requested a review from minjiaz minjiaz 4 years ago
jeffra jeffra requested a review from niumanar niumanar 4 years ago
jeffra jeffra requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 4 years ago
jeffra jeffra requested a review from samyam samyam 4 years ago
jeffra jeffra requested a review from ShadenSmith ShadenSmith 4 years ago
jeffra jeffra requested a review from tjruwase tjruwase 4 years ago
jeffra formatting
7ae324d3
jeffra jeffra changed the title pass GAS boundary state from PP -> ZeRO Correctness fix PP+ZeRO for gradient accumulation 4 years ago
samyam
samyam approved these changes on 2021-07-29
tjruwase
tjruwase commented on 2021-07-30
tjruwase Merge branch 'master' into jeffra/pp-zero-gas-fix
7d623926
MichaelEk
tjruwase Merge branch 'master' into jeffra/pp-zero-gas-fix
5c4e586d
jeffra jeffra merged b712babe into master 4 years ago
jeffra jeffra deleted the jeffra/pp-zero-gas-fix branch 4 years ago
stas00
jeffra
stas00

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum