DeepSpeed
Fixing gelu_checkpointing memory issue
#812
Merged
Overview
Commits
3
Changes
View On GitHub

Fixing gelu_checkpointing memory issue #812

RezaYazdaniAminabadi merged 3 commits into master from transformer/fix-gelu-checkpoint
RezaYazdaniAminabadi
fixing buffers in transformer kernel when gelu-checkpoint is enabled
455ec722
fixing the test issue for other memory optimization flags
9e5ca614
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from arashashari arashashari 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from awan-10 awan-10 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from cli99 cli99 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from conglongli conglongli 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from eltonzheng eltonzheng 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from jeffra jeffra 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from minjiaz minjiaz 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from niumanar niumanar 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from samyam samyam 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from ShadenSmith ShadenSmith 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from tjruwase tjruwase 4 years ago
fixing a bug for when attn_dropout_checkpoint is enabled
83acfad8
owmohamm
eltonzheng
eltonzheng approved these changes on 2021-03-03
RezaYazdaniAminabadi RezaYazdaniAminabadi merged 8295d7a8 into master 4 years ago
mrwyattii mrwyattii deleted the transformer/fix-gelu-checkpoint branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum