DeepSpeed
8295d7a8
- Fixing gelu_checkpointing memory issue (#812)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
4 years ago
Fixing gelu_checkpointing memory issue (#812) * fixing buffers in transformer kernel when gelu-checkpoint is enabled * fixing the test issue for other memory optimization flags * fixing a bug for when attn_dropout_checkpoint is enabled
References
#812 - Fixing gelu_checkpointing memory issue
Author
RezaYazdaniAminabadi
Parents
937c5cee
Loading