DeepSpeed
6cbf6661 - fix MegatronLayerPolicy to be compatible with the newest ParallelTransformerLayer (#4236)

Commit

2 years ago

fix MegatronLayerPolicy to be compatible with the newest ParallelTransformerLayer (#4236) Co-authored-by: Reza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>

References

#4236 - fix MegatronLayerPolicy to be compatible with the newest ParallelTransformerLayer

Author

Dino Chen

Parents

5dbc5313

DeepSpeed 6cbf6661 - fix MegatronLayerPolicy to be compatible with the newest ParallelTransformerLayer (#4236)

DeepSpeed
6cbf6661 - fix MegatronLayerPolicy to be compatible with the newest ParallelTransformerLayer (#4236)