DeepSpeed
066644d7 - fix the sequence-parallelism for the dense models

Commit

2 years ago

fix the sequence-parallelism for the dense models

References

#4530 - Fix the sequence-parallelism for the dense model architecture

Author

Reza Yazdani

Parents

Loading