DeepSpeed
bfcf5a9d - fuse query, key, and value all-2-all for better SP perforamnce

Commit

2 years ago

fuse query, key, and value all-2-all for better SP perforamnce

References

#4735 - SP Comm-optimization: fuse query, key, and value all-2-all for better SP perforamnce

Author

Reza Yazdani

Parents

Loading