DeepSpeed
Fixing the reshape bug in sequence parallel alltoall, which corrupted all QKV data
#5664
Merged
Overview
Commits
2
Changes
View On GitHub

Fixing the reshape bug in sequence parallel alltoall, which corrupted all QKV data #5664

loadams merged 2 commits into deepspeedai:master from YJHMITWEB:master
YJHMITWEB
fixing sequence parallel alltoall reshape bug
8477221d
YJHMITWEB YJHMITWEB requested a review from mrwyattii mrwyattii 1 year ago
YJHMITWEB YJHMITWEB changed the title Fixing a severe reshape bug in sequence parallel alltoall Fixing the reshape bug in sequence parallel alltoall, which caused the model unable to converge 1 year ago
YJHMITWEB YJHMITWEB changed the title Fixing the reshape bug in sequence parallel alltoall, which caused the model unable to converge Fixing the reshape bug in sequence parallel alltoall, which caused the training unable to converge 1 year ago
YJHMITWEB YJHMITWEB changed the title Fixing the reshape bug in sequence parallel alltoall, which caused the training unable to converge Fixing the reshape bug in sequence parallel alltoall, which corrupted all QKV data 1 year ago
tohtana
tohtana tohtana requested a review from tohtana tohtana 1 year ago
tohtana
tohtana approved these changes on 2024-06-17
loadams Merge branch 'master' into master
c24c0c7d
loadams loadams merged 3bdd187e into master 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum