PR #2662 Handle MoE models with DeepSpeed

Handle MoE models with DeepSpeed #2662

pacman100 merged 7 commits into huggingface:main from pacman100:smangrul/fix-moe-support

Handle MoE models with DeepSpeed

cb81cdf8

Update launch.py

ed585c0e

Update test_deepspeed.py

9ddfd2b0

Update test_deepspeed.py

55795e78

pacman100 marked this pull request as ready for review 1 year ago

pacman100 requested a review from

muellerzr 1 year ago

pacman100 requested a review from

BenjaminBossan 1 year ago

BenjaminBossan approved these changes on 2024-04-12

muellerzr approved these changes on 2024-04-12

Update src/accelerate/utils/dataclasses.py

826718cc

address comments

6dd56e4c

Update deepspeed.md

be507b3d

pacman100 merged 701e24c5 into main 1 year ago

Reviewers

BenjaminBossan

muellerzr

Assignees

No one assigned

Labels

None yet

Milestone

No milestone