accelerate
Resolve ZeRO-3 Initialization Failure in Pre-Set Torch Distributed Environments (huggingface/transformers#28803)
#2578

Merged

Resolve ZeRO-3 Initialization Failure in Pre-Set Torch Distributed Environments (huggingface/transformers#28803) #2578

pacman100 merged 4 commits into huggingface:main from sword865:deepspeed

Resolve ZeRO-3 Initialization Failure in Pre-Set Torch Distributed En…

e5742b93

Merge branch 'main' into deepspeed

282e4376

pacman100 commented on 2024-03-22

add unit test for deepspeed zero3 intergation

4fe8021c

sword865 force pushed from 59b16367 to 4fe8021c 2 years ago

muellerzr commented on 2024-03-22

update test case then keep it accelerate spec

e526f8c9

muellerzr approved these changes on 2024-03-26

muellerzr requested a review from

pacman100 2 years ago

pacman100 approved these changes on 2024-04-01

pacman100 merged d9a1b8f9 into main 2 years ago

Reviewers

pacman100

muellerzr

Assignees

No one assigned

Labels

None yet

Milestone

No milestone