PR #29879 Make clearer about zero_init requirements

Make clearer about zero_init requirements #29879

muellerzr merged 8 commits into main from muellerzr-deepspeed-description

Docstring to note about zero init

b47f8f10

muellerzr requested a review from

pacman100 2 years ago

muellerzr requested a review from

asartran 2 years ago

Check for accelerate

02348331

muellerzr changed the title ~~Docstring to note about zero init~~ Make clearer about zero_init requirements and add an early check 2 years ago

Change conditional return

125cd16d

muellerzr commented on 2024-03-26

Tweak

eb122c7d

asartran commented on 2024-03-28

Add new accelerate-specific zero3 check

e455bc61

muellerzr requested a review from

asartran 2 years ago

asartran commented on 2024-03-28

Fix import

93ff7e70

muellerzr requested a review from

asartran 2 years ago

pacman100 commented on 2024-04-01

Revert to RTFM

5c2370b4

muellerzr requested a review from

pacman100 2 years ago

asartran approved these changes on 2024-04-03

Update src/transformers/modeling_utils.py

1740e293

muellerzr changed the title ~~Make clearer about zero_init requirements and add an early check~~ Make clearer about zero_init requirements 2 years ago

muellerzr merged 863e2562 into main 2 years ago

muellerzr deleted the muellerzr-deepspeed-description branch 2 years ago

Reviewers

asartran

pacman100

abhishekkrthakur

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers Make clearer about zero_init requirements #29879 Merged

Make clearer about zero_init requirements #29879

transformers
Make clearer about zero_init requirements
#29879

Merged