transformers
Fix initialization for missing parameters in `from_pretrained` under ZeRO-3
#28245

Merged

Fix initialization for missing parameters in `from_pretrained` under ZeRO-3 #28245

amyeroberts merged 6 commits into huggingface:main from XuehaiPan:fix-zero-3-init-missing

Fix initialization for missing parameters in `from_pretrained` under …

ceda28b3

pacman100 commented on 2023-12-27

pacman100 requested a review from

amyeroberts 2 years ago

pacman100 requested a review from

ArthurZucker 2 years ago

Test initialization for missing parameters under ZeRO-3

33e67e06

XuehaiPan force pushed to 33e67e06 2 years ago

XuehaiPan requested a review from

pacman100 2 years ago

Add more tests

10ae1845

ArthurZucker commented on 2024-01-03

Only enable deepspeed context for per-module level parameters

855db0fd

Enable deepspeed context only once

a781cb10

XuehaiPan requested a review from

ArthurZucker 2 years ago

pacman100 approved these changes on 2024-01-08

ArthurZucker approved these changes on 2024-01-09

amyeroberts approved these changes on 2024-01-09

Move class definition inside test case body

5170a013

XuehaiPan force pushed to 5170a013 2 years ago

amyeroberts merged 976189a6 into main 2 years ago

XuehaiPan deleted the fix-zero-3-init-missing branch 2 years ago

Reviewers

ArthurZucker

amyeroberts

pacman100

Assignees

No one assigned

Labels

None yet

Milestone

No milestone