exclude fsdp from delay_optimizer_creation #34140
ArthurZucker
merged
34 commits into
huggingface:main
from
eljandoubi:fix_fsdp_with_fp8_in_trainer
exclude fsdp from delay_optimizer_creation
cd0e8bb8
muellerzr
commented
on 2024-10-14
add test case for trainer: FSDP mode and fp8 as mixed precision
d18e6424
rearrange imports
3344110c
ruff formatted
5055e2a2
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
656d7cc5
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
22cc58dd
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
4827a392
adapt _init_fsdp to fp8
4a84f0f0
use _init_fsdp only when resume_from_checkpoint
2e91c5f4
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
f5a3796a
In case of FDP, self.layer will be CheckpointWrapper which has no len…
af73835e
delete _init_fsdp
a2f30b0c
solve conflict
a838ba55
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
cc5b4c3f
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
d84336fc
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
acffb63c
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
78eed705
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
49882f88
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
9ac46640
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
5acf8e05
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
d7a01949
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
58d18f67
fix conflict
b94376d4
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
b9b9eb4f
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
f4665139
muellerzr
approved these changes
on 2024-10-23
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
2948b297
Merge branch 'main' into fix_fsdp_with_fp8_in_trainer
748270da
make fixup
0ec8e587
Merge branch 'huggingface:main' into fix_fsdp_with_fp8_in_trainer
09df2edc
Merge branch 'main' into fix_fsdp_with_fp8_in_trainer
a3265d9c
Merge branch 'main' into fix_fsdp_with_fp8_in_trainer
33902fdf
Merge branch 'main' into fix_fsdp_with_fp8_in_trainer
cfd81524
Merge branch 'main' into fix_fsdp_with_fp8_in_trainer
571e58fd
Merge branch 'main' into fix_fsdp_with_fp8_in_trainer
02a63c7f
ArthurZucker
approved these changes
on 2024-10-28
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub