accelerate
[FSDP2] Issues in Wrap Policy and Mixed Precision
#3528

Merged

[FSDP2] Issues in Wrap Policy and Mixed Precision #3528

S1ro1 merged 5 commits into huggingface:main from jhliu17:fsdp2-wrap-policy-and-mixed-precision

jhliu17 changed the title ~~FSDP wrap policy and mixed precision~~ [FSDP2] Issues in Wrap Policy and Mixed Precision 1 year ago

S1ro1 commented on 2025-04-23

SunMarc commented on 2025-04-23

SunMarc approved these changes on 2025-04-24

S1ro1 approved these changes on 2025-04-24

fix fsdp2 wrap policy

afb61804

nn.Module doesn't have the dtype attribute

d7667905

Revert "nn.Module doesn't have the dtype attribute"

98148dbe

Fix dtype handling in fsdp2_prepare_model to accommodate nn.Module wi…

1c3e8361

jhliu17 force pushed from 5f632aa2 to 1c3e8361 1 year ago

fix format problem

bed58ded

S1ro1 merged 806ac848 into main 1 year ago

jhliu17 deleted the fsdp2-wrap-policy-and-mixed-precision branch 1 year ago

Reviewers

SunMarc

S1ro1

Assignees

No one assigned

Labels

None yet

Milestone

No milestone