accelerate
FSDP2: Activation-checkpoint modules directly matching auto-wrap policy, not those whose parent module matches policy
#3909
Open
Open
FSDP2: Activation-checkpoint modules directly matching auto-wrap policy, not those whose parent module matches policy #3909
excepshenal
wants to merge
3 commits into
huggingface:main
from
excepshenal:dshen-fix-fsdp2-ac-and-auto-wrap
fix: only grab no_split_modules if transformer_auto_wrap_policy
432e7e8e
fix: use torch apply_activation_checkpointing and fsdp_plugin.auto_wr…
626d4d2b
fix: use fsdp2_plugin.auto_wrap_policy in fsdp2_prepare_model and del…
f4188647
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub