transformers

883c731b - 🚨🚨🚨 [Trainer] Default to FSDP2, simplify API around fsdp + fsdp_config (#45640)

🚨🚨🚨 [Trainer] Default to FSDP2, simplify API around fsdp + fsdp_config (#45640) * FSDP: default to v2, simplify API around fsdp + fsdp_config - TrainingArguments.fsdp is now a boolean on-switch. String / list values are still accepted for backward compatibility but are translated into fsdp_config entries (and emit a deprecation warning). - fsdp_config now defaults to FSDP2 (version=2). FSDP1 usage still works but logs a deprecation warning (to be removed in v5.20). - New fsdp_config keys: auto_wrap_policy (default TRANSFORMER_BASED_WRAP), cpu_offload, state_dict_type (default FULL_STATE_DICT so trainer.save_model() produces an HF-compatible checkpoint out of the box). - FSDP1-only handling (forward_prefetch, backward_prefetch, use_orig_params, sync_module_states, and the string form of reshard_after_forward) is now isolated in a single branch for easy removal once v1 support is dropped. Legacy `fsdp` string/list parsing lives in a single _apply_legacy_fsdp_to_config helper for the same reason. - Trainer reads args.fsdp / args.fsdp_config guarded so that running without FSDP (or via `accelerate launch` with no transformers-side config) no longer crashes on None. - Docstring rewritten around fsdp_config; FSDP2-only keys surfaced, FSDP1-only keys (sync_module_states, use_orig_params, limit_all_gathers) dropped from the public docstring. Verified on the repro from PR #42521: TrainingArguments(fsdp=True, fsdp_config={"fsdp_version": 2, "reshard_after_forward": True}) now produces fsdp_version=2 and reshard_after_forward=True. * Apply repo consistency fixes * Cleanup: drop unused bool() casts and stale comment - `args.fsdp` is already a bool after `_process_fsdp_args`, so wrapping it with `bool(...)` in trainer.py was redundant. - Remove the explanatory comment above `state_dict_type` default; the docstring already covers it. * Forward activation_checkpointing / limit_all_gathers via plugin kwargs Drop the post-accelerator `setattr` loop that re-pushed these two keys onto the already-constructed FSDP plugin. They are now forwarded during plugin construction via `fsdp_plugin_args`: - `activation_checkpointing` is shared between FSDP1 and FSDP2. - `limit_all_gathers` is FSDP1-only (obsolete in FSDP2), so it lives in the v1 branch. The `activation_checkpointing` + `gradient_checkpointing` conflict check stays in trainer.py (still needs the post-plugin state to compare). * Apply suggestions from code review Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * fix * Apply repo consistency fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>