Bump min peft 0.19.1 remove weight conversion duplicate code (#46442)
* [PEFT] Use PEFT weight conversion code, bump min version to 0.19.1
As discussed internally.
Context
When Transformers v5 was released with the new weight conversion
feature, it required extra logic to deal with loading PEFT
adapters. We agreed to move that logic to PEFT, but at the time there
was no PEFT release containing the code, so the logic lived in
Transformers. The code has later been duplicated in PEFT and is
contained in the 0.19 release (with a bugfix in 0.19.1).
Description
Now that PEFT v0.19.1 has been out for a long time, we can safely
remove the duplicated code in Transformers and bump the min required
PEFT version to v0.19.1 (of course, PEFT still is an optional
dependency).
All duplicated code has been removed and docs have been updated where
needed. I have confirmed on my machine that the Transformers PEFT
tests still pass with this change. (There are failing tests relating
to hotswapping, but those are being worked on elsewhere.) On the other
side, PEFT tests for weight conversion also still pass.
Open
Some PEFT tests use patch_moe_parameter_targeting, but they are
correctly guarded against removal of that function. After the next
Transformers release, these tests can be removed in PEFT.
* Fix more instances referring to older PEFT versions