Extend from_pretrained to models with disk-offloaded modules (#1431)
This PR extends PeftModel.from_pretrained() to be compatible with disk
offloading. The approach is to re-make the offload_index map to the
original (offloaded) model's safetensors files and then save the LoRA
parameters with renamed model module parameters to new safetensors
folders, updating the offload_index accordingly.
This is a complement to PRs #1190 and transformers #27412, and is
designed to allow for the loading of PeftModels using only the memory
equivalent of one safetensors file.