peft
45373179 - Extend from_pretrained to models with disk-offloaded modules (#1431)

Commit
1 year ago
Extend from_pretrained to models with disk-offloaded modules (#1431) This PR extends PeftModel.from_pretrained() to be compatible with disk offloading. The approach is to re-make the offload_index map to the original (offloaded) model's safetensors files and then save the LoRA parameters with renamed model module parameters to new safetensors folders, updating the offload_index accordingly. This is a complement to PRs #1190 and transformers #27412, and is designed to allow for the loading of PeftModels using only the memory equivalent of one safetensors file.
Author
Parents
Loading