transformers
c220ea9e - Fix load_adapter OOM caused by full-model warmup sizing (#46145)

Commit
54 days ago
Fix load_adapter OOM caused by full-model warmup sizing (#46145) * Fix load_adapter OOM caused by full-model warmup sizing load_adapter passed every named parameter on the model, including the base model, as expected_keys to _load_pretrained_model. Downstream, caching_allocator_warmup summed those into a full base-model byte count and issued a single same-size allocation on top of the already-resident base model, OOMing whenever the base model occupies more than ~half of GPU memory. The file already defined an is_adapter_key helper for identifying parameters belonging to the freshly-injected adapter, but it was declared after the _load_pretrained_model call. Hoist the helper above the call and apply it to expected_keys. Adds a regression test that captures the device map passed to caching_allocator_warmup during load_adapter and asserts it contains only adapter-owned parameter names, not base-model names. * Address review: use unittest.mock.patch and expand test docstring
Author
Parents
Loading