transformers
8631167e - Fix `safe_open` mmap memory exhaustion on Windows by using `pread` backend (#48341)

Commit
74 days ago
Fix `safe_open` mmap memory exhaustion on Windows by using `pread` backend (#48341) * Fix `safe_open` mmap memory exhaustion on Windows by using `pread` backend On Windows, memory-mapping safetensors files reserves copy-on-write commit charge for the entire file. For large multi-shard checkpoints this exhausts virtual memory. Switch to the `pread` backend on `win32` in both `_load_pretrained_model` and `MtpModel` loading paths. * Address review: add MPS backend fix to MtpModel, hoist backend selection before loop, use targeted sys.platform patch in test * Address review: remove tests
Author
Parents
Loading