transformers
1339a14d - Add safe_globals to resume training on PyTorch 2.6 (#34632)

Commit
1 year ago
Add safe_globals to resume training on PyTorch 2.6 (#34632) Starting from version 2.4 PyTorch introduces a stricter check for the objects which can be loaded with torch.load(). Starting from version 2.6 loading with weights_only=True requires allowlisting of such objects. This commit adds allowlist of some numpy objects used to load model checkpoints. Usage is restricted by context manager. User can still additionally call torch.serialization.add_safe_globals() to add other objects into the safe globals list. Accelerate library also stepped into same problem and addressed it with PR-3036. Fixes: #34631 See: https://github.com/pytorch/pytorch/pull/137602 See: https://pytorch.org/docs/stable/notes/serialization.html#torch.serialization.add_safe_globals See: https://github.com/huggingface/accelerate/pull/3036 Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
Author
Parents
Loading