Implemented safetensors checkpoints save/load for Trainer (#22498)
* implemented safetensors save/load
* remove duplicated file
* added tests
* more tests
* style fix
* fix tf tests
* change to list comprehension
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* review fixes + safe load for sharded checkpoint
* style fix
* remove rogue import
* remove partial to avoid undefined exception
* use naming alias instead of safetensors.torch
* fix safe sharding in tests
* grammar
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* update docs
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* update docs
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* minor corrections
* style
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>