DeepSpeed
d4a02b49 - Non-reentrant activation checkpoint CPU offload

Commit
55 days ago
Non-reentrant activation checkpoint CPU offload saved_tensors_hooks context manager that offloads checkpoint hidden-state inputs to a pinned CPU buffer pool on a side stream; works with use_reentrant=False. Adapted from axolotl PR 3776. Signed-off-by: Wing Lian <wing@axolotl.ai>
Author
Committer
Parents
Loading