DeepSpeed
38561463 - Split DeepCompile ZeRO-3 memory scheduler (#8233)

Commit
35 days ago
Split DeepCompile ZeRO-3 memory scheduler (#8233) Splits the DeepCompile ZeRO-3 scheduling and gather/release lifetime changes out of #8169. The exact candidate passed 10/10 tests on a single node with 2 H100 GPUs. Signed-off-by: Masahiro Tanaka <tanaka.masahiro@gmail.com>
Author
Parents
Loading