DeepSpeed
f6f32023 - stop tracking backward chain of broadcast (z3)

Commit

1 year ago

stop tracking backward chain of broadcast (z3)

References

tohtana/bcast_warning_z3

#5113 - Stop tracking backward chain of broadcast (ZeRO3)

Author

Masahiro Tanaka

Parents

Loading