DeepSpeed
d6fbd476 - [AutoTP] Preserve universal checkpoint metadata for HuggingFace tp_plan (#8187)

Commit
2 days ago
[AutoTP] Preserve universal checkpoint metadata for HuggingFace tp_plan (#8187) ## Summary Attach AutoTP universal checkpoint metadata before returning from a successfully converted HuggingFace `tp_plan`. Previously, this path replaced modules and returned without preserving the metadata required by `ds_to_universal`, unlike custom-config and heuristic AutoTP paths. ## Testing Add coverage for: - `colwise` - `rowwise` - replicated row-parallel bias - `colwise_gather_output` (Due to #8146 merged) --------- Signed-off-by: iLeGend <824040212@qq.com> Co-authored-by: Ma, Guokai <guokai.ma@gmail.com>
Author
Parents
Loading