DeepSpeed
Fix hook count performance regression from v0.18.5
#7886

Merged

Commits

Add should_refresh_expected_hook_count and harden enter_backward

tohtana committed 7 days ago
Fix ZeRO-2 hook to cache count_used_parameters_in_backward result

tohtana committed 7 days ago
Fix ZeRO-3 hooks to cache count_used_parameters_in_backward result

tohtana committed 7 days ago
Add regression tests for hook count performance fix

tohtana committed 6 days ago
Add comment clarifying refresh-before-reenter ordering

tohtana committed 6 days ago