PyTorch-compatible backward API #7665
tohtana
merged
39 commits into
deepspeedai:master
from
tohtana:tohtana/backward_non_scalar
rename backward prologue method
8a27283f
refactor loss scaling
d85cfd96
refactor backward
bded5c8a
fix for bf16 optimizer
1f413d6e
simplify preprocess/postprocess of backward
cc879771
fix order of backward postprocess
95018a3c
enable non-scalar backward only for ZeROOptimizer
80d0e7de
fix zero+fp16 case
db70476b
add config to enable allow_user_backward
50b29d8a
fix flag for error handling
076b1878
resolve conflict
f6748d12
add test cases
280b1fa2
Merge branch 'master' into tohtana/backward_non_scalar
5d5e64ed
fix format
6ce26f3f
return scaled loss from engine's backward
0c579d58
Merge branch 'master' into tohtana/backward_non_scalar
16150368
remove option to enable user backward
c8758f7b
add hook utility
9962f2c5
fix for z2
a8f15a02
fix scaling
1d0a7215
exclude unused params from counter
39372ac1
set default flag
7eacbc79
handle non-zero optimizer
6cce9371
call epilogue in engine's backward
b72b5a71
prevent hooks from being called from nested backward
1307a870
run post hook fo rz3
98cc8657
Merge branch 'master' into tohtana/backward_non_scalar
adb69907
added comments
9328dfac
remove hard-coded tolerances
01b3251e
add test for multiple engines
78f7ad49
update document
73f7ff1a
sfc-gh-truwase
commented
on 2025-11-17
sfc-gh-truwase
commented
on 2025-11-17
sfc-gh-truwase
commented
on 2025-11-17
sfc-gh-truwase
commented
on 2025-11-17
remove deprecated comment
26308cdb
simplify utility func to count effective grad nodes
9963546f
fix combination with leaf module
b730f464
refactor tests
08b15999
refactor tests
92d30687
sfc-gh-truwase
approved these changes
on 2025-11-18
fix loss scaling
ebac40bc
Merge branch 'master' into tohtana/backward_non_scalar
fcf7c8c0
Merge branch 'master' into tohtana/backward_non_scalar
90e1b7de
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub