DeepSpeed
PyTorch-compatible backward API
#7665
Merged

PyTorch-compatible backward API #7665

tohtana
rename backward prologue method
8a27283f
refactor loss scaling
d85cfd96
refactor backward
bded5c8a
fix for bf16 optimizer
1f413d6e
simplify preprocess/postprocess of backward
cc879771
fix order of backward postprocess
95018a3c
enable non-scalar backward only for ZeROOptimizer
80d0e7de
fix zero+fp16 case
db70476b
add config to enable allow_user_backward
50b29d8a
fix flag for error handling
076b1878
resolve conflict
f6748d12
sfc-gh-truwase
add test cases
280b1fa2
tohtana Merge branch 'master' into tohtana/backward_non_scalar
5d5e64ed
fix format
6ce26f3f
return scaled loss from engine's backward
0c579d58
tohtana Merge branch 'master' into tohtana/backward_non_scalar
16150368
remove option to enable user backward
c8758f7b
add hook utility
9962f2c5
fix for z2
a8f15a02
fix scaling
1d0a7215
exclude unused params from counter
39372ac1
set default flag
7eacbc79
handle non-zero optimizer
6cce9371
call epilogue in engine's backward
b72b5a71
prevent hooks from being called from nested backward
1307a870
run post hook fo rz3
98cc8657
tohtana Merge branch 'master' into tohtana/backward_non_scalar
adb69907
added comments
9328dfac
remove hard-coded tolerances
01b3251e
add test for multiple engines
78f7ad49
update document
73f7ff1a
tohtana tohtana marked this pull request as ready for review 1 year ago
tohtana tohtana requested a review from tjruwase tjruwase 1 year ago
tohtana tohtana requested a review from loadams loadams 1 year ago
tohtana
sfc-gh-truwase
sfc-gh-truwase commented on 2025-11-17
sfc-gh-truwase
sfc-gh-truwase commented on 2025-11-17
sfc-gh-truwase
sfc-gh-truwase commented on 2025-11-17
sfc-gh-truwase
sfc-gh-truwase commented on 2025-11-17
remove deprecated comment
26308cdb
simplify utility func to count effective grad nodes
9963546f
fix combination with leaf module
b730f464
refactor tests
08b15999
refactor tests
92d30687
sfc-gh-truwase
sfc-gh-truwase approved these changes on 2025-11-18
fix loss scaling
ebac40bc
tohtana Merge branch 'master' into tohtana/backward_non_scalar
fcf7c8c0
tohtana Merge branch 'master' into tohtana/backward_non_scalar
90e1b7de
tohtana tohtana enabled auto-merge (squash) 1 year ago
tohtana tohtana merged 53e91a09 into master 1 year ago

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone