DeepSpeed
zero3: defer param release during retain_graph backward #7352
#8045
Open
Overview
Commits
21
Changes
View On GitHub

zero3: defer param release during retain_graph backward #7352 #8045

nathon-lee wants to merge 21 commits into deepspeedai:master from nathon-lee:test/zero-multi-loss-separate-backward-7352
nathon-lee
Copilot Initial plan
001f77c3
Copilot Revert "fix: update 1 file reformatted."
b90aee5a
nathon-lee Merge pull request #5 from nathon-lee/copilot/git-revert-ff886701
b6da9afd
nathon-lee Merge branch 'deepspeedai:master' into master
bb7f64fd
Copilot Initial plan
cbc816c9
Copilot Reapply "fix: update 1 file reformatted."
5fcc9a7e
nathon-lee Merge pull request #6 from nathon-lee/copilot/remove-commits-from-master
f7c5d75d
nathon-lee Merge branch 'deepspeedai:master' into master
18efbcc3
nathon-lee Merge branch 'deepspeedai:master' into master
e2ac74d2
nathon-lee Merge branch 'deepspeedai:master' into master
da07382d
nathon-lee Merge branch 'deepspeedai:master' into master
5d8875cc
nathon-lee Merge branch 'deepspeedai:master' into master
316b6dda
nathon-lee Merge branch 'deepspeedai:master' into master
2020543f
nathon-lee Merge branch 'deepspeedai:master' into master
1a8694c6
nathon-lee Merge branch 'deepspeedai:master' into master
d6725be0
nathon-lee Merge branch 'deepspeedai:master' into master
a06c5487
nathon-lee Merge branch 'deepspeedai:master' into master
6959eb4b
nathon-lee Merge branch 'deepspeedai:master' into master
e88eb3e3
nathon-lee Merge branch 'deepspeedai:master' into master
683bd0bc
nathon-lee nathon-lee marked this pull request as ready for review 1 day ago
nathon-lee nathon-lee requested a review from tjruwase tjruwase 1 day ago
nathon-lee nathon-lee requested a review from loadams loadams 1 day ago
nathon-lee nathon-lee requested a review from tohtana tohtana 1 day ago
nathon-lee nathon-lee changed the title tests: temporarily skip ZeRO-3 in two-loss separate-backward regression tests: adds a regression test for the behavior reported in issue #7352 1 day ago
nathon-lee nathon-lee marked this pull request as draft 1 day ago
nathon-lee nathon-lee force pushed from 52590f30 to 60938ff6 1 day ago
nathon-lee nathon-lee force pushed from 60938ff6 to 5bc41cac 1 day ago
nathon-lee zero3: fix retained-graph second backward (#7352)
b41bb4cc
nathon-lee nathon-lee force pushed from 5bc41cac to b41bb4cc 1 day ago
nathon-lee nathon-lee changed the title tests: adds a regression test for the behavior reported in issue #7352 zero3: defer param release during retain_graph backward #7352 1 day ago
nathon-lee nathon-lee marked this pull request as ready for review 1 day ago
chatgpt-codex-connector
chatgpt-codex-connector commented on 2026-06-03
nathon-lee zero3: cover manual scale().backward() for retain_graph release deferral
5c75f999
nathon-lee nathon-lee requested a review from hwchen2017 hwchen2017 1 day ago
nathon-lee nathon-lee force pushed from 8ae5113e to 5c75f999 1 day ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum