Add DDP Communication Hooks #2841
muellerzr
merged
29 commits into
huggingface:main
from
yhna940:feature/torch-ddp-grad-hook
Add ddp comm hook
26c6106c
Fix dataclass order
ddd8ef55
Merge ddp grad hook to ddp kwargs handler
f574aa3e
Reset ddp kwargs key
96e46e6b
Add test
619005aa
Fix test case
ac10b756
Split ddp grad test
2b5222bd
Fix test case
1025b52c
Merge remote-tracking branch 'origin/main' into feature/torch-ddp-gra…
f89b1bbb
Ehance docstring
e70a5bee
Minor
66275d01
Use naive baseenum for ddp comm hook type
82b20c16
Add by feature example
435f5fc3
Add multi device deco
d0bdbc6f
Add user guide
fa07bfb2
Update examples/by_feature/ddp_comm_hook.py
60b287ed
Update examples/by_feature/ddp_comm_hook.py
e4109bd4
SunMarc
approved these changes
on 2024-06-12
Add wrapper and state option details
00661a5b
SunMarc
approved these changes
on 2024-06-12
Update toctree
1638d782
Update docs/source/usage_guides/ddp_comm_hook.md
d62e953d
Update docs/source/usage_guides/ddp_comm_hook.md
3b48f23d
Update docs/source/usage_guides/ddp_comm_hook.md
f698e8af
Update docs/source/usage_guides/ddp_comm_hook.md
5ae0127a
Update docs/source/usage_guides/ddp_comm_hook.md
a81d48ec
Update docs/source/usage_guides/ddp_comm_hook.md
ab7113c8
Update docs/source/usage_guides/ddp_comm_hook.md
8d0014ae
Mv ddp comm hook index
2358f53e
Fix ddp comm hook user guid
693c3968
Del empty line
c0970352
Login to write a write a comment.
Login via GitHub