DeepSpeed Communication Profiling and Logging #2012
jeffra
merged
54 commits into
master
from
staging-comms-logging-v1
Staging comms v1 (#301)
867a8537
Delete stage1.py
c93fcfef
Delete distributed.py
7f8ca013
revert deepspeed/__init__.py logging calls
977ee324
Delete test.py
68eb9f4e
Update comments and move custom comm ops to internal functions
54796bb8
Merge branch 'staging-comms-next' of https://github.com/microsoft/Dee…
c06c72d3
Remove unnecessary print and update backend description
f070a0c8
Relax assertion to allow Megatron-DeepSpeed MoE to use ZeRO 1
9976681a
Simplify ZeRO stage 1 check for previous commit
09063a3d
Remove misleading world_size prints
656b4152
Add commslogger class, and introduce rough prototype comms logging
2e7129c6
Clean up logger
0023b3e1
Add more robust arg checks
e55c8e93
Add labels to common collective calls for logger
31c7dcf7
Add more annotations
8e23f504
Fix up log_summary_new and fix logging bug for barrier
79983505
Clean up arg sweep logic and add isend/irecv
227874e1
Merge branch 'master' into staging-comms-logging-v1
27c38f9b
Clean up logging branch
26e15aef
Unify naming and fix circular import
3aa3e383
Fix deepspeed comm imports for logging.py
d2561dca
Added comms config support, removed some log names
c85f3c1c
Add comms config file
f70addba
Add pydantic to requirements
a1533316
Add configure non-op to old torch
351f384d
Update logging call for old torch
bcb3afd4
Add log_name placeholder args for old torch
2f8320a2
Add basic verbosity setup
95aa7d86
Complete verbosity setup
93d1a314
move comms logging to separate file and clean up
4a6236d3
Change debug message design
393c90a4
refactor debug helper and clean up
527d1c8c
Refactor a bit and clean up prints
40482a83
Merge branch 'master' into staging-comms-logging-v1
a6beecf1
config docs, remove old log_summary func, fix imports
9343f878
Finished docs, added import, fixed non-debug calls
c07bc134
Ran pre-commit
f5fd1f29
Removed old comments
1b317985
Updated fn signatures for torch1.2
298349d7
Remove lingering prof arg
102ae1d6
Merge branch 'master' into staging-comms-logging-v1
2185f168
Update logging tutorial
4faf3b94
Quentin-Anthony
commented
on 2022-06-30
Quentin-Anthony
commented
on 2022-06-30
Quentin-Anthony
commented
on 2022-06-30
Quentin-Anthony
commented
on 2022-06-30
Quentin-Anthony
commented
on 2022-06-30
Quentin-Anthony
commented
on 2022-06-30
Quentin-Anthony
commented
on 2022-06-30
Quentin-Anthony
commented
on 2022-06-30
Quentin-Anthony
changed the title
Removed unnecessary imports and cleaned up comments
6381187f
Take master's cleaner comms init logic
56dbd71b
Fixed bw calculations and made all logging calls blocking
ae524f04
Added comms logging synch disclaimer
19bcf79c
jeffra
commented
on 2022-07-21
jeffra
approved these changes
on 2022-07-21
Merge branch 'master' into staging-comms-logging-v1
b9cb4d36
Added using_mpi flag for logging
c6925a1d
Formatting
5a0715c8
Merge branch 'master' of https://github.com/microsoft/DeepSpeed into …
b4449a2e
Merge branch 'master' into staging-comms-logging-v1
b6489791
Merge branch 'master' into staging-comms-logging-v1
9357a168
Merge branch 'master' into staging-comms-logging-v1
c85e3235
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub