DeepSpeed
Checks for None tensors and skip them when splitting the buckets in zero stage 2.
#728
Merged
Overview
Commits
48
Changes
View On GitHub

Checks for None tensors and skip them when splitting the buckets in zero stage 2. #728

cli99 merged 48 commits into deepspeedai:master from cli99:fix_zero2_none_tensor
cli99
cli99 add flops count profiler
86a33ca5
cli99 add run script for the small test model
eedb939c
cli99 fix flops sum and add batch counter
0b0b7270
cli99 remove hook handles
558ad138
cli99 add conv2d flops
22ae7520
cli99 add flops compute for major functionals and rnn modules
b093b716
cli99 work
5ce1d1fa
cli99 fix flops count cal in post hook
2575dd39
cli99 add flops of embedding and dropout as 0
f5d18923
cli99 fix
d4d332e1
cli99 add duration and throughput
b3914c7f
cli99 add time and throughput
dbc24c85
cli99 add basic tracer for wall clock time breakdown
3ad7bc12
cli99 Added top module info summary
e8bb9f48
cli99 refactor
e67c6504
cli99 refactor and add readme
4c9db1af
cli99 reorg folders
01db9435
cli99 update readme
f91b552a
cli99 fix xsp init import
964c689b
cli99 update readme and rename batch to step
68b5260a
cli99 fix multiple steps calc
9f38c8db
cli99 update readme
83ba5edb
cli99 rename pytorch-profiler to flops-profiler
9bd30d3a
cli99 rename pytorch-profiler to flops-profiler
a061bf37
cli99 fix steps calc and update readme
c6a6a3f7
cli99 update readme
9c4f3ad1
cli99 update ds
10aee869
cli99 remove tracer code
aecccaf5
cli99 fix incorrect merging
c6dfa070
cli99 fix formatting
b48ed309
cli99 fix
ab576677
cli99 fix sort value divided by steps
6e5ad79f
cli99 print info when steps is 0
03a521cb
jeffra Merge branch 'master' into cheng/flops_profiler
9b1a7b62
cli99 add flops profiler class and unit test
8ad406b6
cli99 delete examples
99e97e1c
cli99 add reload functionals in end profile
8403aaff
cli99 use config dict for flops profiler config
d1aaaf96
cli99 add doc strings and update readme
b26f2fbe
cli99 Merge branch 'master' into cheng/flops_profiler
cb2dfa0f
cli99 Merge branch 'master' into cheng/flops_profiler
78c4cc37
cli99 Merge branch 'master' into cheng/flops_profiler
0a1b04f5
cli99 Merge remote-tracking branch 'upstream/master' into master
5019a6eb
cli99 Merge remote-tracking branch 'upstream/master'
859b837a
cli99 Merge remote-tracking branch 'upstream/master' into master
625df299
cli99 check none tensors when splitting buckets
31830847
cli99 cli99 requested a review from arashashari arashashari 5 years ago
cli99 cli99 requested a review from awan-10 awan-10 5 years ago
cli99 cli99 requested a review from conglongli conglongli 5 years ago
cli99 cli99 requested a review from eltonzheng eltonzheng 5 years ago
cli99 cli99 requested a review from jeffra jeffra 5 years ago
cli99 cli99 requested a review from minjiaz minjiaz 5 years ago
cli99 cli99 requested a review from niumanar niumanar 5 years ago
cli99 cli99 requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 5 years ago
cli99 cli99 requested a review from samyam samyam 5 years ago
cli99 cli99 requested a review from ShadenSmith ShadenSmith 5 years ago
cli99 cli99 requested a review from tjruwase tjruwase 5 years ago
cli99 cli99 marked this pull request as draft 5 years ago
cli99 Merge branch 'master' into fix_zero2_none_tensor
246d53bb
cli99 cli99 marked this pull request as ready for review 5 years ago
samyam
samyam approved these changes on 2021-02-16
cli99 Merge branch 'master' into fix_zero2_none_tensor
dbbbb7f5
cli99 cli99 merged 7cab55c7 into master 5 years ago
cli99 cli99 deleted the fix_zero2_none_tensor branch 5 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum