Megatron-DeepSpeed
[debug] ModelInspector
#155
Open
Overview
Commits
25
Changes
View On GitHub

[debug] ModelInspector #155

jaketae wants to merge 25 commits into main from log-grad-norm
jaketae
feature: add debug grad norm cls
222bb0b3
jaketae jaketae marked this pull request as draft 4 years ago
jaketae jaketae assigned jaketae jaketae 4 years ago
jaketae jaketae changed the title Gradient norm logger tool Gradient norm logger 4 years ago
jaketae
stas00
jaketae
stas00
jaketae
stas00
jaketae
stas00
jaketae
fix: use named_modules, add min max logging
eabb8c5a
fix: grad_outputs instead of inputs, iterate over tuple
4f42e591
stas00
jaketae
fix: check if `grad_output` is `None`
dd3bef26
fix: clone input for backward hook
daa2b630
jaketae jaketae marked this pull request as ready for review 4 years ago
jaketae jaketae requested a review from stas00 stas00 4 years ago
stas00
stas00 make it work with any torch version
f301ffd9
stas00
stas00
jaketae
stas00
docs: add comment on why `.clone()` was added
1cce6594
chore: clean up print format
934c0b30
stas00
jaketae
stas00
stas00
stas00
stas00 experimental forward tb logger
e4445dab
stas00 Merge remote-tracking branch 'origin/main' into log-grad-norm
ab0f78ed
stas00
stas00 rework to integrate real iterations
6b11ad70
jaketae
stas00 stas00 removed review request from stas00 stas00 4 years ago
stas00
stas00 reworking the state dumper
5edefea4
stas00
stas00 update
11ef669e
stas00 Merge remote-tracking branch 'origin/main' into log-grad-norm
f1c9ff89
stas00 Merge remote-tracking branch 'origin/main' into log-grad-norm
98e153f3
stas00 Merge remote-tracking branch 'origin/main' into log-grad-norm
cc6d66a8
stas00 Merge remote-tracking branch 'origin/main' into log-grad-norm
114c473f
stas00 Merge remote-tracking branch 'origin/main' into log-grad-norm
2c9c6655
stas00 switch to pp_rank as a global unique identifier
0aeab4f8
stas00 skip params on bwd; add argmin/argmax
349c45c7
stas00 rename to an easier to remember ModuleInspector
5aa8c248
stas00 rename to an easier to remember ModelInspector
f135a172
stas00 stas00 changed the title Gradient norm logger [debug] ModelInspector 4 years ago
stas00
stas00 Merge remote-tracking branch 'origin/main' into log-grad-norm
3dd99121
stas00 Merge remote-tracking branch 'origin/main' into log-grad-norm
20c9bfb3
stas00 Merge remote-tracking branch 'origin/main' into log-grad-norm
f689231c

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum