PR #1292 API for obtaining global gradient norm

API for obtaining global gradient norm #1292

tjruwase merged 4 commits into big-science from olruwase/global_gradient_norm

FP16 fused and unfused grad norm query.

e35dd697

Merge branch 'big-science' of github.com:microsoft/DeepSpeed into big…

f6b65ad0

API for obtaining global unclipped gradient norm across parameter groups

5cda8e51

tjruwase requested a review from

ShadenSmith 4 years ago

tjruwase requested a review from

awan-10 4 years ago

tjruwase requested a review from

cli99 4 years ago

tjruwase requested a review from

conglongli 4 years ago

tjruwase requested a review from

eltonzheng 4 years ago

tjruwase requested a review from

jeffra 4 years ago

tjruwase requested a review from

minjiaz 4 years ago

tjruwase requested a review from

niumanar 4 years ago

tjruwase requested a review from

RezaYazdaniAminabadi 4 years ago

tjruwase requested a review from

samyam 4 years ago

tjruwase removed review request from

samyam 4 years ago

tjruwase removed review request from

conglongli 4 years ago

tjruwase removed review request from

awan-10 4 years ago

tjruwase removed review request from

cli99 4 years ago

tjruwase removed review request from

eltonzheng 4 years ago

tjruwase removed review request from

minjiaz 4 years ago

tjruwase removed review request from

RezaYazdaniAminabadi 4 years ago

tjruwase removed review request from

niumanar 4 years ago

tjruwase requested a review from

samyam 4 years ago

Use global norm not group norms

dd02eee5

ShadenSmith approved these changes on 2021-08-09

tjruwase merged cce85b89 into big-science 4 years ago

mrwyattii deleted the olruwase/global_gradient_norm branch 2 years ago

Reviewers

ShadenSmith

jeffra

samyam

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed API for obtaining global gradient norm #1292 Merged

API for obtaining global gradient norm #1292

DeepSpeed
API for obtaining global gradient norm
#1292

Merged