Transformer/fix layer norm #1404
RezaYazdaniAminabadi
wants to merge
5 commits into
master
from
transformer/fix-layer-norm
fix the workspace allocation for the transformer kernel
4017a7c9
Modify normalize kernels to save variance in floating-point for backward
f26900ec
remove unused grad keys
5eae60f0
Merge branch 'master' into transformer/fix-layer-norm
af34cc47
hyunwoongko
commented
on 2021-10-01
fix some issue with saving the variance for normalize forward kernel
135ffe30
Reviewers
Assignees
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub