transformers
e02037b3
- Fix bug in gpt2's (from-scratch) special scaled weight initialization (#17877)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
3 years ago
Fix bug in gpt2's (from-scratch) special scaled weight initialization (#17877) * only special scale init each gpt2 c_proj weight once, on exact match * fix double quotes Co-authored-by: leandro <leandro.vonwerra@spoud.io>
References
#17877 - Fix bug in gpt2's (from-scratch) special scaled weight initialization
Author
karpathy
Parents
6dd00f6b
Loading