transformers
Fix bug in gpt2's (from-scratch) special scaled weight initialization
#17877

Merged

Fix bug in gpt2's (from-scratch) special scaled weight initialization #17877

sgugger merged 2 commits into huggingface:main from karpathy:fix-gpt2-scaled-init

only special scale init each gpt2 c_proj weight once, on exact match

0637d69e

fix double quotes

0d9b891d

lvwerra commented on 2022-06-25

sgugger approved these changes on 2022-06-25

sgugger merged e02037b3 into main 3 years ago

Reviewers

sgugger

lvwerra

Assignees

No one assigned

Labels

None yet

Milestone

No milestone