flax
8a090e3e
- Allow Adafactor to not update certain parameters proportional
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
4 years ago
Allow Adafactor to not update certain parameters proportional to their scale. This is a potential cause of bad quality when the relative positional embeddings are updated proportional to their scale in large LMs. PiperOrigin-RevId: 368110589
References
test_368110589
Author
a-googler
Committer
a-googler
Parents
4a0e32e5
Loading