transformers
add custom RMSNorm to `ALL_LAYERNORM_LAYERS`
#26227
Merged
Overview
Commits
3
Changes
View On GitHub

add custom RMSNorm to `ALL_LAYERNORM_LAYERS` #26227

ArthurZucker merged 3 commits into huggingface:main from shijie-wu:llama-decay
shijie-wu
shijie-wu add LlamaRMSNorm to ALL_LAYERNORM_LAYERS
f7f22562
ArthurZucker
ArthurZucker approved these changes on 2023-09-18
shijie-wu fixup
b8dd436b
shijie-wu add IdeficsRMSNorm to ALL_LAYERNORM_LAYERS and fixup
493b24ba
shijie-wu
shijie-wu shijie-wu requested a review from ArthurZucker ArthurZucker 2 years ago
ArthurZucker
ArthurZucker approved these changes on 2023-09-20
shijie-wu shijie-wu changed the title add `LlamaRMSNorm` to `ALL_LAYERNORM_LAYERS` add custom RMSNorm to `ALL_LAYERNORM_LAYERS` 2 years ago
HuggingFaceDocBuilderDev
shijie-wu
ArthurZucker ArthurZucker merged e3a4bd2b into main 2 years ago
shijie-wu shijie-wu deleted the llama-decay branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum