transformers
860d11ff - Fix Debertav2 embed_proj (#24205)

Commit
2 years ago
Fix Debertav2 embed_proj (#24205) * MLM prediction head output size from embed_size Take the output size of the dense projection layer from embedding_size instead of hidden_size since there could be a projection of the input embedding into hidden_size if they are different * project TFDebertaV2 mlm output to embedding size embedding size can be different that hidden_size, so the final layer needs to project back to embedding size. like in ELECTRA or DeBERTaV3 style pertaining. This should solve an error that occurs when loading models like "almanach/camemberta-base-generator". * fix the same issue for reshaping after projection * fix layernorm size * add self.embedding_size to scope * fix embed_proj scope name * apply the same changes to TF Deberta * add the changes to deberta * added self.embedding_size instead of config.embedding_size * added the same change to debertav2 * added coppied from deberta to deberta2 model * config.embedding_size fix * black * fix deberta config name
Author
Parents
Loading