transformers
b7fe69cb - Fix Reformer LM head bias not applied since the weight-loading refactor (#50811)

Commit
1 day ago
Fix Reformer LM head bias not applied since the weight-loading refactor (#50811) Fix Reformer LM head bias dropped since the weight-loading refactor Since #41580, `ReformerOnlyLMHead.decoder` is created with `bias=False` and the line that set `self.decoder.bias = self.bias` was removed, so the trained output bias stored in checkpoints as `lm_head.decoder.bias` is reported as an unexpected key and never applied. Use `bias=True` as BERT does since #41580, and re-record the 2 seeded random-init LSH tests whose forward outputs change with this change. Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Author
Parents
Loading