transformers
a3536326 - Compute MoE load-balancing loss per layer to avoid giant one-hot materialization −99.7% @ 128k (#48131)

Commit

4 days ago

Compute MoE load-balancing loss per layer to avoid giant one-hot materialization −99.7% @ 128k (#48131) Compute MoE load-balancing loss per layer to avoid giant one-hot materialization

References

#48131 - Compute MoE load-balancing loss per layer to avoid giant one-hot materialization −99.7% @ 128k

Author

qgallouedec

Parents

a530903e

transformers a3536326 - Compute MoE load-balancing loss per layer to avoid giant one-hot materialization −99.7% @ 128k (#48131)

transformers
a3536326 - Compute MoE load-balancing loss per layer to avoid giant one-hot materialization −99.7% @ 128k (#48131)