transformers
898efca7 - Fix to removing ESM special tokens (#22870)

Commit

2 years ago

Fix to removing ESM special tokens (#22870) Fix to make sure the EOS token doesn't come back

References

Author

Rocketknight1

Parents