transformers
06e737fb - Tweak ESM tokenizer for Nucleotide Transformer (#22770)

Commit

2 years ago

Tweak ESM tokenizer for Nucleotide Transformer (#22770) * If EOS is None, don't add it to sequences * If EOS is None, don't add it to sequences

References

Author

Rocketknight1

Parents