Megatron-DeepSpeed
07f0ce57 - I think GPT2 tokenizer might be missing <sep> token

Commit

3 years ago

I think GPT2 tokenizer might be missing <sep> token

References

#286 - WIP: Shared t5 code

Author

thomasw21

thomasw21

Parents

Loading