🚨🚨🚨 `[NLLB Tokenizer]` Fix the prefix tokens 🚨🚨🚨 (#22313)
* fix the prefix tokens
* update fast and test values
* add legacy behaviour
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
* update disclaimer, linkissue PR and behaviral changes
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <hi@lysand.re>
* styling
* make a quote
* quote this time
---------
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Lysandre Debut <hi@lysand.re>