transformers
83dbebc4 - [trainer] ensure special tokens in model configs are aligned with tokenizer at train time (#38441)

Commit

282 days ago

[trainer] ensure special tokens in model configs are aligned with tokenizer at train time (#38441) * tmp commit * add test * make fixup * reset warns/info in test

References

#38441 - [trainer] ensure special tokens in model configs are aligned with tokenizer at train time

Author

gante

Parents

9977cf17

transformers 83dbebc4 - [trainer] ensure special tokens in model configs are aligned with tokenizer at train time (#38441)

transformers
83dbebc4 - [trainer] ensure special tokens in model configs are aligned with tokenizer at train time (#38441)