transformers
a2a7f716 - [ `TokenizationLlama`] fix the way we convert tokens to strings to keep leading spaces 🚨 breaking fix (#29453)

Commit

1 year ago

[ `TokenizationLlama`] fix the way we convert tokens to strings to keep leading spaces 🚨 breaking fix (#29453) * nit * update test and fix test * fixup

References

#29453 - [ `TokenizationLlama`] fix the way we convert tokens to strings to keep leading spaces 🚨 breaking fix

#29969 - [SigLIP] Add fast tokenizer

#32831 - [Docs] Update resources

#33111 - [Backbone] Remove out_features everywhere

#33174 - [Zero-shot image classification pipeline] Remove tokenizer_kwargs

#39821 - Support MetaCLIP 2

#59 - Fix attention mask handling in EoMT-DINOv3 converter

#41212 - Add EoMT with DINOv3 backbone

#62 - Add initial DEIMv2 model implementation

Author

ArthurZucker

Parents

e677479c

transformers a2a7f716 - [ `TokenizationLlama`] fix the way we convert tokens to strings to keep leading spaces 🚨 breaking fix (#29453)

transformers
a2a7f716 - [ `TokenizationLlama`] fix the way we convert tokens to strings to keep leading spaces 🚨 breaking fix (#29453)