transformers
81df408a - [MistralCommonBackend] Stop relying on mistral-common's Tokenized.text field (#47646)

Commit

6 days ago

[MistralCommonBackend] Stop relying on mistral-common's Tokenized.text field (#47646) Stop relying on mistral-common's Tokenized.text and decode tokens explicitly

References

#47646 - [MistralCommonBackend] Stop relying on mistral-common's Tokenized.text field

Author

juliendenize

Parents

fd12552d

transformers 81df408a - [MistralCommonBackend] Stop relying on mistral-common's Tokenized.text field (#47646)

transformers
81df408a - [MistralCommonBackend] Stop relying on mistral-common's Tokenized.text field (#47646)