transformers
2ba040a7 - apply_chat_template: consistent behaviour for return_assistant_tokens_mask=True return_tensors=True (#35582)

Commit
314 days ago
apply_chat_template: consistent behaviour for return_assistant_tokens_mask=True return_tensors=True (#35582) * apply_chat_template: consistent return_tensors behaviour with return_assistant_tokens_mask flag * test_chat_template_return_assistant_tokens_mask: support tokenizers with no attention mask * test_chat_template_return_assistant_tokens_mask: skip tokenizers with no padding token * test_chat_template_return_assistant_tokens_mask: force tokenizer padding_side=right --------- Co-authored-by: Eduard Allakhverdov <goncharova@airi.net> Co-authored-by: d.tarasov <d.tarasov@airi.net>
Author
Parents
Loading