transformers
apply_chat_template: consistent behaviour for return_assistant_tokens_mask=True return_tensors=True
#35582
Merged
Merged
apply_chat_template: consistent behaviour for return_assistant_tokens_mask=True return_tensors=True #35582
ArthurZucker
merged
4 commits into
huggingface:main
from
mrsndmn:apply_chat_template_return_assistant_masks_return_tensors
apply_chat_template: consistent return_tensors behaviour with return_…
7756366c
test_chat_template_return_assistant_tokens_mask: support tokenizers w…
db7e2b26
test_chat_template_return_assistant_tokens_mask: skip tokenizers with…
10f9b596
test_chat_template_return_assistant_tokens_mask: force tokenizer padd…
7349d163
ArthurZucker
approved these changes
on 2025-02-04
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub