Refactor MistralTokenizer #26358
DarkLight1337
merged
16 commits into
vllm-project:main
from
juliendenize:refactor_mistral_tokenizer
gemini-code-assist
commented
on 2025-10-07
DarkLight1337
commented
on 2025-10-07
patrickvonplaten
commented
on 2025-10-07
patrickvonplaten
commented
on 2025-10-07
patrickvonplaten
commented
on 2025-10-07
patrickvonplaten
approved these changes
on 2025-10-07
DarkLight1337
approved these changes
on 2025-10-08
disabled auto-merge 133 days ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
disabled auto-merge 133 days ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
Refactor MistralTokenizer
5d4f8088
Fix
ef9f9fde
Fix mypy
9391ae17
Fix md
b55b9dc0
Revert suboptimized decoding
82ed2a59
Revert suboptimized decoding
4b2df0c2
Make truncation condition clear
d900b724
Refactor do_truncation condition
4721d75d
Change spm model to not gated
858a7efe
Fix grammar guidance
54400139
Fix grammar guidance
d4522673
Fix grammar guidance
f8725e3e
Fix pre-commit issue (not relevant to this pr)
b98e7472
Sort vocab dict by token id
18da4678
Merge branch 'main' into refactor_mistral_tokenizer
3926ba4a
Merge branch 'main' into refactor_mistral_tokenizer
9df26723
Reviewers
Assignees
No one assigned
Labels
documentation
structured-output
frontend
ready
ci/build
v1
multi-modality
tool-calling
Milestone
No milestone
Login to write a write a comment.
Login via GitHub