PR #13757 convert : fix nomic-bert-moe mask token

convert : fix nomic-bert-moe mask token #13757

CISC merged 9 commits into master from cisc/fix-nomic-bert-moe-mask

add missing mask token

cb5d91b5

set mask token lstrip attribute

c55c1baf

str -> bytes

57c8470d

github-actions added python

CISC requested a review from

slaren 262 days ago

CISC requested a review from

compilade 260 days ago

Merge branch 'master' into cisc/fix-nomic-bert-moe-mask

5030d48a

CISC removed review request from

slaren 257 days ago

remove dead padding code that got readded in merge

958eea66

apply correct padding with AutoTokenizer as well

7789616e

spaces--

a835a0c0

compilade commented on 2025-05-27

Merge branch 'master' into cisc/fix-nomic-bert-moe-mask

29462773

only add mask token when using sentencepiece

4f2f228e

CISC requested a review from

compilade 254 days ago

compilade approved these changes on 2025-06-01

CISC merged 5e1c3aed into master 254 days ago

CISC deleted the cisc/fix-nomic-bert-moe-mask branch 254 days ago

Reviewers

compilade

Assignees

No one assigned

Labels

python

Milestone

No milestone