transformers
Explicitly setting encoding in tokenization_utils_base.py
#38553

Merged

Explicitly setting encoding in tokenization_utils_base.py #38553

Rocketknight1 merged 1 commit into huggingface:main from Muqi1029:patch-1

Muqi1029 changed the title ~~Update tokenization_utils_base.py~~ Explicitly setting encoding in tokenization_utils_base.py 1 year ago

Rocketknight1 approved these changes on 2025-06-03

Rocketknight1 enabled auto-merge (squash) 1 year ago

Update tokenization_utils_base.py

3f637865

Rocketknight1 force pushed from 93f1c102 to 3f637865 1 year ago

Rocketknight1 merged 8cb96787 into main 1 year ago

Muqi1029 deleted the patch-1 branch 1 year ago

Reviewers

Rocketknight1

Assignees

No one assigned

Labels

None yet

Milestone

No milestone