PR #31938 clean_up_tokenization_spaces=False if unset

clean_up_tokenization_spaces=False if unset #31938

ArthurZucker merged 10 commits into main from clean_up_tokenization_spaces_false_default

itazap requested a review from

ArthurZucker 2 years ago

ArthurZucker commented on 2024-07-22

itazap force pushed 1 year ago

itazap force pushed to ec6f78a1 1 year ago

itazap commented on 2024-07-25

itazap marked this pull request as ready for review 1 year ago

itazap requested a review from

ArthurZucker 1 year ago

itazap commented on 2024-07-26

ArthurZucker commented on 2024-07-31

clean_up_tokenization_spaces=False if unset

676b55e5

deprecate warning

50029b2a

updating param for old models

cc7d05fd

update models

f0b4591f

make fix-copies

227d54e6

fix-copies and update bert models

8d047bc4

warning msg

1995376f

update prophet and clvp

15111007

updating test since space before is arbitrarily removed

b7b2b097

itazap force pushed from 45b6a0e8 to b7b2b097 1 year ago

itazap requested a review from

ArthurZucker 1 year ago

ArthurZucker approved these changes on 2024-09-25

ArthurZucker commented on 2024-09-26

remove warning for 4.45

9c344440

ArthurZucker merged 6730485b into main 1 year ago

ArthurZucker deleted the clean_up_tokenization_spaces_false_default branch 1 year ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers clean_up_tokenization_spaces=False if unset #31938 Merged

clean_up_tokenization_spaces=False if unset #31938

transformers
clean_up_tokenization_spaces=False if unset
#31938

Merged