transformers
[Json configs] Make json prettier for all saved tokenizer files & ensure same json format for all processors (tok + feat_extract)
#17457

Merged

[Json configs] Make json prettier for all saved tokenizer files & ensure same json format for all processors (tok + feat_extract) #17457

patrickvonplaten merged 8 commits into huggingface:main from patrickvonplaten:correct_saving_format_json

[Json dump] Make json prettier

abbc4b82

correct more tokenizeirs

2fb3a1a2

more patterns

82ab50aa

patrickvonplaten changed the title ~~[Json dump] Make json prettier for all processors~~ [Json dump] Make json prettier for all saved tokenizeir filese 3 years ago

patrickvonplaten changed the title ~~[Json dump] Make json prettier for all saved tokenizeir filese~~ [Json dump] Make json prettier for all saved tokenizer files 3 years ago

Merge branch 'main' of https://github.com/huggingface/transformers in…

e48b48ee

add aggressive test

542ea2ea

the aggressive test was actually useful :-)

872e59f9

more tests

c14b7e79

patrickvonplaten changed the title ~~[Json dump] Make json prettier for all saved tokenizer files~~ [Json dump] Make json prettier for all saved tokenizer files & ensure same json format for all processors (tok + feat_extract) 3 years ago

patrickvonplaten changed the title ~~[Json dump] Make json prettier for all saved tokenizer files & ensure same json format for all processors (tok + feat_extract)~~ [Json configs] Make json prettier for all saved tokenizer files & ensure same json format for all processors (tok + feat_extract) 3 years ago

patrickvonplaten requested a review from

sgugger 3 years ago

patrickvonplaten requested a review from

julien-c 3 years ago

patrickvonplaten commented on 2022-05-27

Apply suggestions from code review

ca9ef7f2

patrickvonplaten commented on 2022-05-27

julien-c commented on 2022-05-27

sgugger approved these changes on 2022-05-31

patrickvonplaten requested a review from

LysandreJik 3 years ago

LysandreJik approved these changes on 2022-05-31

patrickvonplaten merged f394a2a5 into main 3 years ago

patrickvonplaten deleted the correct_saving_format_json branch 3 years ago

Reviewers

LysandreJik

sgugger

julien-c

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers [Json configs] Make json prettier for all saved tokenizer files & ensure same json format for all processors (tok + feat_extract) #17457 Merged

[Json configs] Make json prettier for all saved tokenizer files & ensure same json format for all processors (tok + feat_extract) #17457

transformers
[Json configs] Make json prettier for all saved tokenizer files & ensure same json format for all processors (tok + feat_extract)
#17457

Merged