[`Whisper`] Add conversion script for the tokenizer #27338
ArthurZucker
merged
25 commits into
main
from
whisper-v3-nots
draft
63a860cd
updates
8e04b79a
full conversion taken from `https://gist.github.com/xenova/a452a64744…
f1b3b5ba
psuh
412bd98a
nits
deb624a3
updates
491352ca
more nits
4ddc1e93
Add co author
16d01528
fixup
5f63066c
cleanup
1512ea1a
styling
26aa3ab2
add proper path
10e611f1
update
18325b15
nits
8a9c954e
don't push the exit
a78464ca
clean
7f826a79
update whisper doc
fcbe06b1
don't error out if tiktoken is not here
c7cbe886
make sure we are BC with conversion
1efcd15d
nit
608ca8c0
ArthurZucker
commented
on 2023-11-07
Merge branch 'main' of github.com:huggingface/transformers into whisp…
8eedcb0b
Update docs/source/en/model_doc/whisper.md
d9eeee89
merge and update
2350a287
update markdwon
47c30651
ArthurZucker
commented
on 2023-11-07
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply…
ea118208
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub