[Whisper] Fix timestamp processor #21187
ArthurZucker
merged
76 commits into
huggingface:main
from
ArthurZucker:whisper-timestamp-processor
add draft logit processor
ee96bc25
add template functions
757c78c3
Merge branch 'main' of https://github.com/huggingface/transformers in…
f9f89588
update timesapmt processor parameters
b8de9ed6
draft script
e600cee0
simplify code
a3e69a16
cleanup
7deb7667
fixup and clean
69a41572
update pipeline
672199c3
style
157ad3f3
clean up previous idea
b2b28f6c
add tokenization utils
d3f7ae19
update tokenizer and asr output
f2ae07c2
fit whisper type
fafa6703
style and update test
f9204447
clean test
4ee2ae78
style test
55bd2688
update tests
6efafac6
update error test
111a52d2
udpate code (not based on review yet)
3ae58770
update tokenization
8ae9a8ae
update asr pipeline
6a644786
Merge branch 'main' of https://github.com/huggingface/transformers in…
8f272c2f
update code
2f7ac2d8
cleanup and update test
47433e75
fmt
c4dd8a11
remove text verificatino
9dd27b6b
cleanup
fc91e026
cleanup
60947e53
add model test
6548a0f3
update tests
04e25688
update code add docstring
c79fca1e
update code and add docstring
41d453fc
fix pipeline tests
077de885
add draft logit processor
22864b85
Small update.
6b6c6beb
Fixup.
778d102b
Tmp.
b0b7389c
More support.
aa0e90ed
Making `forced_decoder_ids` non mandatory for users to set.
e729ddc9
Merge branch 'whisper-timestamp-processor' of https://github.com/Arth…
fcabe66b
update and fix first bug
4f0fe7d2
properly process sequence right after merge if last
1882b090
tofo
cd21c0fc
allow list inputs + compute begin index better
2268fd46
start adding tests
de076dbb
add the 3 edge cases
5b9cbd52
style
221b39bf
format sequences
7abab3c0
fixup
aec06ed0
update
26701976
update
3ed833d5
style
5d74a10f
test passes, edge cases should be good
c8027bf5
update last value
a302560d
remove Trie
ddbe998b
update tests and expec ted values
56ddecc3
handle bigger chunk_length
15573920
clean tests a bit
32ca6937
refactor chunk iter and clean pipeline
fd8a211a
update tests
a450e755
style
6bdc97e4
refactor chunk iter and clean pipeline
41ac1885
upade
9b806df8
resolve comments
2a7624e1
Apply suggestions from code review
ca7dd752
take stride right into account
95b55bbd
update test expected values
a63b102b
Merge branch 'whisper-timestamp-processor' of https://github.com/Arth…
64221d78
Update code based on review
f00c8be5
major refactor
d96d41ef
add correct strides for tests
30ae5f0a
Merge branch 'main' of https://github.com/huggingface/transformers in…
e842fb65
ArthurZucker
commented
on 2023-01-19
ArthurZucker
commented
on 2023-01-19
ArthurZucker
commented
on 2023-01-19
ArthurZucker
commented
on 2023-01-19
ArthurZucker
commented
on 2023-01-19
Update src/transformers/pipelines/automatic_speech_recognition.py
c0cc996a
fix whisper timestamp test
1d62a6a5
Merge branch 'whisper-timestamp-processor' of https://github.com/Arth…
7209088b
sgugger
approved these changes
on 2023-01-19
Narsil
approved these changes
on 2023-01-19
Login to write a write a comment.
Login via GitHub