Add Number Normalisation for SpeechT5 #25447
ArthurZucker
merged
8 commits into
huggingface:main
from
tanaymeh:add_normalization_speecht5
add: NumberNormalizer works for integers, floats, common currencies, …
2211a405
fix: renamed number normalizer class and added normalization to Speec…
05ef5427
fix: restyled with black and ruff, should pass code quality tests
00f88928
fix: moved normalization to tokenizer and other small changes to norm…
ed2e0aa6
add: test for normalization and changed the existing full tokenizer test
28c1240f
fix: tokenization tests now pass, made changes to existing tokenizati…
afa90898
fix: changed default normalize setting to False, modified the tests a…
a49f17c8
fix: added support for comma separated numbers, tokenization on the f…
682bee8d
ArthurZucker
approved these changes
on 2023-08-18
sanchit-gandhi
commented
on 2023-08-21
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub