Add VibeVoice ASR (#43625)
* Add vibevoice tokenizer files.
* Address style tests.
* Revert to expected outputs previously computed on runner.
* Enable encoder output test.
* Update expected output from runner
* Add note on expected outputs
* remove code link and better init
* Update src/transformers/models/vibevoice_acoustic_tokenizer/modular_vibevoice_acoustic_tokenizer.py
Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com>
* Update src/transformers/models/vibevoice_acoustic_tokenizer/modular_vibevoice_acoustic_tokenizer.py
Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com>
* Update src/transformers/models/vibevoice_acoustic_tokenizer/modular_vibevoice_acoustic_tokenizer.py
Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com>
* Update src/transformers/models/vibevoice_acoustic_tokenizer/modular_vibevoice_acoustic_tokenizer.py
Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com>
* modular
* Same changes to decoder layers.
* Update src/transformers/models/vibevoice_acoustic_tokenizer/modular_vibevoice_acoustic_tokenizer.py
Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com>
* doc nits
* Use decoder_depths for decoder!
* Doc nits
* Nits
* Trim feature extraction for tensor only usage.
* Start files for ASR
* Add cache logic to encoder.
* Nit
* Revert to previous sampling approach.
* Nits
* Passing equivalence test
* Fix for chat template to use sampling rate other than 16kHz
* Better logic for vae sampling?
* More standard conversion script.
* Revert to sample flag
* Nits
* Make style
* Better modular and cleanup.
* update asr docs
* Fix GLM docstring
* Docs, cleanup, nits.
* Nit
* Cleaner modular and nits
* Nits
* Nit
* Skip parallelism
* Update docs.
* Finish integration tests, and nits
* Repo checks
* doc nits
* Doc nits
* Remove bad file
* Skip testing of encoder.
* Shift cache creation to when it's used.
* Shift cache creation to where it's used.
* Updated checkpoint path
* Processor nit
* Modeling and processing tests.
* Nits
* Ensure torch compile and nits
* Update src/transformers/models/vibevoice_asr/modular_vibevoice_asr.py
Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com>
* Refactor vibevoice acoustic tokenizer to have encoder and decoder configs.
* Update asr encoder config directly from tokenizer.
* Nits and make style happy.
* Simplify acoustic tokenizer config.
* Make style
* Update src/transformers/models/vibevoice_asr/modular_vibevoice_asr.py
Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com>
* Renaming and switching away from deprecated approaches.
* Better decode, add test, and update docs.
* Clearer code paths.
* Better pipeline example with exposed post-processing methods
* Docstring nit.
* Use voxtral cache, cleaner token init, better naming of chunk size.
* Add missing docstrings.
* Update to official checkpoint.
---------
Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com>
