transformers
36e30f0b - add PocketTTS (#49589)

Commit
31 days ago
add PocketTTS (#49589) * add PocketTTS * [processor] audio_tokenizer_kwargs, forwarded to the audio tokenizer's from_pretrained * [processor] load the audio tokenizer in get_processor_dict, as before * [pocket_tts] conversion: latent_proj in the audio tokenizer, frame-size padding in its feature extractor * clean * nit * test cleaning * loss update * proc update * [processor] drop audio_tokenizer_kwargs: only register PocketTTSAudioTokenizerModel as an audio tokenizer * [pocket_tts] docs: place the processor's audio tokenizer with device_map * [pocket_tts] place a generation step's latent position-wise, leaving torch_compilable_check untouched * [pocket_tts] processor tests: chat template from the converted checkpoint, no .jinja fixtures * [pocket_tts] drop the feature extractor tests, covered by the voice-cloning integration tests * [pocket_tts] drop the torch_compilable_check test, its import_utils change is reverted * test updates * add comment * nits * [pocket_tts] docs: load the audio tokenizer without its encoder for cached voices * [pocket_tts] drop the module docstrings that restate the file name * add PocketTTSTokenizer * [pocket_tts] tokenizer: text preparation as tokenizers normalizers, only the used steps * [pocket_tts] drop PocketTTSTokenizer: the normalizer ships in tokenizer.json, the chat template capitalizes * [pocket_tts] processor: split_text skips the markers the vocabulary lacks * [pocket_tts] PocketTTSTokenizer: tokenizer.json's normalizer, then the capitalization * [pocket_tts] back to TokenizersBackend: the capitalization as per-letter normalizer steps * [pocket_tts] training-only state comment: its share of the weights * [pocket_tts] latent statistics in fp32: in bf16, their EMA updates round away - `_keep_in_fp32_modules_strict`: `latent_mean`, `latent_std`, and `freqs` (already built and saved in fp32) - casts back to the input dtype after normalising / denormalising - the EMA gate skips the `.item()` sync once the EMA is over - common `test_keep_in_fp32_modules`: skip integer buffers, which are never cast * [pocket_tts] processor: return_tensors default in tokenizer_config.json, no _defaults (TRF019) * [pocket_tts] tests: readable voice cache loading * [pocket_tts] feature extractor tests: frame-size padding, end_on_pause, get_num_frames * [pocket_tts] drop unused Llama imports * [pocket_tts] allow the LSD loss config attributes in check_config_attributes (read in loss/loss_lsd.py)
Author
Parents
Loading