Add moonshine streaming (#43702)
* Add Moonshine Streaming
* sliding window attention in encoder (first 2 and last 2 layers)
* different dims for encoder and decoder
* projection and position embedding addition in adapter
* Move fully defined MoonshineStreamingConfig to configuration
Not auto generated anymore.
* Move preprocessor in to encoder
* Inherit from MoonshinePretrainedModel
* Reuse MoonshineAttention
* Remove unused imports
* Add moonshine_streaming to SEQ_2_SEQ dict for pipeline to work
* batch_decode->decode to keep it v5 friendly
* Remove unsued get_logits, move float cast up
* Make the frame overlap function tracing friendly
* draft modular refacto
* update test values
* working commit
* update
* some more updates
* config udpate
* some more updates
* udpate modular
* Minor changes to make the tests pass
Mostly fixing the config such that the encoder and decoder dimensions can be
different.
* Minor fix to numbers to match GPU output
* Use the correct config attribute
* Moved encoder dimensions fixup to config
* Rely on encoder_config only and not encoder_ prefix
* Removed decoder_hidden_act parameter to config
* make
* updates
* revert conversion mapping
* fix output attention and hidden states
* update doc
* fix test
* doc
* add config doc
* move in init
* nit
* revert output_attentions output_hidden_states
* fix
* fix
* fix
* doc update
* gradient checkpointing
* address review
---------
Co-authored-by: Manjunath Kudlur <keveman@usefulsensors.com>
Co-authored-by: Manjunath Kudlur <keveman@gmail.com>
