transformers
Add Nemotron ASR Streaming
#46332
Merged

Add Nemotron ASR Streaming #46332

eustlb merged 107 commits into main from nemotron-asr
eustlb
eustlb add ParakeetForRNNT
e03fdbb6
eustlb ParakeetForRNNT config
328c94e3
eustlb generation mixin for RNNT
178a33b3
eustlb conversion script update
f406106b
eustlb auto mappings
38d72d87
eustlb tests
1efefcfc
HuggingFaceDocBuilderDev
eustlb eustlb force pushed from de296c0c to 55b042d5 171 days ago
eustlb eustlb changed the base branch from main to parakeet-rnnt-from-scratch 171 days ago
eustlb draft PR
e78d8f85
eustlb add chunking logic to the feature extractor
39182c74
eustlb on top of #46331
783f67e8
eustlb eustlb force pushed from 55b042d5 to 783f67e8 171 days ago
eustlb better streaming design
6d6dfa97
eustlb RNNT as the main class
aee4bb7b
eustlb rnn-t as main config
29f75424
eustlb nit
a90b87d4
eustlb wip: streaming encoder cache (pre-merge snapshot)
4ac90061
eustlb Merge remote-tracking branch 'origin/parakeet-rnnt-from-scratch' into…
721d9514
eustlb prop changes to modular
f962d4e6
eustlb nit
851a5d72
eustlb update test with reproducers
f38f7510
eustlb add RNN-T loss
a1c0785b
eustlb fix generate
b7e43d07
eustlb correct loss function usage
299f1494
eustlb processing handle ctc/rnnt/tdt diffs
23a01a99
eustlb udpate doc + fix pipeline
4bdf5c8b
eustlb working commit
9119f8c7
eustlb cleaner
d8739626
eustlb nits
17356f00
eustlb fix
fb763505
eustlb fix
d0d78c24
eustlb nit
eb35831e
eustlb add nemotron asr processor
63fc55b3
eustlb tmp commit
8908100d
eustlb update loss reduction to match NeMo
8c3e43ae
eustlb udpate expected value
f4f2c771
eustlb loss reduction
a67da591
eustlb loss reduction to tdt loss
ddcb571e
eustlb conversion udpate
182f4d53
eustlb fix
9b3ecc73
eustlb update checkpoint
a8e121b6
eustlb Merge branch 'main' into parakeet-rnnt-from-scratch
bc77d82b
eustlb nit
31a1ea7f
eustlb nit
3ae2ee49
eustlb fix tdt loss
9d6e6a69
eustlb fix loss
309fe6e2
eustlb fix loss test
99650000
eustlb make
6d041fb1
eustlb use correct checkpoint
cdfb7cca
eustlb Merge branch 'main' into parakeet-rnnt-from-scratch
bfb2eec6
eustlb AutoModelForRNNT in auto.md
d5b552ac
eustlb Merge branch 'parakeet-rnnt-from-scratch' of github.com:huggingface/t…
5d140793
eustlb add reproducable tests + other small fixes
1e39472b
eustlb fix
7632aa11
eustlb Merge branch 'main' into parakeet-rnnt-from-scratch
e98c3dcd
eustlb add revision
4207b26f
eustlb Merge branch 'parakeet-rnnt-from-scratch' of github.com:huggingface/t…
b784e393
eustlb Merge branch 'parakeet-rnnt-from-scratch' into nemotron-asr
2443e68e
eustlb nit
97909015
eustlb add loss
eb412997
eustlb fix
cc5e007f
eustlb nit
f9339d5e
eustlb better formulated forward
8871730d
eustlb add groups to VoxtralRealtimeCausalConv1d
f257e524
eustlb updates
7517d5de
eustlb clean up
6f21c7e8
eustlb add nemotron_asr to mapping for pipeline
49eaf96e
eustlb update doc
806895b3
eustlb eustlb added New model
eustlb eustlb added Audio
eustlb use inheritance on generate
d4daefe0
Base automatically changed from parakeet-rnnt-from-scratch to main 162 days ago
eustlb draft streaming
dfc8f2e5
eustlb Merge remote-tracking branch 'origin/main' into nemotron-asr
53e68c01
github-actions
eustlb nits
12297f93
eustlb fixes
44850165
eustlb update num_lookahead_tokens API
c6e12ad0
eustlb add streaming latency
5d3c299a
eustlb nit
1d230bf8
eustlb improve doc
18fab335
eustlb doc update
a1eda371
eustlb cleaning generation loop
97dffe49
eustlb fix
aead605f
eustlb Merge branch 'main' into nemotron-asr
57e8cea8
eustlb use hub checkpoints
227cbf08
eustlb udpate tests
c7e4df0d
eustlb NemotronAsrConfig update
03ebf543
eustlb Merge branch 'nemotron-asr' of github.com:huggingface/transformers in…
cecd05b6
eustlb update doc
3a063fca
eustlb NemotronAsr -> NemotronAsrStreaming
486494cb
eustlb update doc
bdd2e699
eustlb update license
1fb78a4d
eustlb nit
50c22936
eustlb test update
2ca01f78
eustlb eustlb marked this pull request as ready for review 155 days ago
eustlb Merge branch 'main' into nemotron-asr
08f2d22e
eustlb
eustlb commented on 2026-06-18
eustlb
ArthurZucker
ArthurZucker approved these changes on 2026-06-18
eustlb make
fb2a580f
eustlb simplify a bit modular
4c1ef7f3
eustlb remove cached_property
9a5f8f0b
eustlb nit
7790f13e
github-actions
eustlb improve comment
4abedd9a
eustlb refacto NemotronAsrStreamingEncoderSubsamplingConv2D
ca3c035f
eustlb address comment
083beaa0
eustlb add all_masked_rows as a kwarg
2667dea9
eustlb
eustlb Merge branch 'main' into nemotron-asr
31cd80d3
github-actions
eustlb rename fixtures
5b86bf1c
eustlb check-repo
c2be332d
eustlb Merge branch 'main' into nemotron-asr
7a518c10
eustlb use shorter streaming example to reduce compute
88f031c7
eustlb update test
9695a62e
eustlb Merge branch 'main' into nemotron-asr
986e0006
eustlb
eustlb eustlb enabled auto-merge 154 days ago
eustlb Merge branch 'main' into nemotron-asr
08cd7e65
eustlb fix tests
51c7f20e
eustlb Merge branch 'nemotron-asr' of github.com:huggingface/transformers in…
3539e6b9
github-actions
eustlb eustlb merged ed51a079 into main 154 days ago
eustlb eustlb deleted the nemotron-asr branch 154 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone