Add Dia model #38405
vasqu
merged
217 commits into
huggingface:main
from
buttercrab:add-dia
add dia model
86d38f16
add tokenizer files
ad7302e0
cleanup some stuff
3aefa5fc
Merge branch 'main' into add-dia
782af0ef
brut copy paste code
c61f8856
Merge branch 'add-dia' of github.com:huggingface/transformers into ad…
810a8df6
rough cleanup of the modeling code
4ac684e7
nuke some stuff
9cc0d6b9
more nuking
d7491abd
more cleanups
a447adfe
updates
5f3b0c3d
add mulitLayerEmbedding vectorization
64273232
nits
79e4f039
more modeling simplifications
df780fd9
updates
f7b2c085
update rope
fdabeb5c
update rope
9861ab52
just fixup
007d4803
update configuration files
14a502ed
more cleanup!
d9e9585d
default config values
677481b9
update
919ef033
forgotten comma
73acbdde
another comma!
87375efc
update, more cleanups
f1dfefdc
just more nits
13117926
more config cleanups
f795327c
time for the encoder
68650efb
fix
10066b6d
sa=mall nit
738b858d
nits
233db050
n
9f2608c4
refacto a bit
87efaf0e
cleanup
525a40dd
update cv scipt
4d358083
fix last issues
3dfe8cd7
fix last nits
0ee2ac64
styling
8ac8a9e8
small fixes
8e63b288
just run 1 generation
43e10821
fixes
ab43e723
nits
83ce1f8d
fix conversion
35a317f8
fix
a12a7667
more fixes
ed0d00c0
full generate
2af24189
ouf!
41990003
fixes!
5f396557
updates
f09666c3
fix
badc5d05
fix cvrt
463f366e
fixup
37b19430
nits
81497ed1
delete wrong test
afb3915c
update
0d60608a
update
06912280
test tokenization
41c4f39b
let's start changing things bit by bit - fix encoder step
465d92f6
removing custom generation, moving to GenerationMixin
10893cb5
add encoder decoder attention masks for generation
30007d95
merge upstream main
6ecf9535
mask changes, correctness checked against ad29837 in dia repo
daaa83de
refactor a bit already --> next cache
f88ea865
too important not to push :)
a72e6e46
minimal cleanup + more todos
7c3c230d
Merge branch 'main' into add-dia
02ce881a
make main overwrite modeling utils
b1534265
add cfg filter & eos filter
6891a655
add eos countdown & delay pattern
93670a3b
update eos countdown
cd3d95c5
add max step eos countdown
47de6881
fix tests
a4fdcab7
merge
601143e0
fix some things
40b9c647
fix generation with testing
00d15a68
move cfg & eos stuff to logits processor
a4a750d2
make RepetitionPenaltyLogitsProcessor flexible
616e70a0
fix input_ids concatenation dimension in GenerationMixin for flexibility
0837c73e
Add DiaHangoverLogitsProcessor and DiaExponentialDecayLengthPenalty c…
20535742
merge main
3d339513
Add stopping criteria
35f564d0
refactor
baa56770
move delay pattern from processor to modeling like musicgen.
fa47036d
fix processor & fix tests
5fe7b82c
refactor types
52628a5c
refactor imports
bd2ac7e1
format code
7e22d243
fix docstring to pass ci
f793f0d5
Merge branch 'main' into add-dia
c1016a1f
add docstring to DiaConfig & add DiaModel to test
efce6629
fix docstring
093ad3b5
add docstring
b6121254
fix some bugs
607fa7ae
Merge branch 'main' into add-dia
4e140973
check
85febf32
porting / merging results from other branch - IMPORTANT: it very like…
34299856
experimental testing of left padding for first channel
23574475
whoops
036c9aec
Fix merge to make generation work
b8e648cf
fix cfg filter
15b43adb
add position ids
fc6e4c59
add todos, break things
68b75d8a
revert changes to generation --> we will force 2d but go 3d on custom…
d8b892e6
refactor a lot, change prepare decoder ids to work with left padding …
19a9dffa
some first fixes to get to 10. in generation
4e7c5508
some more generation fixes / adjustment
7d08de5f
style + rope fixes
e988dda4
move cfg out, simplify a few things, more todos
7f9a0c8e
nit
1a274d77
start working on custom logit processors
d0ab6932
nit
5804eedd
quick fixes
8d0979ee
cfg top k
5dc7b3f8
more refactor of logits processing, needs a decision if gen config ge…
686e754a
lets keep changes to core code minimal, only eos scaling is questiona…
52d983a7
simpler eos delay logits processor
2e7c6c70
that was for debugging :D
e636bde0
proof of concept rope
d46de542
small fix on device mismatch
2db8338d
cfg fixes + delay logits max len
a9d95209
transformers rope
371b9534
modular dia
5cfac34d
more cleanup
93829266
keep modeling consistently 3D, generate handles 2D internally
07f2720c
decoder starts with bos if nothing
3f8c1be9
post processing prototype
a0f61edf
style
59e2bccd
lol
282de2d1
force sample / greedy + fixes on padding
0f9171f3
style
db69ca32
fixup tokenization
4d3805a8
nits
0b26d2b5
revert
5e104a43
start working on dia tests
210faf56
fix a lot of tests
291fe544
more test fixes
ef9ab752
nit
da1ad0d7
Merge branch 'main' into add-dia
cedd8a5b
more test fixes + some features to simplify code more
8d93cd3f
more cleanup
d75899cc
forgot that one
a31ae4bc
autodocs
456a9cdd
small consistency fixes
1a1953e7
fix regression
0a41da29
small fixes
a1bf0f78
dia feature extraction
9eb506ab
docs
6d33db80
wip processor
af9b65fb
fix processor order
2cf7eda2
processing goes brrr
12bf32f5
transpose before
f8354bd5
small fix
57e687ba
fix major bug but needs now a closer look into the custom processors …
402f60f1
small thing on logits
737dcc9d
nits
7e81f453
simplify indices and shifts
9c692571
add simpler version of padding tests back (temporarily)
9937e329
add logit processor tests
d3e522e5
starting tests on processor
fc96004e
fix mask application during generation
0d5a6836
some fixes on the weights conversion
15ab87f6
style + fixup logits order
889e323f
simplify conversion
2a8d0d57
nit
0ccfea8e
remove padding tests
368d6b36
nits on modeling
53409795
hmm
e5a5dc33
fix tests
584cde26
trigger
f53f43cf
probably gonna be reverted, just a quick design around audio tokenizer
5beae320
fixup typing
1c90b378
Merge branch 'main' into add-dia
adc6e6c7
post merge + more typing
2acbe81c
initial design for audio tokenizer
c35bc88d
more design changes
31dea2a6
nit
3572cad5
more processor tests and style related things
1003a52e
add to init
6a61da20
protect import
c95defd2
not sure why tbh
b647afad
add another protect
ce98e6d0
more fixes
980fa377
wow
ac080630
it aint stopping :D
b291f253
another missed type issue
1e198b14
...
135ed158
change design around audio tokenizer to prioritize init and go for au…
caac9e75
change to new causal mask function + docstrings
06a30c83
Merge branch 'main' into add-dia
623547a1
change ternary
16add9bb
docs
828c55ad
remove todo, i dont think its essential tbh
0f114083
remove pipeline as current pipelines do not fit in the current scheme…
1895c418
closer to wrapping up the processor
7598dbde
text to audio, just for demo purposes (will likely be reverted)
148f1fd1
vasqu
commented
on 2025-06-20
check if it's this
aa684ddc
save audio function
9ec42248
ensure no grad
aab87b05
fixes on prefixed audio, hop length is used via preprocess dac, devic…
32237e3a
integration tests (tested locally on a100) + some processor utils / f…
27057101
style
7ca3d9b0
nits
005224bb
another round of smaller things
756b4087
Merge branch 'main' into add-dia
6afb932a
docs + some fixes (generate one might be big)
a144382c
msytery solved
34278249
small fix on conversion
8e9daf6f
add abstract audio tokenizer, change init check to abstract class
d55130be
nits
d2597ae9
update docs + fix some processing :D
75ed1aca
change inheritance scheme for audio tokenizer
4d87181a
delete dead / unnecessary code in copied generate loop
96ca4e23
last nits on new pipeline behavior (+ todo on tests) + style
5828d1b1
Merge branch 'main' into add-dia
279944d2
trigger
798330e8
Merge branch 'main' into add-dia
9d1ea004
fixup loss
fa59c1ed
Login to write a write a comment.
Login via GitHub