Add CSM model #36719
eustlb
merged
163 commits into
huggingface:main
from
eustlb:add-csm
draft structure
48d26477
depth decoder with forward pre hook
61db81f6
full model forward draft
7bfe4b78
draft update
b81db076
depth decoder update
a305fa30
ConversationalSpeechModelForCausalLM udpates
36f76329
add generate
6c4d2d56
max length criteria small fix
3f3625c9
udpate
67b6cc47
updates
89b2e95c
generation update
f9c19f99
update in loss compute
1fa74651
conversion script
2aabe94c
update for correct input embeddings
f0a72120
handle interleaved rope
eb712ded
update
bb39449f
update
34dc04df
update
602f6d79
support compile
63c160d3
update training
936282bb
add doc
1e92e69a
update doc
abee0f7f
correct inits
4269b061
Merge branch 'main' into add-csm
a56f9afd
SeungyounShin
commented
on 2025-04-06
ConversationalSpeechModel -> Csm
0774d720
conf update
c0bc80aa
name update
e32d1eb2
tests CsmForCausalLMTest
05fa1c42
convert use cached_file
3dc02241
conf + modeling updates
cc19ee96
generate utils handle third dim shape
541128d6
integration test
02f61000
modeling + conf updates
24094a2c
common test handle more than 2 dims
35757b35
add nested audio list utils
3b7bd2f5
processing handle nested audio list
8473820c
csm processing draft
26024906
mimi util
b5f83988
init updates
2a94bbca
modular update
99ecb455
convert modular
f2cfc398
processing update
52c0d643
csm tests update
c1eed72f
generate tests handle third dim
ab83cd0d
generate utils handle third dim
ac2373f6
propagate _get_initial_cache_position update
8dcafaa8
tied_weight_keys update + convert correctly
3d99d28a
fix inputs_embeds
a4fb4014
revert audio nested list
198f4a54
batch inference update + return audio
f3a21a5f
audio_utils update
cb072790
processor update
ede5e9c0
some more integration tests
09b0c6dd
remove old test
23a7d38e
porcessing output labels
481c2562
improve
0fc9078a
fix
39a23ba0
update rope values with equivalent ones
b637cca2
conversion update
8d860e5a
udpate tests
fcb64e60
handle depth decoder generation config
03ca2f23
remove default eos_token_id
d6b8ba65
make style
7477ea02
revert modeling_mimi
a4b628ee
add default generation_config
32f34fc4
remove sdpa since handled by default
d87b30b4
make
dc486bc4
fix conflict
a7724a83
fix conflicts
856a3e70
Merge branch 'main' into add-csm
a2423817
correct naming
380bc4c0
correct imports
c9cef41d
make
c6287cb6
causal -> conditional naming
45176029
causal -> conditional naming
d9d1cc87
Merge branch 'main' into add-csm
91d4871a
auto update
bca3d403
make
312f57c5
make
d1dba6f5
Merge branch 'main' into add-csm
2151964b
add doc
cbda8891
Merge branch 'main' into add-csm
44b4f442
test update
6d9cca31
fix weight init
9ec03e43
audio tokens offsets as buffer
8f7f0203
4d mask in conditional class
0ce70595
make
35d9901f
Merge branch 'main' into add-csm
6961e39d
Merge branch 'main' into add-csm
c5d9a7d5
Merge branch 'main' into add-csm
8130c4e5
doc update
a68cd027
fix causal mask
b4831ebc
fix causal mask
616c12d7
doc update
a0d2d7ed
doc update
34f2373b
Merge branch 'main' into add-csm
7919e9f9
add processor doc
bf2aa574
update doc
eb8a48d7
fix 4d causal mask
c046ca58
update make_list_of_audio
0e664849
Merge branch 'main' into add-csm
0e2b49bb
copilot-pull-request-reviewer
commented
on 2025-04-29
do not default to mutable
bf5adc4a
remove duplicates
1e1eccb5
Merge branch 'main' into add-csm
56506c0b
remove useless reset_parameters
9d329b64
use GradientCheckpointingLayer
f5dfa773
use can_return_tuple
57fcdf22
formatting
92e26a58
prepend placeholder in _sample
72df6a69
torch compile fix
93018727
some more fixies
ca19f88f
convert modular
45faaa12
Merge branch 'main' into add-csm
2ead9ecf
fix
e5fbb9ff
default max_length in convert
0ddb8351
handle depth decoder generation config correctly
d76d9ab3
clearer formulation
adc10c46
handle output_loading_info
0089658d
handle softmax warning
9536cff0
Merge branch 'main' into add-csm
f2467461
Merge branch 'main' into add-csm
2478ef78
add doc
27a6a187
propagate _get_initial_cache_position changes
8d10f9de
generation in its own module
fe7f971c
add processor tests
3e9dc637
fix compile witu cuda graphs
d017f121
fix compile with cuda graphs
65447996
add csm.md
523eabaf
include CSM loss
d324b0a5
Merge branch 'main' into add-csm
c275a8b8
doc nit
73971f45
Merge branch 'add-csm' of github.com:eustlb/transformers into add-csm
90a9ad73
doc nit
498edee0
doc nit
76f5b58c
Update docs/source/en/model_doc/csm.md
0afbe137
add save_audio to processor
c528bd67
Merge branch 'add-csm' of github.com:eustlb/transformers into add-csm
7e7d0ef2
Update src/transformers/models/csm/modular_csm.py
7148da10
doc update
c5f4fcba
simplify audio_codes_mask computation
cabba017
doc update
e2e86020
Merge branch 'main' into add-csm
77d9e110
simplify loss computation
297a6263
fix static cache test
45fc46ea
fix
3204df4b
remove comment
85498292
Merge branch 'main' into add-csm
b518012a
Merge branch 'main' into add-csm
4a514d19
simplify encoded length computation
aa00fa63
use hf-internal-testing
050907df
Merge branch 'main' into add-csm
d5a86f6c
Merge branch 'main' into add-csm
1a71dfec
doc update
abef3489
Merge branch 'add-csm' of github.com:eustlb/transformers into add-csm
99995912
cast to float before numpy
a1fc7174
nit
42d88a72
mem efficient codebook head
7bf37b0f
Merge branch 'main' into add-csm
7b6a275b
nit
08728439
Merge branch 'add-csm' of github.com:eustlb/transformers into add-csm
d6272749
Merge branch 'main' into add-csm
0ae894fc
cat input values with cutoffs
b4bca5c0
Merge branch 'main' into add-csm
3d72be35
Assignees
No one assigned
Labels
New model
Audio
Milestone
No milestone
Login to write a write a comment.
Login via GitHub