vllm
[V1] [Hybrid] Support using float32 for state in Hybrid Models (Mamba2, Mamba1, Minimax)
#22928
Merged
Merged
[V1] [Hybrid] Support using float32 for state in Hybrid Models (Mamba2, Mamba1, Minimax) #22928
hmellor
merged
35 commits into
vllm-project:main
from
tdoublep:fp32_mamba_cache
support fp32 for mamba ssm cache
fa1bc753
rebase
a1418a9f
fix
ceccd1ec
ci: trigger vLLM pipeline
ecea362c
deal with mamba_cache_params None
bee90b27
Merge branch 'main' into fp32_mamba_cache
0835e4df
working changes
a31f5cd4
working changes
3bcff4dc
working for bamba
821a0a38
Fix type
49e7181f
Clean up
69db9b02
gemini-code-assist
commented
on 2025-08-14
minor
73d281a1
Add support for Falcon H1
f7dbfece
Add support for Zamba2
12a413b7
Add support for GraniteMoeHybrid
933d5b1e
Add support for Mamba2
6d0de8d1
Add support for mamba; linear_attention
e458090f
Support for minimax
a3b80a5c
Add support for mamba; linear_attention
94d54408
tdoublep
changed the title
Adding co-author.
5b005ec4
Add fp32 state test
b61b5a61
Use None as default
65f81404
Use consistent type
01ee9af8
Fix typo
bbd27827
refactor dtype
ceb8c98a
Fix typo
fe9ebc5f
Don't pass ssm argument to minimax dtype calc
63f0656e
Simplify dtype logic
4d83dd1d
Add assert for storage_offset_bytes.
8a03a21d
heheda12345
approved these changes
on 2025-08-14
disabled auto-merge 269 days ago
Manually disabled by user
Manually disabled by user
Merge branch 'main' of github.com:vllm-project/vllm into fp32_mamba_c…
aaee9028
str to torch dtype
a73483ac
Merge branch 'main' into fp32_mamba_cache
1b9ee5ee
Fix V1 test
e11b4867
disabled auto-merge 268 days ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
merge in main
b5b47a2f
Merge branch 'main' into fp32_mamba_cache
1416a74e
mgoin
approved these changes
on 2025-08-15
Reviewers
Assignees
No one assigned
Labels
ready
v1
Milestone
Login to write a write a comment.
Login via GitHub