vllm
[Model] Reduce redundant computations in mamba2 blocks for Bamba-9B
#15423
Merged
Merged
[Model] Reduce redundant computations in mamba2 blocks for Bamba-9B #15423
tlrmchlsmth
merged
13 commits into
vllm-project:main
from
cyang49:pr_mamba_opt_reuse_across_layers
fabianlim
commented
on 2025-03-25
fabianlim
commented
on 2025-03-25
fabianlim
commented
on 2025-03-25
tlrmchlsmth
commented
on 2025-03-25
yury-tokpanov
commented
on 2025-03-25
tlrmchlsmth
commented
on 2025-03-31
Reduce cpu-gpu synchronization in mamba2 chunked prefill
2be3f161
Reduce redundancy in mamba2 blocks
08df5615
refactored mixer and bamba
a7b363cd
lint
e06a170e
disable assertions that causes d2h copies in mamba2
678e6910
simplify seq_idx computation
21ebc660
Revert "simplify seq_idx computation" as repeat_interleave
569185e8
remove assertions that cause synchronizations
01d5a332
Pack mamba2 metadata
017597e9
pack more into mamba2 metadata
c0ead4d3
Patching mamba2 and zamba2
4382192e
Add back torch.any on has_inital_states to recover original semantics
080eee92
moving more redundant computation into metadata prep
7236a55b
tlrmchlsmth
approved these changes
on 2025-04-10
Reviewers
Assignees
No one assigned
Labels
ready
Milestone
No milestone
Login to write a write a comment.
Login via GitHub