[Bugfix] Fix stale SSM state for new Mamba requests scheduled as decode #32118
Changed reorder to prefill and decode logic
e46c1dda
Added two new test cases to test_batch_reordering
56bcd6d0
Merge branch 'main' into fix_ssm_stale_states
9955fa23
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub