PR #8659 Add ragged paged attention

Add ragged paged attention #8659

vanbasten23 merged 9 commits into master from xiowei/add_ragged_paged_attention

bythew3i commented on 2025-01-31

vanbasten23 force pushed from ad2f87c2 to 9e4b227a 1 year ago

bythew3i approved these changes on 2025-02-01

add the first version. All tests pass except for test_paged_attention…

b129e9a9

all tests passed.

6c3bf730

all tests passed except for one test which oom'ed.

dbc3fee2

Improved the tests and all tests passed except for the OOM one. Also …

e589c860

clean up

9b3cdab6

linter

ab69feb7

address pr comments

cd65cc4b

fix the rest of comments

7fe50717

vanbasten23 force pushed from 9e4b227a to 7fe50717 1 year ago

Trigger CI

51704029

vanbasten23 requested a review from

lsy323 1 year ago

vanbasten23 requested a review from

miladm 1 year ago

miladm approved these changes on 2025-02-03

miladm assigned

vanbasten23 1 year ago

miladm added pallas

miladm commented on 2025-02-03

vanbasten23 merged 84800944 into master 1 year ago

Reviewers

miladm

bythew3i

lsy323

Assignees

vanbasten23

Labels

pallas

Milestone

No milestone