xla
Add heuristic default block sizes for different cases in ragged attention kernel
#8922

Merged

Add heuristic default block sizes for different cases in ragged attention kernel #8922

yaochengji merged 8 commits into master from chengji/ragged-attn

Add heuristic default block sizes for different cases in ragged atten…

64ac8b68

yaochengji requested a review from

vanbasten23 1 year ago

bythew3i commented on 2025-04-02

vanbasten23 commented on 2025-04-02

fix

437fa7d9

fix

e4a23760

fix

3e54ecbe

fix

faa2d2a1

fix block size

56147e0f

vanbasten23 approved these changes on 2025-04-02

set v4 of vmem_limit_bytes to 16MB

8286d566

yaochengji force pushed from 64d9bad4 to 8286d566 1 year ago

fix

119a50b6

yaochengji merged 6c3f2313 into master 1 year ago

Reviewers

vanbasten23

bythew3i

Assignees

No one assigned

Labels

None yet

Milestone

No milestone