text-generation-inference
fix: attempt forward on flash attn2 to check hardware support
#2335

Merged

fix: attempt forward on flash attn2 to check hardware support #2335

drbh merged 8 commits into main from validate-flash-attn2-on-arch

fix: attempt forward on flash attn2 to check hardware support

4b1005c7

fix: warn window_size_left when using flash attn 1

51239251

danieldk commented on 2024-07-31

Narsil commented on 2024-07-31

fix: prefer version check over test op and avoid window_size_left if …

cae28dcb

fix: improve condtional and error message

5b649d67

danieldk commented on 2024-08-02

fix: update sliding window conditional

cf279542

fix: simplify changes and revert model changes

afc0fb5a

fix: avoid changing conditional

ad942a1d

fix: typo tweak

645a6f80

danieldk approved these changes on 2024-08-05

drbh merged 215ed3ad into main 1 year ago

drbh deleted the validate-flash-attn2-on-arch branch 1 year ago

Reviewers

danieldk

Narsil

Assignees

No one assigned

Labels

None yet

Milestone

No milestone