PR #2575 Improve support for GPUs with capability < 8

Improve support for GPUs with capability < 8 #2575

danieldk merged 4 commits into main from bugfix/pre-cc-8

danieldk force pushed from 491c0269 to 5bd79b8a 1 year ago

Improve support for GPUs with capability < 8

bee5ee1f

nix: add flash-attn-v1 to the server environment

8c0f9312

danieldk force pushed from 5bd79b8a to 8c0f9312 1 year ago

danieldk marked this pull request as ready for review 1 year ago

Narsil commented on 2024-09-27

Move disabling prefix caching into the block of exceptions

a29636ee

Capability as `usize`s

3eb68a37

danieldk requested a review from

Narsil 1 year ago

Narsil approved these changes on 2024-09-27

danieldk merged 5b6b74e2 into main 1 year ago

danieldk deleted the bugfix/pre-cc-8 branch 1 year ago

Reviewers

Narsil

Assignees

No one assigned

Labels

None yet

Milestone

No milestone