PR #25603 cuda: Support Q2_0

cuda: Support Q2_0 #25603

dfriehs wants to merge 4 commits into ggml-org:master from dfriehs:q2_0-cuda

cuda: Q2_0

dd5decdb

tests: enable Q2_0 for test-backend-ops perf

b35bf1c7

cuda: adjust Q2_0 to new group size of 64

71bbe0bc

cuda: extract Q2_0 elements via __byte_perm

d19075f4

dfriehs requested a review from

ggerganov 1 day ago

dfriehs requested a review 1 day ago

github-actions added testing

github-actions added ggml

github-actions added CUDA

dfriehs marked this pull request as draft 17 hours ago

Reviewers

ggerganov

Assignees

No one assigned

Labels

testing ggml CUDA

Milestone

No milestone