onnxruntime
d1bd6a98
- fix(attention): correct grouped-query caches
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
57 days ago
fix(attention): correct grouped-query caches This commit addresses feedback by refining KV transposes, ensuring correct shared cache handling, validating 4-D K/V tensors, and ensuring robust regression testing.
References
#31680 - Add support for grouped-query MultiHeadAttention for CPU and CUDA
Author
tianleiwu
Committer
tianleiwu
Parents
555fb892
Loading