onnxruntime
Add flash attention v2 and INT4 CUDA for LLaMA E2E benchmarking
#20149
Merged
Overview
Commits
5
Changes
View On GitHub
Commits
  • Enable flash attention v2 for PyTorch models when benchmarking
    kunal-vaishnavi committed 2 years ago
  • Add instructions for installing flash attention v2
    kunal-vaishnavi committed 2 years ago
  • Add INT4 CUDA benchmarking for PyTorch eager
    kunal-vaishnavi committed 2 years ago
  • Add instructions for installing PyTorch quantization
    kunal-vaishnavi committed 2 years ago
  • Use flash attention v2 for CUDA and SDPA for CPU
    kunal-vaishnavi committed 2 years ago
FAQ Terms Privacy Refunds Impressum
Loading