llama.cpp
ad7ef7af
- Skip computation of unused logits during batch prompt eval (drop other batch positions after writing their kv to cache)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
2 years ago
Skip computation of unused logits during batch prompt eval (drop other batch positions after writing their kv to cache)
Author
ochafik
Committer
ochafik
Parents
604b8bdf
Loading