metal : try to improve batched decoding - SemanticDiff

Commit

1 year ago

metal : try to improve batched decoding

References

gg/pad-kv-cache

#4280 - llama : pad KV cache size

Author

ggerganov

ggerganov

Committer

ggerganov

ggerganov

Parents

Loading