llama.cpp
52af7826 - cuda : new cublas gemm branch for multi-batch quantized src0

Commit

1 year ago

cuda : new cublas gemm branch for multi-batch quantized src0

References

#3776 - cuda : improve text-generation and batched decoding performance

Author

ggerganov

ggerganov

Parents

Loading