llama.cpp
a2075615 - examples : add example for batched decoding

Commit

2 years ago

examples : add example for batched decoding

References

#3228 - llama : custom attention mask + parallel decoding + no context swaps

Author

ggerganov

ggerganov

Parents

Loading