llama.cpp
0161372b - parallel : example for serving multiple users in parallel

Commit

2 years ago

parallel : example for serving multiple users in parallel

References

#3228 - llama : custom attention mask + parallel decoding + no context swaps

Author

ggerganov

ggerganov

Committer

ggerganov

ggerganov

Parents

Loading