llama.cpp
ded9b43c - parallel : fix cases where the input prompts can overflow the batch

Commit

1 year ago

parallel : fix cases where the input prompts can overflow the batch

References

#3228 - llama : custom attention mask + parallel decoding + no context swaps

Author

ggerganov

ggerganov

Parents

Files1

examples/parallel
- parallel.cpp