llama.cpp
113dd600 - server : bach has to be allocated for n_parallel sequences

Commit

2 years ago

server : bach has to be allocated for n_parallel sequences

References

#3677 - server : parallel decoding and multimodal (cont)

Author

ggerganov

ggerganov

Parents

Loading