llama.cpp
466b5138 - parallel : disable hot-plug to avoid cache fragmentation

Commit

2 years ago

parallel : disable hot-plug to avoid cache fragmentation

References

#3228 - llama : custom attention mask + parallel decoding + no context swaps

Author

ggerganov

ggerganov

Parents

Loading