llama.cpp
6952a460 - llama : add cell_max heuristic for more efficient kv_cache

Commit

3 years ago

llama : add cell_max heuristic for more efficient kv_cache

References

#3228 - llama : custom attention mask + parallel decoding + no context swaps

Author

ggerganov

ggerganov

Parents

Loading