llama.cpp
c44bc1ee - llama : keep the KV related layers on the device

Commit

1 year ago

llama : keep the KV related layers on the device

References

#4309 - llama : per-layer KV cache

Author

ggerganov

ggerganov

Parents

Loading