llama.cpp
5b685032 - use tensor split for deciding cache size

Commit

34 days ago

use tensor split for deciding cache size

References

aman/moe-cache-multigpu

#30112 - llama: support MoE cache over multiple GPUs

Author

am17an

am17an

Parents

Loading