llama.cpp
a5355a02 - server: keep router model refcount to avoid unloading models that have running requests

Commit
155 days ago
server: keep router model refcount to avoid unloading models that have running requests this avoids a deadlock when models A and B don't fit together, but both have requests, so the server gets into a loop unloading A, loading B, unloading B, loading A again, and so on
Author
Committer
Parents
Loading