llama.cpp
PoC server with fully functional router, model load/unload (multiple models in parallel)
#37
Open

Commits
  • fully working model maganement
    ngxson committed 1 year ago
  • improve maybe_load_it_why_not
    ngxson committed 1 year ago
  • thread-safe
    ngxson committed 1 year ago
Loading