llama.cpp
server: fix LRU hang on multiple requests same model
#28539
Merged

server: fix LRU hang on multiple requests same model #28539

ngxson merged 2 commits into master from xsn/models_fix_hang
ngxson
ngxson server: fix LRU hang on multiple requests same model
f2dc72b8
ngxson ngxson requested a review 41 days ago
github-actions github-actions added server
ServeurpersoCom server: keep a queued model out of the victim pool until its waiters …
eee2fa9c
ServeurpersoCom
ServeurpersoCom approved these changes on 2026-09-07
ServeurpersoCom
ngxson ngxson merged 160bd031 into master 41 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone