PR #26572 server: (router) add LRU scheduler

server: (router) add LRU scheduler #26572

ngxson merged 5 commits into master from xsn/server_lru_sched

ngxson requested a review 98 days ago

ngxson force-pushed the xsn/server_ref_count branch from c54fd5d8 to 8a35de8a 98 days ago

ngxson force pushed from 0d156430 to 57223881 98 days ago

ngxson changed the title ~~server: add LRU scheduler~~ server: (router) add LRU scheduler 98 days ago

github-actions added server

ggerganov assigned

ggerganov 98 days ago

ggerganov force-pushed the xsn/server_ref_count branch from 8a35de8a to 43f3209c 98 days ago

ggerganov force pushed from 8bc1c035 to 78ce58f6 98 days ago

ggerganov approved these changes on 2026-08-05

ngxson force-pushed the xsn/server_ref_count branch from 43f3209c to d20a8211 97 days ago

ngxson force pushed from 78ce58f6 to 4bf9ec41 97 days ago

ngxson force-pushed the xsn/server_ref_count branch from d20a8211 to 9184647b 95 days ago

ngxson force pushed from 4bf9ec41 to 90749344 95 days ago

ngxson force-pushed the xsn/server_ref_count branch from 9184647b to 1621aade 95 days ago

ngxson force pushed 95 days ago

Base automatically changed from xsn/server_ref_count to master 95 days ago

add lru_sched

02f5052f

handle coalescing (req leaves waiting queue)

3bf32c0a

add tests

07491664

fix stream case

3954a9c3

address review comments

fbb29b3f

ngxson force pushed to fbb29b3f 95 days ago

ggerganov approved these changes on 2026-08-07

allozaur approved these changes on 2026-08-07

ngxson merged dff15d4a into master 95 days ago

Reviewers

allozaur

ggerganov

Assignees

ggerganov

Labels

server

Milestone

No milestone