text-generation-inference
feat(router): Remove second lock from batcher hot path
#27
Merged

feat(router): Remove second lock from batcher hot path #27

OlivierDehaene merged 2 commits into main from feat/optim_hot_path_router
OlivierDehaene
njhill
njhill commented on 2023-01-20
OlivierDehaene feat(router): Remove second lock from batcher hot path
67ee1907
OlivierDehaene use IntMap
b96fe73b
OlivierDehaene OlivierDehaene force pushed from 665a21ae to b96fe73b 3 years ago
OlivierDehaene OlivierDehaene merged 1539d3cb into main 3 years ago
OlivierDehaene OlivierDehaene deleted the feat/optim_hot_path_router branch 3 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone