server : enable packing for d1-2B and d1-Omni-2B models #32553
tdakhran
requested a review
10 hours ago
tdakhran
requested a review
10 hours ago
tdakhran
requested a review
10 hours ago
tdakhran
changed the title model : enable packing for d1-2B and d1-2B models model : enable packing for d1-2B and d1-Omni-2B. models 10 hours ago
tdakhran
changed the title model : enable packing for d1-2B and d1-Omni-2B. models model : enable packing for d1-2B and d1-Omni-2B models 10 hours ago
llama-bench : size the batch to the prompt for models without memory
7d4ecc11
server : read up to 16 lfm3-d1 questions in one pass
09429a16
tdakhran
force pushed
from
3bbbbcd4
to
09429a16
9 hours ago
tdakhran
changed the title model : enable packing for d1-2B and d1-Omni-2B models server : enable packing for d1-2B and d1-Omni-2B models 9 hours ago
ngxson
commented
on 2026-11-18
ngxson
approved these changes
on 2026-11-18
ngxson
merged
bafa0420
into master 6 hours ago
Assignees
No one assigned
Labels
documentation
server
Login to write a write a comment.
Login via GitHub