llama.cpp
bddfd2b1 - server: refactor batch construction (#24843)

Commit
61 days ago
server: refactor batch construction (#24843) * server: refactor batch construction * wip * wip 2 * wip 3 * wip 4 * add abort_all_slots * handle batch full more carefully * fix assert * rm debug log * small nits * (debug) add timings * debug: force llama_synchronize for accurate timings * address comments * disable DEBUG_TIMINGS
Author
Parents
Loading