text-generation-inference
eecca271 - feat: improve qwen2-vl startup (#2802)

Commit
1 year ago
feat: improve qwen2-vl startup (#2802) * feat: tokenize each request individually and increase warmup image size * feat: adjust rotary embed and avoid cuda graphs of size 2 and smaller * fix: address image resize and rebase changes * feat: update to run qwen2-vl tests * fix: tweak param types
Author
Parents
Loading