PR #2943 Improve qwen vl impl

Improve qwen vl impl #2943

drbh merged 18 commits into main from improve-qwen-vl-impl

danieldk commented on 2025-01-23

drbh marked this pull request as ready for review 1 year ago

Narsil commented on 2025-01-23

feat: refactor model, improve startup and re enable tests

77ef5430

fix: improve multimodal rotary embed caching

d12e0759

fix: limit vision flop calc to qwen2 vl models and update config typing

a0ab962b

fix: include clippy lint

cf5c6604

feat: refactor position ids in warmup and bump tests

7ab99bc6

fix: prefer default dtype

eef3c7bd

fix: enable all cuda graphs and bump snapshots

5f416f6e

fix: adjust rotaty init path

6893eb38

fix: simplify get position ids and remove usused vision config

68e3ee8e

fix: update position ids so first dim is batch, simplify rotary and b…

c75c01e9

fix: improve position id init during cuda warmup for mrope and simplf…

79a2c956

fix: check existance before accessing rope type in cuda warmup

d0e2332d

fix: check key before access

585e270a

fix: improve mrope check in cuda graph warmup

cb7ec9cb

fix: remove check for default rope type

79550f8b

drbh force pushed from dca2f125 to 79550f8b 362 days ago

fix: add more test and improve model generation

9eaa1632

danieldk commented on 2025-02-03

fix: improve and simplify get_cos_sin, refactors and cleanup get_pos…

6cb0cb68

fix: adjust signatures with types

58f5f2ee

danieldk approved these changes on 2025-02-04

Narsil commented on 2025-02-04

Narsil approved these changes on 2025-02-04

drbh merged c1cf36c0 into main 358 days ago

drbh deleted the improve-qwen-vl-impl branch 358 days ago

Reviewers

danieldk

Narsil

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

text-generation-inference Improve qwen vl impl #2943 Merged

Improve qwen vl impl #2943

text-generation-inference
Improve qwen vl impl
#2943

Merged