Add support for Qwen2VL #10361
ggerganov
merged
35 commits into
ggml-org:master
from
HimariO:qwen2-vl
Barebone Qwen2VL LLM convertor
c17546ff
Add Qwen2VL cli entrypoint
7c6f7934
[WIP] add qwen2vl arch
b24bd89e
Verify m-rope output
35411963
Add vl-rope/2d-rope support for qwen2vl ViT
9d389a05
update qwen2vl cli tool
f661483e
update 5D tensor op workaround
3c3691e1
[WIP] qwen2vl vision model
c13edfed
make batch and clip utils compatible with qwen2vl
7e9fc720
[WIP] create inference workflow, gguf convert script but fix
bcd49f59
correcting vision-rope behavior, add the missing last layer back to ViT
023f0076
add arg parser to qwen2vl_surgery
3d19dd44
replace variable size array with vector
53480d2b
cuda-gdb cmake preset
0882f576
add fp32 mrope, vision rope kernel
3237bb46
add fp16 support for qwen2vl and m-rope
201f7043
add `GGML_ROPE_TYPE_MROPE`, `GGML_ROPE_TYPE_VISION`
f1fa60f8
fix rope op mode switching, out dated func args
241bb457
update `llama_hparams`
07553cfb
update to keep up stream changes
fac03453
resolve linter, test errors
cbd08b42
add makefile entry, update speical image padding token
6c39aa38
add mrope unit test, fix few compiler warnings
ac2089c3
rename `mrope` related function, params
12f17f75
minor updates on debug util, bug fixs
3ba7664d
add `m-rope` testcase to `test-backend-ops`
b24ab863
Merge branch 'master' into qwen2-vl
d7edc550
Apply suggestions from code review
9abb2527
Merge branch 'ggerganov:master' into qwen2-vl
c292bf1d
fix traililng whitespce
e9748e4a
store `llama_hparams.rope_sections` with fixed size array
ef7f74bd
update position id tensor size check in GGML_OP_ROPE
e2e9a6ca
ggerganov
commented
on 2024-12-13
minor updates
a02a1907
ggerganov
commented
on 2024-12-13
ggerganov
commented
on 2024-12-13
update `ggml_backend_*_supports_op` of unsupported backends
19aba1db
remote old `rope_section` compare operator
f96909e2
Login to write a write a comment.
Login via GitHub