Improve vlm support (add idefics3 support) #2437
drbh
merged
17 commits into
main
from
improve-vlm-support
feat: expand vlm support and add image token logic and tests
cf29c5b5
fix: avoid unused perceiver config
305db7ea
feat: integrate image tokens into inputs embeds
a59b7faf
feat: add simple idefics3 test
ebef284b
feat: update docs, image token logic and weight names
dbe1666b
fix: improve image processing
c9573ddf
feat: improve prefix for idefics3
53b2bea6
fix: bump idefics3 tests and snapshots
34174af8
fix: improve text model loading
064e040e
feat: consolidate changes with existing vlms and add support and test…
0d1bf9e9
Narsil
commented
on 2024-12-20
fix: create new idefic3 file, simplify logic and adjust llama weight …
575d9733
fix: lint with ruff
4c8f5cdc
fix: clean up idefics 3 and improve prefix handling
765ca780
fix: improve typing
df504e9f
fix: improve prompt_split_image with ref to original impl
d397748c
fix: adjust ruff lints and small refactors
78004db1
fix: adjust FlashLlamaModel prefix logic
daa397c5
danieldk
approved these changes
on 2025-01-09
Login to write a write a comment.
Login via GitHub