llama.cpp
Implement multimodal models (LLaVA)
#3436
Merged

Implement multimodal models (LLaVA) #3436

monatis merged 36 commits into master from llava
monatis
monatis WIP: start implementing LLaVA
59aa1acf
staviq
monatis
staviq
monatis rm scratch buf for now, will revert after cleanup
0f0e7c64
monatis LLaVA image encoder is working. will combine with llama
7e9120f7
monatis Add llava inference code, but it's buggy. debugging
d37ed475
ggerganov ggerganov added model
monatis
monatis LLaVA is working e2e, needs to optimize memory allocation + cleanup
8690f425
monatis
monatis
monatis Use ggml_allocr + rm unnecessary code
94eeac35
monatis fix: crlf -> lf
0c2bd797
monatis fix: new line at EoF
204d08be
monatis fix: trailing whitespace
95da79e7
monatis Merge branch 'master' into llava
2a04d0b5
monatis Add readme
444dbce8
monatis
monatis monatis requested a review from ggerganov ggerganov 1 year ago
monatis monatis marked this pull request as ready for review 1 year ago
monatis Update readme
8af7e210
monatis Some cleanup
54495c94
monatis Are you happy editorconfig?
9b0ec4d2
monatis rm unused batch image preprocessing
8278a736
monatis rm unused import
d78e8163
monatis fix: rm designated initializers
4759bfd6
ggerganov
ggerganov ggerganov added high priority
ggerganov ggerganov added need feedback
Green-Sky
Green-Sky
monatis
ggerganov
monatis
Green-Sky
monatis
staviq
Green-Sky
monatis
Green-Sky
monatis introduce pad-to-square mode for non-square images
325d2400
monatis are you happy editorconfig?
d75a0315
monatis
monatis gitignore /llava
ae01c859
Green-Sky
Green-Sky
monatis Handle cases where image file does not exist
5009ae90
monatis add llava target to Makefile
96171de5
monatis
monatis
Galunid
monatis
monatis add support for 13b model variant
d640aae7
monatis
Green-Sky
monatis
ggerganov
monatis
Green-Sky
monatis
ggerganov
monatis
ggerganov
Green-Sky
monatis
ggerganov
monatis
aiaicode
BarfingLemurs
monatis Maybe seed is unlucky?
587bde8e
monatis
monatis
monatis Merge branch 'master' into llava
f1564bb2
Galunid
monatis
Galunid
monatis Check if apples are compared to apples
ab215879
monatis
monatis are you happy editorconfig?
0409ae00
BarfingLemurs
monatis
Green-Sky
Green-Sky
Green-Sky commented on 2023-10-11
monatis Use temperature = 0.1 by default
f0f78345
monatis
gcardoso2314
cebtenzzre
monatis command line: use gpt_params_parse()
2bc1710e
rlancemartin
Green-Sky
monatis Merge master and fix conflicts
1403d87c
monatis
ggerganov
rlancemartin
ilteris
cednats
monatis
monatis
cednats
haotian-liu
monatis
ggerganov
monatis minor
dc913ea3
monatis handle default n_predict
56ccf97b
doomed151
doomed151 commented on 2023-10-12
monatis fix typo
e9534ea6
ggerganov
ggerganov Merge branch 'master' into llava
346e3c16
ggerganov llava : code formatting, rename files, fix compile warnings
4bc5c9c5
ggerganov
ggerganov approved these changes on 2023-10-12
ggerganov
ilteris
monatis do not use Wno-cast-qual for MSVC
0bd7e69d
monatis monatis merged 370359e5 into master 1 year ago
aiaicode
monatis
haotian-liu
cebtenzzre
ggerganov
monatis
bkbasavaraju
monatis
bkbasavaraju
QueryType
monatis
QueryType
monatis
QueryType
aisensiy
LumenYoung
monatis
LumenYoung
monatis
LumenYoung
TikaToka
monatis
pudepiedj
monatis
TikaToka
pudepiedj
Lurrobert
pudepiedj
monatis
pudepiedj
Lurrobert
monatis
monatis monatis deleted the llava branch 1 year ago
kiiwee
Lurrobert
kiiwee
ASmallPotato
djasil
monatis
ASmallPotato
RachelShalom
Galunid
pudepiedj
RachelShalom
aiaicode
TikaToka
ExtReMLapin
Green-Sky
ExtReMLapin
gamester2665
ExtReMLapin
chigkim
cjpais
cmp-nct
cjpais
gamester2665
BBC-Esq
gamester2665
BBC-Esq
ExtReMLapin
aymenabid-lab
cebtenzzre
phymbert phymbert added llava

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone