server : vision support via libmtmd #12898
ngxson
merged
79 commits into
ggml-org:master
from
ngxson:xsn/server_mtmd
server : (experimental) vision support via libmtmd
466c6cdd
mtmd : add more api around mtmd_image_tokens
2317e618
mtmd : add more api around mtmd_image_tokens
a46b6db6
mtmd : ability to calc image hash
7ac0b7b7
shared_ptr for mtmd_image_tokens
58c47674
move hash to user-define ID (fixed)
d3c3e20c
Merge branch 'xsn/mtmd_image_api' into xsn/server_mtmd
a44029a1
abstract out the batch management
5e6c7ba4
Merge branch 'master' into xsn/server_mtmd
78a76de3
qnixsynapse
commented
on 2025-04-15
Merge branch 'master' into xsn/server_mtmd
c734b539
small fix
a6a36537
ngxson
commented
on 2025-04-21
refactor logic adding tokens to batch
f8bc4662
implement hashing image
f5420e1d
ggerganov
commented
on 2025-04-22
Merge branch 'master' into xsn/server_mtmd
aae2e698
use FNV hash, now hash bitmap instead of file data
cd115854
allow decoding image embedding to be split into batches
8afa9528
rm whitespace
989730c6
Merge branch 'master' into xsn/server_mtmd
19b9fe1b
disable some features when mtmd is on
2df8c1a4
fix --no-mmproj-offload
b9ef895f
mtmd_context_params no timings
add9e215
Merge branch 'master' into xsn/server_mtmd
0f397701
refactor server_inp to server_tokens
58100b39
ngxson
commented
on 2025-04-25
fix the failing test case
e82fea8f
init
4a4f35c8
wip
f6b6517c
Merge branch 'master' into xsn/mtmd_c_api
e0806c29
working version
82f4246e
add mtmd::bitmaps
f8c27b9e
add test target
33579615
rm redundant define
92d24045
test: mtmd_input_chunks_free
111d5afa
rm outdated comment
08d0f9cf
ggerganov
commented
on 2025-05-02
Merge branch 'master' into xsn/mtmd_c_api
a230804b
fix merging issue
863db315
explicitly create mtmd::input_chunks
a0fb7016
mtmd_input_chunk_copy
6bc7a30a
add clone()
4d842eb9
Merge branch 'master' into xsn/server_mtmd
f91fb97e
improve server_input struct
2cedd180
clip : fix confused naming ffn_up and ffn_down
3ee071c8
rm ffn_i/o/g naming
3fbf0bd0
rename n_embd, n_ff
f3870a63
small fix
ae832291
Merge branch 'master' into xsn/clip_ffn_up_down_fix
0009f767
no check n_ff
246a4e0a
Merge branch 'xsn/clip_ffn_up_down_fix' into xsn/server_mtmd
57b288fd
fix detokenize
5f1fe1b3
ngxson
commented
on 2025-05-03
Merge branch 'master' into xsn/mtmd_c_api
06cb5952
add const to various places
e9f7ff92
add warning about breaking changes
049ae242
Merge branch 'xsn/mtmd_c_api' into xsn/server_mtmd
91613c05
add c api
d3fece59
helper: use mtmd_image_tokens_get_n_pos
076e3b93
Merge branch 'xsn/mtmd_c_api' into xsn/server_mtmd
574d403c
Merge branch 'master' into xsn/server_mtmd
036f6827
fix ctx_shift
01c623e6
fix name shadowing
a0f2562e
Merge branch 'master' into xsn/server_mtmd
9149f390
Merge branch 'master' into xsn/server_mtmd
b353038e
more strict condition
3304b44e
support remote image_url
88461f2c
Merge branch 'master' into xsn/server_mtmd
4adce866
remote image_url log
a9b21f42
ngxson
changed the title
add CI test
2f30530d
ngxson
commented
on 2025-05-06
do not log base64
5ffde386
ggerganov
commented
on 2025-05-08
add "has_multimodal" to /props
aaebc336
remove dangling image
eeda075e
speculative: use slot.cache_tokens.insert
bef122e1
Merge branch 'master' into xsn/server_mtmd
72824563
ggerganov
commented
on 2025-05-09
ggerganov
commented
on 2025-05-09
ngxson
changed the title
Apply suggestions from code review
51afc0a2
rm can_be_detokenized
f10fc561
on prmpt processing done, assert cache_tokens.size
689035cc
handle_completions_impl returns void
b2906a98
Merge branch 'master' into xsn/server_mtmd
abfd821d
adapt the new web ui
f5fbc03e
update docs and hot topics
5fe8d72c
ngxson
commented
on 2025-05-09
rm assert
b8000fd5
small fix (2)
9ed430ca
Reviewers
Assignees
No one assigned
Labels
documentation
testing
examples
python
server
Milestone
No milestone
Login to write a write a comment.
Login via GitHub