mtmd: Add Vision Support for Minimax-M3 #25113
ngxson
merged
73 commits into
ggml-org:master
from
timkhronos:MSA-Vision
Add preliminary MiniMax-M3 support
87cc29bc
MiniMax-M3 vision tower (mmproj + clip graph)
53c81ddb
Delete m3_vision_ref.py
f07f1d43
Update clip.cpp
4a3206fd
MSA
bbf1a808
Update constants.py
09657dd7
Update minimax.py
8fe2e015
Cache creation. Working withotu flash attention
8c953a9c
Added flash attention for sparse layers
ea6fbd6f
Decomposed slow cpu OP into GPU + CPU ops. Massive speedup over long ctx
2e827590
Rewrote indexer op to be cuda native. Modified flash attention to mat…
0152226b
Implement sparse attention calc out of stock ops.
d1a04f7c
Fix a cache allocation and cont issue
b1b174ee
Fixed -fa auto crash, flagged debug spots
cea714a8
Delete vocab.json
69c958d9
Delete model.safetensors.index.json
afc09f1f
Delete generation_config.json
f40d0f58
Delete Minimax directory
714bbe93
Handled multi stream case to fall back on Dense Attention
8136a9c6
Development scaffolding cleanup. No functional change to the decode or
3ed9b18b
Remove redundant comment from minimax-m3.cpp
79a6eecf
Changed 3 Gelu Ops for vision into Gelu_erf ops
35990be6
Assert that n_kv is multiple of 128
fa158507
Rename MSA index tensors to indexer convention
eae55e2c
Fix incorrect Assert
2bb7eeb3
Review driven changes (#3)
d6f9426d
Remove comment from conversion minimax.py
0a7b2dc5
Remove whitespaces from constants.py
636143a0
Tighten comment in minimax.py
7b7ff65e
inherit MiniMax-M3 from MiniMax-M2
1cd03ef8
drop dead text_config fallbacks
c70c8a93
Add indexer writer methods
25199fa2
Reuse LLM_FFN_SWIGLU_OAI_MOE
d54b4ebd
Remove duplicate indexer setters, add only block_size/local_blocks, …
f57efcee
Fix conversion error /gguf_writer.py
618e1454
Update gguf-py/gguf/gguf_writer.py
5a24782d
Update gguf-py/gguf/tensor_mapping.py
5dfe8387
Update conversion/minimax.py
b99a8d5e
Update conversion/minimax.py
56ba5416
Remove whitespace in src/llama-kv-cache.cpp
044391ac
Remove Whitespace in Update src/llama-model.h
c314fafe
Remove whitespace in src/llama-hparams.h
5015a6bb
Update minimax_m3.cpp
768346ae
ngxson
commented
on 2026-06-29
ngxson
commented
on 2026-06-29
ngxson
commented
on 2026-06-29
Rename minimax_m3.cpp to minimax-m3.cpp
e819f324
Update CMakeLists.txt
492e357b
Remove debug code from clip.cpp
0d64726f
Update clip.cpp
98e15718
Update comments in tools/mtmd/models/minimax-m3.cpp
9693dc28
Merge branch 'master' into MSA-Vision
06833d28
Permute Q/K at conversion, drop precomputed sin/cos
296f98b8
Merge branch 'ggml-org:master' into MSA-Vision
11395a9e
Log cache size on launch, block ctx shift, support prompt caching
07e607a3
Merge branch 'ggml-org:master' into MSA-Vision
fce0d07b
Update minimax-m3.cpp
96b30add
Optimize implementation, add multi stream support.
e56d2f02
set default cache type to F32
ad35386f
Fix potential DSA double indexer cache allocation bug, only allocate…
5cea3a3b
remove F16 downcasts in MSA attention, force F32 indexer score accum
962d0acc
Add Minimax eos to llama vocab
498ef417
Guard edge case where idx cache can become stale after a tail trim
09644b5d
Change resize Pad to none, resize alg to Bicubic Pillow
f096bbb4
Merge branch 'ggml-org:master' into MSA-Vision
59262637
Merge branch 'master' into MSA-Vision
c995548f
Review driven changes
e5695030
Update llama-kv-cache.cpp
1209cbff
Merge branch 'ggml-org:master' into MSA-Vision
0fbac4c1
ngxson
changed the title
Merge remote-tracking branch 'origin/master' into MSA-Vision
2c1be73c
rm unrotated pos_t
3c53a58b
fused rope w + pad
f1f0a870
rename merge --> merger for consistency
522d51de
add review skill for mtmd
ee3735fb
graph should use hparams n_merge
a4c1cd83
ngxson
approved these changes
on 2026-07-26
fix lint
dd0fe8b0
Login to write a write a comment.
Login via GitHub