Add Granite 4.1 Vision (granite4_vision) #45597
vasqu
merged
73 commits into
huggingface:main
from
artem-spector:add-gv41
zucchini-nlp
commented
on 2026-05-05
zucchini-nlp
commented
on 2026-05-05
Add Granite 4.1 Vision model (granite4_vision)
ce772c94
Fix auto-registration after upstream auto_mappings refactor
4a1a122b
Fix conflict marker in image_processing_auto.py
1de55666
Fix check-repo: remove spatial_stride (unused in modeling), fix auto_…
405b7d9f
Fix duplicate legacy key in conversion_mapping.py
a78cc090
Fix check-repo failures for granite4_vision
6ec99ac0
Regenerate auto_mappings.py after rebase onto upstream/main
803968f4
Fix CI failures after upstream rebase
2f55d004
Fix Sam3 auto_mappings.py entries corrupted by rebase
b1c182c1
Restore hy_v3, openai_privacy_filter, slanet entries dropped by bad r…
5cfe6b10
Revert dependency_versions_table.py to match setup.py (upstream state)
a21086ad
Fix bad rebase: remove hy-v3/slanet/openai-privacy-filter entries fro…
3f1e3c74
Remove merge_lora_adapters and PEFT adapter-toggling generate override
3ed6ac2b
Add Granite4VisionTextModel with deepstack injection, replace manual …
5c63a095
Add Granite4VisionTextConfig, fix missing TextConfig import in genera…
4180e1a0
Fix Granite4VisionTextConfig to inherit PreTrainedConfig, add missing…
e3ed3aa0
Fix class ordering: define Granite4VisionPreTrainedModel before TextM…
ffb2e8b3
Fix inv_freq corruption in Granite4VisionTextRotaryEmbedding during f…
da6e0e91
Inline downsampling into modular, add qformer_config sub-config, conv…
7c20e162
Return Granite4VisionImageFeaturesOutput from get_image_features
4255bf3b
Drop Granite4Vision image processor re-definitions, delegate to Llava…
7f2bb006
Address medium PR review items 8-14
2a2023e5
Address nit PR review items 15-23
9a67d461
Address remaining review items 20 and 29
e24fa90f
Fix test failures found by Granite4VisionModelTest
178d831e
Use lazy import for Blip2QFormerConfig; qformer_config sub_configs us…
e785c7c1
Add Granite4VisionTextModel to check_repo ignore lists; document miss…
6b30a6ab
Fix missing imports in modular: math, AutoConfig, select_best_resolution
00a7a662
Pop output_attentions/output_hidden_states from **kwargs in Granite4V…
272cc641
Fix check_modeling_structure violations (TRF002, TRF009, TRF010)
828c0c0b
Fix ruff I001 import ordering and processing consistency check
8f765e15
Remove converter-regenerated files that should not exist
ae4ced53
Remove autodoc entries for ImageProcessor classes that don't exist in…
b98fb551
Fix model card date for add_dates.py check
731e347f
Fix dependency_versions_table.py: sync mlinter version with setup.py
04c6511d
Address review round 3: config, capture_outputs, hidden states, proje…
5d7a3b9d
Nits: rename one-letter vars, AttributeError() for unused inherited c…
79c17397
Fix qformer_config: build fully-specified at init, no post-super fiel…
8682caa3
Move _can_record_outputs and _deepstack_inject to Granite4VisionPreTr…
132003a4
Address remaining review items: capture_outputs, output class, Dynami…
7b9e9a81
Regenerate modeling/processing from updated modular; fix copyright he…
27bbb3a3
Fix ruff formatting in modular_granite4_vision.py
680b0c94
Fix add_dates.py: update granite4_vision model card date to 2026-04-30
fc90de76
Remove incorrect skip from test_can_init_all_missing_weights
bb0c69eb
Fix add_dates.py: update granite4_vision model card date to 2026-05-03
ced790bd
Fix _init_weights: add nn.Embedding, nn.LayerNorm, Granite4VisionText…
316c4dc9
Fix ruff F821: detect RMSNorm by attribute pattern instead of class name
50d0a70b
Fix ruff formatting in modular_granite4_vision.py
ae41ffac
granite4_vision: address round-3 review nits
f37b9e40
granite4_vision: fix __post_init__ ordering — call super() first, dro…
e57da244
granite4_vision: address round-3 nits + fix __post_init__ ordering
3053633b
granite4_vision: replace unbound forward call with self.language_model()
d02350b8
granite4_vision: register Granite4VisionTextConfig in auto_mappings.py
16d9ad2d
granite4_vision: remove use_spatial_sampling from config
bc935397
granite4_vision: update HF Transformers date in model card to 2026-05-05
348a71c1
granite4_vision: fix test_model_is_small — shrink qformer intermediat…
428f13f2
granite4_vision: round 4 review fixes (C14, C15, C16)
987eb55b
granite4_vision: simplify __post_init__ — remove comments, tighten st…
4f3d906f
Update docs/source/en/model_doc/granite4_vision.md
860c9e0b
Apply suggestions from code review
4a5b463d
commets
36fea521
Merge remote-tracking branch 'upstream/main' into add-gv41
912cd4da
fix modular
ae492c8d
config!
2972e054
init weights fix, has an image newline
0e238e1e
fix
587f658d
fixup repo ci
83b97e91
docs fix
aa84cb6c
interesting TIL
7c11c29e
Merge branch 'main' into add-gv41
34cddd22
fix auto doc
91e5551f
granite4_vision: re-apply round 5 fixes lost in force-push
98752e7d
granite4_vision: ruff format modular_granite4_vision.py
9b6a291a
Reviewers
Assignees
No one assigned
Labels
New model
Milestone
No milestone
Login to write a write a comment.
Login via GitHub