Add Inkling2 Mini #51050

Cyrilvallez merged 70 commits into main from add-inkling2-mini
guarin
guarin add initial tests
78762b20
guarin init draft version
0bab3c2a
guarin Merge branch 'main' into add-inkling2-loop
01147ff1
guarin update tests
a69e0178
guarin more todo
02ee7260
guarin update cache layer handling
39b88b8d
guarin cleanup
aa0cc269
guarin split layers
ef5f4bba
guarin cleanup message passing
3f2a85ff
guarin update gates
f8482977
guarin refactor message passing
aea15837
guarin cleanup
7434736b
guarin update
1c79cce9
guarin cleanup
93542507
guarin remove defensive checks
abe71273
guarin move to correct place
e2abf0f4
guarin dequant step 1
29b216e2
guarin flatten loop config
737f9809
guarin merge message queries
6fa00506
guarin fix
6f3482c6
guarin norm
1aa26ba4
guarin update init weights
5a3dab44
guarin split layer loop
a6a4af27
guarin more splitting
2983e57b
guarin fix tests
c1846e6f
guarin rename
ff6dc648
guarin update inkling2 attentions
d03e8d35
guarin remove unneeded layers
95075700
guarin more cleanup
06661c00
guarin cleanup
ef20ef1b
guarin refactor relative logits
e23ef0ff
guarin merge k/v proj
908b61b9
guarin move things
cb6894c9
guarin fix packed sequences
1a8cad7c
guarin add Inkling2GatedOutputProjection
cc4f3074
guarin drop sdpa
d425f551
guarin refactor scales
6427d4fe
guarin refactor rel pos bias
6620ca61
guarin minor
eb892bfc
guarin fix repo
32049ef7
guarin handle unpadded_vocab_size
ed58c546
guarin merge kv proj
3add8847
guarin fix repo
6e657c3b
guarin fix head loading
9a940316
guarin fix mxfp8 loading
3b759727
guarin use attn interface
b01cd5fc
guarin text_hidden_size is now in the config
48141e7c
guarin use dory caching
1a15086b
guarin move norm to proj
c9ccbdb4
guarin move test
9aed9db2
guarin update docs
e843e8f1
guarin add todo
9ed3671b
guarin update tests
18c8ac85
guarin update docs
3ed39b6e
guarin add blog link
46c36e56
guarin tiny checkpoint
6b1044a9
guarin Merge branch 'main' into add-inkling2-loop
9a1ff825
guarin cleanup
eb4b4a37
guarin make linter happy
6819273b
guarin cleanup
8d628c77
guarin add a10 expectations
09c356e8
guarin skip export tests
f0fd853c
guarin Merge branch 'main' into add-inkling2-mini
e916f57d
guarin guarin added New model
guarin update links
c5b376ce
guarin update links
e18dccc2
guarin Merge branch 'add-inkling2-mini' of https://github.com/huggingface/tr…
e3a65e2c
guarin
HuggingFaceDocBuilderDev
guarin update tests
6173a2f6
github-actions
qgallouedec
qgallouedec
qgallouedec
benthecarman
guarin fix fp32 loading
bc3f7155
guarin add kernels note
a10ebc3c
github-actions
guarin
Cyrilvallez consistency
430050f6
Cyrilvallez Cyrilvallez merged 046bdb8c into main 2 hours ago
Cyrilvallez Cyrilvallez deleted the add-inkling2-mini branch 2 hours ago

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
Milestone