add initial tests
78762b20
init draft version
0bab3c2a
Merge branch 'main' into add-inkling2-loop
01147ff1
update tests
a69e0178
more todo
02ee7260
update cache layer handling
39b88b8d
cleanup
aa0cc269
split layers
ef5f4bba
cleanup message passing
3f2a85ff
update gates
f8482977
refactor message passing
aea15837
cleanup
7434736b
update
1c79cce9
cleanup
93542507
remove defensive checks
abe71273
move to correct place
e2abf0f4
dequant step 1
29b216e2
flatten loop config
737f9809
merge message queries
6fa00506
fix
6f3482c6
norm
1aa26ba4
update init weights
5a3dab44
split layer loop
a6a4af27
more splitting
2983e57b
fix tests
c1846e6f
rename
ff6dc648
update inkling2 attentions
d03e8d35
remove unneeded layers
95075700
more cleanup
06661c00
cleanup
ef20ef1b
refactor relative logits
e23ef0ff
merge k/v proj
908b61b9
move things
cb6894c9
fix packed sequences
1a8cad7c
add Inkling2GatedOutputProjection
cc4f3074
drop sdpa
d425f551
refactor scales
6427d4fe
refactor rel pos bias
6620ca61
minor
eb892bfc
fix repo
32049ef7
handle unpadded_vocab_size
ed58c546
merge kv proj
3add8847
fix repo
6e657c3b
fix head loading
9a940316
fix mxfp8 loading
3b759727
use attn interface
b01cd5fc
text_hidden_size is now in the config
48141e7c
use dory caching
1a15086b
move norm to proj
c9ccbdb4
move test
9aed9db2
update docs
e843e8f1
add todo
9ed3671b
update tests
18c8ac85
update docs
3ed39b6e
add blog link
46c36e56
tiny checkpoint
6b1044a9
Merge branch 'main' into add-inkling2-loop
9a1ff825
cleanup
eb4b4a37
make linter happy
6819273b
cleanup
8d628c77
add a10 expectations
09c356e8
skip export tests
f0fd853c
Merge branch 'main' into add-inkling2-mini
e916f57d
update links
c5b376ce
update links
e18dccc2
Merge branch 'add-inkling2-mini' of https://github.com/huggingface/tr…
e3a65e2c
update tests
6173a2f6
fix fp32 loading
bc3f7155
add kernels note
a10ebc3c
consistency
430050f6
Cyrilvallez
deleted the add-inkling2-mini branch 2 hours ago
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub