Add xlstm model #39665
Cyrilvallez
merged
163 commits into
main
from
xlstm-model
Add xLSTM cleanly with optimizations.
e03391c0
Fix style.
cf14cea1
Fix modeling test.
4254fdcb
Make xLSTM package optional.
87dd1dbe
Fix: Update torch version check.
7a8d10e7
Fix: Bad variable naming in test.
4456672a
Fix: Import structure cleaning with Ruff.
41173b83
Fix: Update docstrings.
e4afcd1f
Fix: Mitigate unused config attr tests by explicit usage.
cf5bcdcb
Fix: Skip tests, if xlstm library is not installed.
1d6642f2
Feat: Enable longer context window for inference by chunking.
cfabeca3
Fix: Make training test pass by lowering target accuracy.
fa5859c6
Chore: Increase test verbosity for failing generation test.
6896afb4
Update docs/source/en/model_doc/xlstm.md
34c54c3c
Fix: Make xlstm available even without CUDA.
495de792
Chore: Remove unnecessary import.
dd69d761
Fix: Remove BOS insertion.
87010c69
Chore: Improve xLSTMCache documentation.
1e27a6f1
Integrate basic xLSTM fallback code.
c2232115
Chore: Remove unnecessary import.
207bb98e
Chore: Remove duplicate LayerNorm.
79939877
chore: update copyright, minor reformatting
34182bab
fix: refactor mLSTMStateType due to missing torch import
8087039d
fix: add missing import
3482173c
Chore: Replace einops.
71420768
fix: apply ruff formatting
10732c5d
fix: run `make fix-copies` to re-generate dummy_pt_objects.py
890c5913
fix: make type hints Python 3.9 compatible
6604d59f
fix: remove obsolete import
ffcf0d60
fix: remove obsolete method from docs
e53c0896
chore: remove obsolete `force_bos_token_insert` from config
3f6aec99
Merge branch 'main' into integrate_xlstm_clean
ee2d0487
Chore: Remove duplicated xLSTMCache class.
07e1271c
Merge branch 'main' into integrate_xlstm_clean
e2c44811
Fix: Formatting of modeling_xlstm.py
22be117c
Chore: Remove xlstm package requirement from test. Re-add update_rnn_…
59b421fd
Merge branch 'main' into integrate_xlstm_clean
34af1097
Fix: Update xLSTMCache docstring.
faffe5f8
Feat: Add proper initialization of xLSTM.
3cabdc59
Chore: Re-format files.
2d758e00
Chore: Adapt format.
26f8f02e
Merge branch 'main' into integrate_xlstm_clean
ff55a7cd
Merge branch 'main' into integrate_xlstm_clean
c55c0fba
Merge branch 'main' into integrate_xlstm_clean
e52386b5
Merge branch 'main' into integrate_xlstm_clean
addefe0c
Fix: xLSTMCache import restructuring.
3b567665
Fix: Add __all__ lists to modeling and configuration files.
05152fbb
Chore: Reformat.
75231aa9
Merge branch 'main' into integrate_xlstm_clean
d01db9f1
Merge branch 'main' into integrate_xlstm_clean
1f2ea7db
Merge branch 'main' into integrate_xlstm_clean
98f4f029
Merge branch 'main' into integrate_xlstm_clean
5e5ed329
Merge branch 'main' into integrate_xlstm_clean
ffe2e911
Merge branch 'main' into integrate_xlstm_clean
d43976cd
Merge branch 'main' into integrate_xlstm_clean
e4becc32
Merge branch 'main' into integrate_xlstm_clean
29f8a3c4
Fix: Remove unnecessary update_rnn_state function.
de1d2a08
Fix: Undo test accuracy quickfix.
ad6f87df
Fix: Update copyright year, remvoe config copy.
eb2a6fd8
Chore: Flatten all internal configs to xLSTMConfig.
61c6555b
Merge branch 'main' into integrate_xlstm_clean
adfc4367
Fix: Unused config variables check.
2c480f93
Chore: Remove unnecessary imports.
11654c9d
Merge branch 'main' into integrate_xlstm_clean
4becf2ea
Fix: Unify xlstm cache argument from batch_size to max_batch_size.
01801dc0
Merge branch 'main' into integrate_xlstm_clean
8f1c0b81
Chore: Remove bad default arg value for xLSTMCache.
e04ee6e5
Merge branch 'main' into integrate_xlstm_clean
bfa4c051
Chore: Rename core configuration arguments to HF default in xLSTM.
d4805862
Merge branch 'main' into integrate_xlstm_clean
65d17415
Chore: Fix formatting.
ed7e4103
Merge branch 'main' into integrate_xlstm_clean
3cae2baf
Fix: xLSTM Cache config access.
7a0a34e4
Merge branch 'main' into integrate_xlstm_clean
8a046c5e
Fix: Update xlstm tests for config update.
c57740e3
Feat: Re-add embbeding_dim, num_blocks config options for compat with…
1628294b
Fix: Configuration xLSTM python3.9 syntax.
666e6fce
Fix: Difference to main in test_utils.py assertion.
4d80f00d
Fix: Bad syntax in xlstm config for python3.9.
7519b826
Fix: xLSTMConfig docstring.
2b2bb03d
Fix: xLSTMConfig docstring.
a2df4dd6
Merge branch 'main' into integrate_xlstm_clean
1d6e3296
Fix typing issues in xLSTM and BeiT, Paligemma.
de022f56
Fix: Exclude xLSTM from test cache utils.
f979b822
Merge branch 'main' into integrate_xlstm_clean
80eb87e3
Merge branch 'main' into integrate_xlstm_clean
c6920160
Merge branch 'main' into integrate_xlstm_clean
e5cdc442
Merge branch 'main' into integrate_xlstm_clean
5b586426
Merge branch 'main' into integrate_xlstm_clean
cea2602a
Merge branch 'main' into integrate_xlstm_clean
4d43edd5
Merge branch 'main' into integrate_xlstm_clean
0922dd57
Merge branch 'main' into integrate_xlstm_clean
206b29fd
Merge branch 'main' into integrate_xlstm_clean
3c6a2abc
Merge branch 'main' into integrate_xlstm_clean
2ada509c
Merge branch 'main' into integrate_xlstm_clean
9e6137e2
Merge branch 'main' into integrate_xlstm_clean
9b61860d
Merge branch 'main' into integrate_xlstm_clean
448e6f42
Merge branch 'main' into integrate_xlstm_clean
9da66534
Merge branch 'main' into integrate_xlstm_clean
6f6bcc66
Merge branch 'main' into integrate_xlstm_clean
d90b73c7
Merge branch 'main' into integrate_xlstm_clean
6d3aa9e4
Merge branch 'main' into integrate_xlstm_clean
a725943e
Merge branch 'main' into integrate_xlstm_clean
cce8b597
Merge branch 'main' into integrate_xlstm_clean
3ff2a273
Merge branch 'main' into integrate_xlstm_clean
6754ca17
Merge branch 'main' into integrate_xlstm_clean
c152d23b
Merge branch 'main' into integrate_xlstm_clean
02c88712
Merge branch 'main' into integrate_xlstm_clean
225ec891
Merge branch 'main' into integrate_xlstm_clean
dae0d81f
Chore: Fix style.
feefeab2
Chore: Fix format.
7a66ecfb
Merge branch 'main' into integrate_xlstm_clean
7067c310
Merge branch 'main' into integrate_xlstm_clean
6b2a79bc
Merge branch 'main' into integrate_xlstm_clean
32dcea8c
Merge branch 'main' into integrate_xlstm_clean
75cb489e
Merge branch 'main' into integrate_xlstm_clean
0068ab6b
Merge branch 'main' into integrate_xlstm_clean
b3f40e38
Merge branch 'main' into integrate_xlstm_clean
a3c7fd7d
Merge branch 'main' into integrate_xlstm_clean
3de5f24a
Merge branch 'main' into integrate_xlstm_clean
7a2983f2
Merge branch 'main' into integrate_xlstm_clean
1bde78a3
Chore: Remove unnecessary LayerNorm, NormLayer layer abstractions.
7c239aaa
Chore: Remove asserts and replace with ValueErrors.
f8fcc6ed
Chore: Update __init__.py structure of xLSTM.
44550fa4
Chore: Clean xLSTM initialization of weights.
c7ce6a50
Fix index names in modeling_xlstm.py
5f8a399c
Update xlstm model test typing annotations.
b15d6b8d
Merge branch 'main' into integrate_xlstm_clean
bd288058
Merge branch 'main' into integrate_xlstm_clean
bd9ae673
Merge branch 'main' into integrate_xlstm_clean
3a5bab8a
Fix: Remove all asserts.
a010b18b
Revert changes to the main __init__.py
ab301098
Fix: Move xLSTMCache to modeling_xlstm.py
718e4432
Fix: Remove xLSTMForCausalLM mapping from modeling_auto.py
55ef5915
Remove xLSTMCache from dummy_pt_objects.py
733a42a6
Fix: Remove extended torchdynamo compilation check integrating cuda g…
e955830d
Revert test_cache_utils.py xLSTM change.
0fe56334
Fix: Move xLSTM init functions before init call.
c1a69b38
Remove xLSTMCache from generation utils.
78eb2576
Fix: Clean xLSTM init functionality for recursive calls.
66e98cb2
Fix: Move xLSTMCache before its first call.
8dc3179a
Fix formatting.
70f797c5
Add partial docstring for xLSTMModel forward.
468b97f2
Fix xLSTMCache docstring in xLSTMModel.
89f623f8
Remove xLSTMCache from public documentation. Update auto_docstring.
55bc9724
Merge branch 'main' into integrate_xlstm_clean
2480d312
Merge branch 'main' into integrate_xlstm_clean
5b13a31f
Merge branch 'main' into integrate_xlstm_clean
99137e89
Merge branch 'main' into xlstm-model
55814f09
Remove all agressive shape comments
e4c94e5b
style
bf357449
Fix names
e4969e65
simplify
d867780e
remove output_hidden_states
3adc3d55
Update modeling_xlstm.py
f17ceae0
Update modeling_xlstm.py
35aa0c28
Update test_modeling_xlstm.py
4ee3afcc
Update modeling_xlstm.py
b2f0da55
Update modeling_xlstm.py
00bb1d12
fix
5aa1ee5f
fix
2337f808
style
d496c729
style
a876235a
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub