Add OLMo model family #29890
ArthurZucker
merged
71 commits into
huggingface:main
from
2015aroras:add-olmo-implementation
Add OLMo using add-new-model-like with Llama
0afbd4a3
Fix incorrect tokenizer for OLMo
9842c5c5
Copy-paste relevant OLMo methods and their imports
fc21050a
Add OLMo config
3a748b90
Modify OLMo config to follow HF conventions
e15ed3c1
Remove unneeded Llama code from OLMo model
0090b1a6
Add ability for OLMo model to output attentions
8719408b
Add OLMoPreTrainedModel and OLMoModel
8b1f0821
Add OLMoForCausalLM
49bb675d
Minor fixes to OLMo model for style and missing functions
32ec3808
Implement OLMo tokenizer
cba91d4c
Implement OLMo to HF conversion script
7cc2a9aa
Add tests for OLMo model
a1f92949
Add tests for OLMo fast tokenizer
f4fa8048
Add auto-generated dummy objects
7b287cad
Remove unimplemented OLMo classes from auto and init classes and re-f…
d1ba9c75
Add README and associated auto-generated files
159b124d
Use OLMo names for common properties
8d8baf5b
Run make fixup
42492b90
Remove `|` from OLMo typing
a92d52a3
Remove unneeded tokenization_olmo.py
f59ea67f
ArthurZucker
commented
on 2024-03-27
Revert model, config and converter to add-new-model-like Llama
4bc9b6bd
Move logic for adding bos/eos token into GPTNeoxTokenizerFast
4fdd1dad
Change OLMoConfig defaults to match OLMo-7B
3e8ceb14
Use GPTNeoXToknizerFast in OLMo tokenizer tests
f68a915d
Modify auto-generated OLMoModelTests to work for OLMo
43eaeffc
Add non-parametric layer norm OLMoLayerNorm
6dd18528
Update weight conversion script for OLMo
7b438cd8
Fix __init__ and auto structure for OLMo
33461951
Fix errors from make fixup
7ba083f2
Remove OLMoTokenizerFast from documentation
47affa6a
Merge remote-tracking branch 'upstream/main' into add-olmo-implementa…
b468a0a9
Add missing 'Copied from' for OLMoModel._update_causal_mask
98db6a92
Run make fix-copies
dd0f76c1
Rearrange string replacements in OLMoForCausalLM Copied from
03407788
Move OLMo and Llama CausalLM.forward example into global constants
edbc4d1e
Fix OLMO_GENERATION_EXAMPLE doc string typo
75f5ceeb
Add option for qkv clipping to OLMo
94b31e1d
Rearrange OLMoConfig kwargs in convert_olmo_weights_to_hf
3805b97d
Add clip_qkv to OLMoConfig in convert_olmo_weights_to_hf
a5558193
Fix OLMo tokenization bug using conversion script
aea6b062
Keep model in full precision after conversion
0bbb55ad
Do not add eos token automatically
21faf45e
Update references to OLMo model in HF Hub
ee005837
Do not add eos token during encoding by default
63a6fbd0
Fix Llama generation example
76c79709
Run make fixup
450f0381
OLMo 7B integration test fix
4df56a4b
Merge branch 'main' into add-olmo-implementation
faa188f7
Remove unneeded special case for OLMoConfig
0b227bb2
OLMo 7B Twin 2T integration test fix
9ff65a4a
Fix test_model_7b_greedy_generation
70f593d3
Remove test_compile_static_cache
54f4aa59
Fix OLMo and Llama generation example
33d3be71
Run make fixup
eab18b6d
2015aroras
commented
on 2024-04-13
molbap
commented
on 2024-04-13
molbap
commented
on 2024-04-13
Revert "OLMo 7B integration test fix"
628f9d5d
Revert "OLMo 7B Twin 2T integration test fix"
751195f7
Ungate 7B integration tests and fix greedy generation test
68466ddc
Add retries for flaky test_eager_matches_sdpa_generate
91f163f9
Fix output of doc example for OLMoForCausalLM.forward
aa4a92f3
Downsize OLMo doc test for OLMoForCausalLM.forward to 1B model
43ea80ef
Try fix incorrect characters in OLMoForCausalLM.forward doct test
fcc457cc
Try fix incorrect characters in OLMoForCausalLM.forward doc test usin…
e5fdf507
Remove pretraining_tp from OLMo config and model
4f062d76
Add missing 'Copied from' instances
3c60cf3a
Remove unneeded causal_mask from OLMoModel
a037fcac
Revert Llama changes
004f80ed
Ignore copy for OLMoForCausalLM.forward
826c91ba
Change 'OLMo' to 'Olmo' in classes
c7f4e748
Move minimal OLMo tokenization tests to model tests
ec344c3a
Add missed 'Copied from' for repeat_kv
78148fe1
ArthurZucker
approved these changes
on 2024-04-17
Login to write a write a comment.
Login via GitHub