Add OLMo November 2024 #34551
ArthurZucker
merged
26 commits into
huggingface:main
from
2015aroras:shanea/add-olmo1124
Add model skeletion with transformers-cli add-new-model-like
6e747c27
Convert config to modular, add rms_norm_eps, delete clip_qkv
a80ffd18
Convert model to modular, add RMSNorm
ffa794e4
Add flash attention with qk norm and no qkv clipping
75d38f04
Add decoder layer with RMSNorm after attention/feedforward layers
dbd880df
Add base and causal model
06c9c44b
Add converter improvements from OLMo repo
b73f6d39
Update weight loading in OLMo to HF converter
c8d94115
Set correct default for rms_norm_eps
4e3da14b
Set correct pipeline_model_mapping in test
87d54bb3
Run make fixup
b7939d2e
Fix model type
d39587fc
Re-run modular conversion
30c20f6d
Manually set config docs to fix build errors
cdce1572
Convert olmo-1124 to olmo_1124 to fix flash attention docs errors
3a9c61c5
Start updating tests
949648e7
Update tests
0217f409
Copy upstream test_eager_matches_sdpa_inference_1_bfloat16 changes to…
1bdaa051
Rename input_layernorm and post_attention_layernorm to reflect their …
0b1f2bf5
Use correct tokenizer
9e7c77d4
Remove test unsupported by GPT2 tokenizer
11f67eb7
Create GenerationConfig outside of from_pretrained call
0c2a264e
Use simpler init file structure
a22d9366
Add explicit __all__ to support simplified init
a3cca575
Make safetensor serialization the default
82a75c29
Update OLMo November 2024 docs
bfd2e635
ArthurZucker
approved these changes
on 2024-11-18
Login to write a write a comment.
Login via GitHub