mosaicml/llm-foundry

Pull Requests Commits

added json formatter to foundry

jjanezhang committed 2 years ago

831bef19

Update composer to 0.17.2 and streaming to 0.7.2 (#822)

irenedea committed 2 years ago

Verified 1c4ebbf4

Use `sync_module_states: True` when using HSDP (#830)

abhi-mosaic committed 2 years ago

Verified 29b41ba9

Align GLU implementation with LLaMa (#829)

vchiley committed 2 years ago

Verified 1ac5c4bd

Bump turbo version to 0.0.7 (#827)

mvpatel2000 committed 2 years ago

Verified 2b1fa79d

Revert "Default to using tokenizer eos and bos in convert_text_to_mds.py (#823)" (#825)

irenedea committed 2 years ago

Verified f2b7bef4

Default to using tokenizer eos and bos in convert_text_to_mds.py (#823)

irenedea committed 2 years ago

Verified 5eb96e92

Fix token counting to allow there to be no attention mask (#818)

dakinggg committed 2 years ago

Verified 836ab953

Add generic flatten imports to HF checkpointer (#814)

b-chu committed 2 years ago

Verified bbf5cc7e

Remove redundant transposes for rope rotation (#807)

ShashankMosaicML committed 2 years ago

Verified 289536bd

Updating the Flash Attention version to fix cross entropy loss (#812)

ShashankMosaicML committed 2 years ago

Verified 2ba9224f

Async eval callback (#702)

aspfohl committed 2 years ago

Verified a7e916b3

Expand options for MMLU. (#811)

Mansheej Paul committed 2 years ago

Verified dc77451f

Fix remote path check for UC volumes (#809)

irenedea committed 2 years ago

Verified ce675222

Clean up the logs, bump datasets and transformers (#804)

dakinggg committed 2 years ago

Verified 06b9a1f3

Make the ffn activation func configurable (#805)

vchiley committed 2 years ago

Verified 15e79f38

Fix openai not conditioned imports (#806)

dakinggg committed 2 years ago

Verified f56f1228

update openai wrapper to work with tiktoken interface and newest openai version (#794)

bmosaicml committed 2 years ago

Verified 5cc4dd4c

Fix token counting to use attention mask instead of ids (#802)

dakinggg committed 2 years ago

Verified 5388dc0d

clean up resolve_ffn_hidden_and_exp_ratio (#801)

vchiley committed 2 years ago

Verified 5fdcc43b

Enable GLU FFN type (#796)

vchiley committed 2 years ago

Verified 0797aa66

Minor readme updates and bump min python version (#799)

dakinggg committed 2 years ago

Verified 1c3c909d

Adding a fix for Cross Entropy Loss for long sequence lengths. (#795)

ShashankMosaicML committed 2 years ago

Verified 96cf646c

Enable flag to not pass PAD tokens in ffwd (#775)

bcui19 committed 2 years ago

Verified 410d5c71

Auto packing fixes (#783)

irenedea committed 2 years ago

Verified 34ec2f73

Remove from mcli.sdk imports (#793)

aspfohl committed 2 years ago

Verified 693ef102

Update gauntlet v0.2 to reflect results of calibration (#791)

bmosaicml committed 2 years ago

Verified 8d96f9d0

Fix utf-8 decode errors in tiktoken wrapper (#792)

dakinggg committed 2 years ago

Verified 75cc1e18

Remove tests and support for torch <2.1 (#787)

dakinggg committed 2 years ago

Verified 2017c028

tiktoken chat formatting (#784)

rajammanabrolu committed 2 years ago

Verified ef60e8ee

Older