mosaicml/llm-foundry

Pull Requests Commits

Add Model sharding support with deepspeed (#836)

Vivek Goel committed 1 year ago

Verified 3c104fd0

update reqs to 1.13

abhi-mosaic committed 2 years ago

2a52d87e

abhi-mosaic committed 2 years ago

5c2e0cea

abhi-mosaic committed 2 years ago

3a419fd7

abhi-databricks committed 2 years ago

83633065

abhi-mosaic committed 2 years ago

f4f8fb54

Abhinav Venigalla committed 2 years ago

63771789

Add support to run MPT-1b training on Habana device (HPU) using DeepSpeed. (#527)

Vivek Goel committed 2 years ago

9fcdfe5b

remove tokenizer_name field (#735)

dakinggg committed 2 years ago

Verified 45113ebf

Rename log message (#734)

b-chu committed 2 years ago

Verified 753feff9

Better defaults for StreamingDataset subclasses (#723)

snarayan21 committed 2 years ago

Verified d1960f2c

add act checkpoint at sub layer level (#720)

cli99 committed 2 years ago

Verified 8ba697ce

Enable `tie_word_embeddings` config setting to enable / disable weight tied embeddings (#728)

vchiley committed 2 years ago

Verified 78991788

Make TiktokenTokenizerWrapper compatible with convert_composer_to_hf.py (#730)

irenedea committed 2 years ago

Verified d11ba820

Create AWS docker image for fine tuning (#731)

j316chuck committed 2 years ago

Verified e7223dad

Gate the dist calls in build_tokenizer (#732)

dakinggg committed 2 years ago

Verified c3f0cf97

Fix logging verbosity in HF model download script and repair symlinks (#727)

jerrychen109 committed 2 years ago

Verified 7c4d24a8

Combine filters into one, to avoid datasets error (#729)

dakinggg committed 2 years ago

Verified 2f91a64a

Add Unity Catalog support to HF checkpointer (#721)

dakinggg committed 2 years ago

Verified d2ddb834

Attempt to fix a very occasional hang in datasets map/filter (#725)

dakinggg committed 2 years ago

Verified efaa5454

change default overwrite to True (#724)

dakinggg committed 2 years ago

Verified ab9b9385

raise timeout (#719)

mvpatel2000 committed 2 years ago

Verified 84c86e3b

Set persistent_workers = False for packing profiling (#718)

dakinggg committed 2 years ago

Verified dd157918

Add databricks dependency (#717)

irenedea committed 2 years ago

Verified 2b74cb25

Adding support for Rotary Position Embeddings (#675)

ShashankMosaicML committed 2 years ago

Verified 1d504c85

Add Hugging Face model download script (#708)

jerrychen109 committed 2 years ago

Verified 58d7cf3e

Run HF dataset processing on local rank 0 first (#716)

dakinggg committed 2 years ago

Verified c2f5742d

Allow logging metadata (#714)

dakinggg committed 2 years ago

Verified ffb58f18

Remove HumanEval tasks from ICL eval (#715)

tbarton16 committed 2 years ago

Verified be467aee

Add support for auto packing ratio (#683)

irenedea committed 2 years ago

Verified ca8e6b5c

Older