Upgrade to v0.10.0 #1427
hlahkar
wants to merge
399 commits into
mosaicml:habana_alpha
from
hlahkar:version_0.10.0_hpu
improve error msg when checking target_blocks in activation_checkpoin…
2e8982e2
Torch 2.2 upgrade - Part 1 (#976)
1ef7409c
Torch 2.2 - Part 2 (#979)
e0756e10
PyTorch 2.2 - Part 3 (#981)
da2c8632
Remove torch 2.1 from docker build (#982)
3a992708
Async callback: Don't skip checkpoints, reliably only launch async ev…
6e3842b6
Token accuracy metrics (#983)
2431730f
do not mention 1.13 in readme (#988)
63c88d09
Patch test, lock mcli version (#990)
dff2cf4f
Bump gha timeouts (#991)
386ae36e
Fix readme typo (#993)
2478f0ad
if condition in tie weights added (#989)
e5fffacd
bump composer version (#995)
44fd365d
Trim examples ahead of time for auto packing (#994)
d527c9b4
add oom observer callback (#932)
b0825114
Change ci/cd to use ci-testing repo
e3f214e7
Revert "Change ci/cd to use ci-testing repo"
5abbca06
Use ci-testing repo (#1000)
2436c00b
Make CodeEval respect device_eval_batch_size (#956)
d104d16a
Remove try except around imports (#1004)
2dea7373
Deprecate triton, prefix lm, llama attention patch, and text denoisin…
3880d045
add magic filename for sharded state dicts (#1001)
96c8218b
bump (#1009)
cbdddf0f
Fix evaluators actually pulling eval metrics (#1006)
09ff5505
Build torch 2.2.1 images (#1010)
fd8cbaf6
add 2.2.1 tests (#1011)
57289699
Bump min torch pin (#1013)
f4f64140
Fix extra BOS token in front of response for some tokenizers (#1003)
cf0f5e51
Bump min composer pin (#1015)
86c87461
add default for eval interval (#987)
5261a55d
Add support for olmo (#1016)
93d7a051
Add deeper support for multi-turn chats and loss-generating tokens in…
64212cba
Fix profiling packing ratio to explicitly say 1 (#1019)
c2aec307
Bump transformers to 4.38.2 (#1018)
2b17497d
that kwargs (#1020)
36ab1bae
Update readme with pytorch 2.2.1 (#1021)
2fc5d332
Add code import to train/eval scripts (#1002)
d61c53d2
finish (#1022)
4e437922
Bump version to 0.6.0 (#1023)
257c25d5
Fix typo in monolithic chkpt callback docs (#1024)
4e8a875b
Allow code-quality workflow to be callable (#1026)
14e2dec8
Fix llama attention patch (#1036)
cffd75e9
Adds a decorator for experimental features (#1038)
c88169da
Bump version in yamls (#1040)
c173dd36
Remove references to attn_impl: triton (#1041)
c26309de
Registry based config - Part 1 (#975)
9bf3701c
Deprecate attention patching for llama (#1047)
3348b596
Compile GLU (#1049)
26a5fd45
log details to metadata for run analytics (#992)
31e48795
Update README.md (#1056)
c9685cf6
Add chat schema example for mlflow (#1054)
94a05bd3
Metrics registry (#1052)
813d596b
LLM Foundry CLI (just registry) (#1043)
67dcab9f
Bump Composer to 0.21.1 (#1053)
5c8a829f
Dataloaders registry (#1044)
32e14a67
Fix multi model eval (#1055)
648b1bd4
Remove unnecessary test workflow (#1058)
b9a2de63
Fix peft llama test (#1059)
8f25c186
Models registry (#1057)
2d65fc20
Remove under construction from registry (#1060)
e590acf8
Custom Exceptions for Mosaic Logger (#1014)
0ef7cd6b
Bump version to 0.7.0 (#1063)
7f0fdae6
filter files (#1067)
e42ea707
Fix context printing (#1068)
f044d6cf
Update README with DBRX (#1069)
8a69bd70
added attributes to construct msg in mapi (#1071)
28467bba
Output eval logging batch (#961)
7a8a1564
Add expandeable segments flag (#1075)
349c2ff0
Check the user provided eos / bos token id against the tokenizer eos …
d8ea2c56
Triton RMSNorm (#1050)
b765b470
Fix tiktoken vocab size (#1081)
caf7fda9
Doing the loss reduction in foundry instead of in the loss functions.…
632cb739
remove (#1082)
580a4b00
Upgrade hf chat (#1061)
394735b1
Fixes for streaming and auto packing (#1083)
d452c60d
Background mlflow model registration (#1078)
23c31734
Update README.md to include DBRX blog under "Latest News" (#1085)
5455b404
decrease mlflow hf model shard size (#1087)
c766cf95
log packing ratio progress (#1070)
e70e4241
Bump HF version (#1091)
26f0619d
fix typo in expandable_segments (#1088)
06ff95f6
Bump transformers to 4.39.3 (#1086)
607b982d
fix yamls typo (#1092)
f18768b4
Allow overrides for nested PretrainedConfig (#1089)
96b27c59
cleaned up HF/MPT conversion test (#1048)
94301cd1
update (#1097)
60a1ab4c
Norms registry (#1080)
b81897ac
fixing evaluator microbatch size (#1100)
d0d94342
Updating the streaming version in setup.py (#1103)
2939cc98
MegaBlocks release (#1102)
53160f40
Remove torch compile (#1101)
f12bc8aa
Update config_moe_args.py (#1104)
17f8aeb0
Add remote code option to allow execution of DBRX tokenizer (#1106)
7337429d
GRT-2819 fix overwritting in script (#1107)
b5fc0fad
Support ShareGPT chat format (#1098)
4cd2324e
FC layer registry (#1093)
ed3daef0
Attention layer registry (#1094)
560012b6
Dbrx finetune yaml requires save folder specified to enable autoresum…
e9b1c6e6
Revert "Update config_moe_args.py (#1104)" (#1111)
b58d68c1
Update config_moe_args.py (#1112)
6257e5b9
Migrate ICL classes to foundry (#936)
3729ba3e
FFN layer registry (#1095)
cb0de4f7
Param init registry (#1096)
676ad7f9
Add missing init file (#1113)
f01f6256
Update tests to not rely on mistral (#1117)
84b64102
Bump transformers to 4.40 (#1118)
4bb4d4a2
add `.json` to SUPPORTED_EXTENSIONS (#1114)
20cb40c3
Add option for subclasses to convert model and tokenizer in hf checkp…
63a7f125
Bump Composer to 0.21.3 (#1122)
698206d0
catch misconfigured hf dataset (#1123)
f0646e88
Pin mlflow (#1124)
34264151
Change main to a dev version (#1126)
6caa75a0
Fix deprecation versions (#1129)
0c6bd75f
Clean up the publicly exported API (#1128)
49521837
Fix HF checkpointer + mlflow bugs (#1125)
c53622e3
Update JSONL sources in eval README (#1110)
0d62e611
Mlflow datasets (#1119)
72da1d70
Fix InvalidPromptResponseKeysError bug (#1131)
6252f791
First initialize dist with gloo (#1133)
76f74b69
Fix saving of generation_config for Llama-3 (#1134)
15abf8c2
Bump datasets version (#1138)
24f65fd8
Revert "First initialize dist with gloo (#1133)" (#1139)
4aef5dea
Barrier immediately after initialize dist with logs (#1140)
6cfd2a35
Add new FT instructions (#1143)
f97f02e1
Upgrade ci-testing (#1145)
8be32542
Fix typos in callbacks with configs (#1146)
704a90a6
remove olmo (#1148)
fbcf3113
build inner model (#1147)
de5a3949
fix DatasetConstants.splints default value to protect dictionary over…
738956ee
Bump flash attention version (#1150)
bd0d1cbb
Add torch 2.3 images (#1149)
6fde2830
add torch 2.3 CI (#1151)
63ac1a4e
Comment out torch 2.3 tests (#1155)
b570b615
Fix yaml lint (#1156)
2f58965f
Move sentencepiece import (#1157)
6561330d
composer verision BUMP (#1160)
5f39606a
Uncomment GPU tests (#1162)
fa7a78a6
Update setup.py (#1163)
124de4ca
Update pr-gpu.yaml (#1164)
0d58f461
Bump min torch version to 2.3.0 (#1152)
a3e0fb9a
Add line splitting and other linting (#1161)
10b7caf7
refactoring dataloader into registries. (#1165)
3b827353
Migrate eval output logging to foundry (#1166)
ddf4aa4c
Fix import and mocking (#1169)
c0d591c0
minor fix to (#1170)
bfbb8c57
Fix config access for DBRX (#1177)
ab9dde73
Bump version v0.9.0.dev0 (#1181)
a7770148
structuredconfig for train.py and eval.py (#1051)
cc8351c4
update version names (#1185)
46b8bee8
Refactoring attention (#1182)
ac563e6d
checking if attention mask present for ignoring pad tokens in ffn (#1…
0c7bc2a3
Remove python 3.8 and add python 3.11 (#1189)
c6679d3c
Docstring fix for curriculum learning callback (#1186)
139abab7
Set ft dataloader name explicitly (#1187)
51d0d096
Fix to_container call (#1190)
1dd37c5b
fix eval (#1193)
983234da
Log exception on inactivity callback (#1194)
994209cf
Pass FC type along for all FFN types (#1196)
eef4872d
streaming version bump (#1195)
0449b604
Clearer error message for unknown example type (#1202)
8274c6cf
Added torch_dmoe defaults, bug fixes for 2D inputs (#1210)
b4146269
log eval dataset misconfiguration (#1179)
cfee4e4e
using self.shift_labels instead of self.model.transformer.shift_label…
8cd23d57
Add fc to HF export (#1209)
dc3212e3
TransformerEngine Image Build (#1204)
e70891b9
remove dead code (#1213)
3fe7f093
Make `fc_type` a dict to pass fc kwargs through (#1201)
38ae65b6
torch dmoe tests gpu oom (#1216)
3a150821
that mod (#1219)
77f9ab18
Modularize components of megablocks layer builder (#1224)
001e7c34
Add user error superclass (#1225)
8e296988
Make config/class properties on ComposerMPTForCausalLM (#1227)
c891bed3
Quick patch to check that Dataset Keys contain non-None Values (#1228)
9cc945c6
Modularize backbone class and block creation (#1229)
9120c27e
Loss v len callback (#1226)
c213ea86
Fixing the state.timestamp.batch.value issue in loss v len callback (…
6fa6026e
fix attr error for attention_classes (#1230)
b4bb34c9
fix typing (#1235)
09d88921
Add example eval scripts for dbrx PT sizes (#1218)
ef530bfb
configurable submesh (#1236)
ff92f3c7
Add retries to downloads in convert_text_to_mds.py (#1238)
fdaa58be
Move MLFlow dataset outside of log_config (#1234)
1e4bd372
add error when chat template fails (#1222)
2e10d953
Make the exceptions serializable (#1239)
867e58fe
removing rich install (#1198)
c9257b56
Chunk file reads and tokenization for text to mds conversion (#1240)
43d149b0
Make HF conversion automatically add missing imports (#1241)
b82a82bd
Add logging to convert_text_to_mds.py script (#1243)
fb9a2259
Update CODEOWNERS (#1248)
d846731d
replacing icl_task_type question_answering with generation_task_with_…
6c260f53
Change TE docker image to enable te_shard_weight (#1251)
ac56dc5a
Fix MPT HF conversion (#1257)
67928cb4
remove warning (#1258)
3966f0ef
Adding more token encoding types (#1254)
42c2d9a0
Bump Composer to 0.23.0 (#1259)
14f296c3
Bump Version to 0.10.0.dev0 (#1255)
bea61fb5
Fix typo in setup.py (#1263)
e4b8b571
Update TE Dockerfile (#1265)
db701351
Revert "Update TE Dockerfile (#1265)" (#1266)
4e53e744
revert to nvidia code (#1267)
dddb9b81
Bump composer to 0.23.2 (#1269)
dd92abf7
fix linting (#1270)
5571101a
Add torch 2.3.1 docker images (#1275)
ffec54b4
Make expandable segments on by default (#1278)
c30856f9
Add CI for torch 2.3.1 (#1281)
630fc687
Update README.md to use variables (#1282)
9b9fc24b
Add registry for ICL datasets (#1252)
1a2fac0c
fix typo in CI (#1284)
82ef0725
Fix backwards compatibility for ICL arg (#1286)
43509903
Fix packing + streaming + resumption (#1277)
dbd798e0
Dbfs HF (#1214)
1ff6c5bb
bump mlflow (#1285)
ca528d56
Add missing dependency group (#1287)
f8b28753
Update Dockerfile with TE main (#1273)
618db6f3
Fix TE HF checkpoint saving (#1280)
c23be4ab
added systemMetricsMonitor callback (#1260)
4b1fecb4
Extendability refactors (#1290)
8241f9c6
Small refactor for update batch size (#1293)
78e4cc6c
Bump min composer version to 0.23.3 (#1294)
13bd8f95
Fix grad accum type (#1296)
e8ba9b7e
Bump composer to 0.23.4 (#1297)
129e3e18
Allow passing in lbl_process_group directly (#1298)
2196d073
Add `all` transforms to train script (#1300)
8b5a1bbd
Add Retries to run_query (#1302)
fd7b1874
Bumping mlflow version to include buffering (#1303)
2267bc77
Ignore mosaicml logger for exception if excephook is active (#1301)
21c9e0a5
Add curriculum learning callback (#1256)
ef14849a
Avoid circular import in hf checkpointer (#1304)
2412b591
Remove CodeQL workflow (#1305)
bbfebda1
Update CI test to v0.0.8 (#1306)
901eee3a
update (#1307)
3edce07d
bump ci-testing to 0.0.9 (#1310)
14348fad
Fix 4 gpu tests (#1311)
472d009c
2.3.1 (#1312)
f141ee12
Provide default seed value in TrainConfig, matching EvalConfig (#1315)
0ebd7c90
Refactor hf checkpointer (#1318)
88511f78
Allows interweaving of arbitrary kinds of 'attention' layers, like sl…
8604bba9
Add optional logging of text output to EvalOutputLogging (#1283)
68c26256
Update version to release version
742f3401
Add support to run MPT-1b training on Habana device (HPU) using DeepS…
e99ec07f
wip
48b239ea
add act ckpt
a76d8242
fix
d238edc0
cleanup
b723c254
update README
820221a6
update reqs to 1.13
a69273d8
Add Model sharding support with deepspeed (#836)
8ec87168
Make config compliant to 0.10.0
de1240d5
Update ds_gaudi.sh
51beeb0a
Update README
1939ae4d
Login to write a write a comment.
Login via GitHub