Validation #1031
XiaohanZhangCMU
merged
207 commits into
mosaicml:byod/data_validation
from
XiaohanZhangCMU:validation
add validation script
8cb65225
update
c59c11fd
change token count function
66f34ebd
reorganize cells
2cd387b5
Add unit tests
3eac3bf1
Add a printout for CPT
d2d97677
update question
be255917
Add questions
4651be7d
Fix lints
5cd6a94d
Merge branch 'main' into validation
8e2c1f4b
update format
e6e4a81f
Merge branch 'validation' of github.com:XiaohanZhangCMU/llm-foundryX …
34c56904
update
1668b9a4
nb source
22191350
add validation script
86c6e87a
update
678b3766
change token count function
297e057d
reorganize cells
09d0ebbd
Add unit tests
460df65c
Add a printout for CPT
3ffd2003
update question
9362886a
Add questions
898e5acf
Fix lints
a4bef711
update format
4ca9cc6c
update
d636a0f8
nb source
827d1551
Remove license insert for validation notebook
6bbf3fce
Merge branch 'validation' of github.com:XiaohanZhangCMU/llm-foundryX …
4f6a4fb3
Add validation utils
5966b68b
Merge branch 'main' into validation
da178136
Minor cleanups (#858)
a7c36bcc
update utils/__init__.py to include extra validation functions
55e46265
update notebook
45544a1e
update
d2797b35
Merge branch 'validation' of github.com:XiaohanZhangCMU/llm-foundryX …
019da779
update
756fdae2
Read UC delta table (#773)
6de8c37f
Add download remote function to util
93b5a9f9
update
b47c878a
remove fused layernorm (#859)
fa8f3d96
update
13fd34c7
update
610f6694
update
9f2e51b3
update
ec68f10f
update
1e76068e
update
7a5c164f
Merge branch 'main' into validation
e76038fc
update
5b413f55
update
a1aa31f1
update
d24fd5ce
Remove hardcoded combined.jsonl with a flag (#861)
da3bea14
bump (#828)
936e3a1b
Add dask and dataframe_to_mds
55fce374
update
86e2412e
update
bbfec65d
update
b2e880de
update
596443af
Add notebook
ea651873
update
378a4e09
update
af6e9aa9
remove script and tests, keep notebook
4e286ecd
update
09c4892c
update
c82da6cc
update
e5f83cc5
update
17d2b9fd
Merge branch 'main' into validation
6579d552
Merge branch 'byod/data_validation' into validation
56308ffa
Always initialize dist (#864)
6517a307
updated notebook
4daa3248
Merge branch 'main' into validation
b809691f
remove scripts keep notebook
8b75f946
merge with byod/data_validation
99bf2cd5
update notebook. rephrase.
22014d67
Logs upload URI (#850)
d05c0992
merged
d9f28aac
update
43c8ac9d
Delta to JSONL conversion script cleanup and bug fix (#868)
b69318e8
fix mock (#872)
b40b8051
Add response tokens
b8ac771e
update
1b9681c0
merge
16883c22
fix regex (#877)
f43d1cfb
Precompute flash attention padding info (#880)
6dcc0d84
add missing import (#882)
2e4f4b2f
fsdp wrap refac (#883)
19ee0869
Update model download utils to support ORAS (#881)
35bb3390
Update license (#887)
19368c66
Fix tiktoken add generation prompt (#890)
c9a49d0b
update
c7567f13
Upgrade Datasets version (#892)
b2a0c03b
Disable MDSWrite, return token counts
1764b722
Bump transformers version to support Mixtral (#894)
02c44ad6
Add `tokenizer-only` flag to only download tokenizers from HF or oras…
07d6db36
Foundational Model API eval wrapper (#849)
f2614a4a
Change plot settings
808ced53
Fix conflict
26ae516b
update notebook
a212ee89
update
d2798172
update notebook
dbe3f4ef
update
3005718d
update
f5b900cc
Merge branch 'byod/data_validation' of https://github.com/mosaicml/ll…
02d0979d
update notebook
2f883a7c
update
0315cafa
Add better error for non-empty local output folder in convert_text_to…
36fcb5e5
Allow bool input for loggers (#897)
4961436f
Enable QK Group Norm (#869)
ea0521ca
patch (#905)
1469dcb6
Add new GC option (#907)
69bf90ed
No symlinks at all for HF download (#908)
26349879
Adds support for chat formatted finetuning input data. (#884)
ac783541
Add flag to enable/disable param upload (#912)
534f5b4d
Add support for eval_loader & eval_subset_num_batches in async callba…
bdcce637
Add the model license file for mlflow (#915)
34cdaf68
Warn instead of error on tokenizer-only with http (#904)
b48fa58f
Fix fmapi_chat for instruct models and custom tokenizers (#914)
f132f055
Make yamllint consistent with Composer (#918)
34a1dad9
Create HF checkpointer model on meta device (#916)
5ccceffa
Tiktoken chat format fix (#893)
8c5182c4
fix dash issue (#919)
ff8c32be
Fixes yaml linting (#920)
7dc51e96
Adding deprecation warning for Flash Attention 1 and user warning aga…
169b6536
Add rich formatting to tracebacks (#927)
b928de8f
Changes to setup.py invalidate docker cache. Use branch name in docke…
7a81f194
Remove .ci folder and move FILE_HEADER (#931)
203edad0
Throw error when no EOS (#922)
4b82876d
bump (#934)
86b5a981
Update eval_gauntlet_callback.py with math.log2 (#821)
d9874d2a
Switch to the Composer integration of LoRA (works with FSDP) (#886)
15ee0acc
Refactoring the function to accept list of metric names instead of a …
706ea7dd
Remove extra call to .to and load_state_dict in hf checkpointer (#939)
b9d2bfaf
Fixing the gen_attention_mask_in_length function to handle the case w…
ad126a62
Update lora docs (#941)
25599294
fix (#942)
3f21bb7e
Retrieve license information when local files are provided for a pret…
2e0a8458
Add and use VersionedDeprecationWarning (#944)
c1c4bbfb
Bump llm-foundry version to 0.5.0 (#948)
a667ebf5
Fix chain-of-thought tasks (#824)
6591f480
Add finetuning streaming dataset conversion (#933)
105f7663
Add default signature to mlflow saved model (#952)
60ab97fb
allow te to use meta device with deferred init (#958)
8c7d6f43
Update TUTORIAL.md (#957)
fe17c224
Bump mcli yaml foundry version to v0.5.0 (#959)
9f101847
add finutuning with streaming dataset example (#945)
60cdd0be
Add fully configurable activation checkpointing (#951)
2e59620a
Use create_model_version instead of register_model (#953)
2f64a144
Add streams support (#946)
aa0ea6e2
Fix typo (#966)
c7c9d247
Fix eval.py with lora (#965)
12d1ca73
add memorysnapshot to callbacks (#810)
491fa7ab
Adding curriculum learning callback (experimental) (#954)
6afd4460
strengthened chat formatting validation (#960)
122f9655
Add new base images and remove fa1 images (#970)
78cbe085
Add new ICL kwargs in eval.py and long_context yamls (#925)
e88cdf11
Make Composer pins consistent with each other (#972)
bce53741
Make turbo an optional dependency (#964)
a62ee55a
Fix fewshot_random_seed default setting (#974)
73033ee2
improve error msg when checking target_blocks in activation_checkpoin…
2e8982e2
Torch 2.2 upgrade - Part 1 (#976)
1ef7409c
Torch 2.2 - Part 2 (#979)
e0756e10
PyTorch 2.2 - Part 3 (#981)
da2c8632
Remove torch 2.1 from docker build (#982)
3a992708
Async callback: Don't skip checkpoints, reliably only launch async ev…
6e3842b6
Token accuracy metrics (#983)
2431730f
do not mention 1.13 in readme (#988)
63c88d09
Patch test, lock mcli version (#990)
dff2cf4f
Bump gha timeouts (#991)
386ae36e
Fix readme typo (#993)
2478f0ad
if condition in tie weights added (#989)
e5fffacd
bump composer version (#995)
44fd365d
Trim examples ahead of time for auto packing (#994)
d527c9b4
add oom observer callback (#932)
b0825114
Change ci/cd to use ci-testing repo
e3f214e7
Revert "Change ci/cd to use ci-testing repo"
5abbca06
Use ci-testing repo (#1000)
2436c00b
Make CodeEval respect device_eval_batch_size (#956)
d104d16a
Remove try except around imports (#1004)
2dea7373
Deprecate triton, prefix lm, llama attention patch, and text denoisin…
3880d045
add magic filename for sharded state dicts (#1001)
96c8218b
bump (#1009)
cbdddf0f
Fix evaluators actually pulling eval metrics (#1006)
09ff5505
Build torch 2.2.1 images (#1010)
fd8cbaf6
add 2.2.1 tests (#1011)
57289699
Bump min torch pin (#1013)
f4f64140
Fix extra BOS token in front of response for some tokenizers (#1003)
cf0f5e51
Bump min composer pin (#1015)
86c87461
add default for eval interval (#987)
5261a55d
Add support for olmo (#1016)
93d7a051
Add deeper support for multi-turn chats and loss-generating tokens in…
64212cba
Fix profiling packing ratio to explicitly say 1 (#1019)
c2aec307
Bump transformers to 4.38.2 (#1018)
2b17497d
that kwargs (#1020)
36ab1bae
Update readme with pytorch 2.2.1 (#1021)
2fc5d332
Add code import to train/eval scripts (#1002)
d61c53d2
finish (#1022)
4e437922
Bump version to 0.6.0 (#1023)
257c25d5
Fix typo in monolithic chkpt callback docs (#1024)
4e8a875b
update pip install link
1a510ff4
Change done file location
530a55a0
Create the dest folder
81c37573
Allow code-quality workflow to be callable (#1026)
14e2dec8
update notebook
f88917d5
update
4c86f74d
Merge branch 'byod/data_validation' into validation
962974bf
Merge pull request #1 from mosaicml/byod/data_validation
67f7b4ce
update notebook
28cd2e6d
fix conflict
9a19d8ac
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub