Go
Home
Pricing
FAQ
Install
Home
Pricing
FAQ
Install
Login
via GitHub
bigscience-workshop/Megatron-DeepSpeed
Pull Requests
Commits
thomas/fix_layer_norm
LS/alibi
LS/doc
Lucile/add-eval-only-arg
Lucile/delete_unnecessary_brackets
Lucile/useless-parenthesis
add-valid-data
bitfit
bloom-ds-inference-repos2
bloom-inference-meta
bnb-resume-2x
bseval_harness
cc-concurrency
chpt-conversion-fix
ckptavg
cluster_benchmark
consumed_samples_per_valid_dataset
cyclic_valid_dataloaders
debug_with_new_dataset
dependabot/pip/black-24.3.0
ds_ckpt_reshape-with-layer-norm-auto-sync
ds-version-check
fix-sample-ids
fp32-checkpoint-extraction
gpu-direct
hadyelsahar/main
launch-debug
license
log-grad-norm
lumi_eval
lumi_mtf
main
master
megatron-2.4-ds-pipe
mtf_p3
mtf-multival
new-dataset
no-shuffling-option
nozero_reshape
olruwase/ds_ckpt_reshape
olruwase/sync_layer_norms
prefixbseval
preprocess_from_HF_dataset
rm-duplicate-param-count
samson/spm
scratchpad
self_attention_stable_corby
skip-broken-tests
sync4
t0loading
test-conversion
thomas/add_shared_t5
thomas/evaluate_gpt_on_prefix_lm_loss
thomas/evaluate_gpt_speed_if_we_pass_attention_mask
thomas/fix_installation
thomas/fix_layer_norm
thomas/improve_test_to_test_custom_kernel
thomas/mlm_train_script
thomas/opt
thomas/test_different_layer_norm
tp-ln-debug
tr1-13B
tr8-104B
train-no-eval-restart
training_flos_rebase
training_flos
universal_ckpt_info
universal_to_fp32_checkpoint
val_args
Test out the loss from the fp32 weights and optimizer states
thomasw21
committed
3 years ago
189f0547
I'm surprised one can apply inplace operation here
thomasw21
committed
3 years ago
c3844b5c
Update test
thomasw21
committed
3 years ago
007ecb4b
Fix non matching torch_rng_state for tp_rank=0
thomasw21
committed
3 years ago
2c49216a
Test that torch_rng_state
thomasw21
committed
3 years ago
7ebbed16
Test for final weight
thomasw21
committed
3 years ago
1fc4fe82
Figure out why state differs
thomasw21
committed
3 years ago
e0ef1683
New list of matching embeddings
thomasw21
committed
3 years ago
42997b2a
Row column bias should be synchronized as well
thomasw21
committed
3 years ago
89ab0b72
test
thomasw21
committed
3 years ago
ce017338
Compare model_states_rank
thomasw21
committed
3 years ago
0b7afcc9
Now that we have a force sync mechanism, let's try to reproduce
thomasw21
committed
3 years ago
64a62c80
Nit
thomasw21
committed
3 years ago
77abee61
Test that force sync will allow TP changes
thomasw21
committed
3 years ago
09cead38
Fix checkpoint path
thomasw21
committed
3 years ago
e9431002
Woops
thomasw21
committed
3 years ago
04e89d14
Fix load issue
thomasw21
committed
3 years ago
4574ec97
Woops
thomasw21
committed
3 years ago
7cdb1be8
Woops
thomasw21
committed
3 years ago
cc7ff45b
Woops
thomasw21
committed
3 years ago
5fb231c1
Woops
thomasw21
committed
3 years ago
d357839d
Woops
thomasw21
committed
3 years ago
2ab8a3ac
Woops
thomasw21
committed
3 years ago
55e10c63
Woops
thomasw21
committed
3 years ago
05c99db6
Woops
thomasw21
committed
3 years ago
3bc58243
Woops
thomasw21
committed
3 years ago
4b7207b5
Make a test that TP reshaping works
thomasw21
committed
3 years ago
b418b47a
all_reduce is an in_place operation
thomasw21
committed
3 years ago
8271d419
run just the test we want for now
stas00
committed
3 years ago
3ec65f7c
reproduce the crashing on resume
stas00
committed
3 years ago
37795a92
Older