bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

Set iteration to args by default

Muennighoff committed 3 years ago

477cda6d

Add multiple evaluation compat

Muennighoff committed 3 years ago

6c1018f6

Merge branch 't0loading' into lossseq

Muennighoff committed 3 years ago

456327c1

Muennighoff committed 3 years ago

d9a91feb

Reshape loss mask

Muennighoff committed 3 years ago

549f4993

Move norm to batch pipe

Muennighoff committed 3 years ago

a6b26240

Loss mask is already float

Muennighoff committed 3 years ago

2e7554d7

Clarify loss on targets & remove kwarg

Muennighoff committed 3 years ago

fce1a98e

Add reset-progress key

Muennighoff committed 3 years ago

26997216

Merge branch 'main' into t0loading

Muennighoff committed 3 years ago

0a324592

Add norm_target_loss arg

Muennighoff committed 3 years ago

7bc1dd20

BLOOM Inference via DeepSpeed-Inference, Accelerate and DeepSpeed-ZeRO (#308)

stas00 committed 3 years ago

Verified 3932c749

Simplify division

Muennighoff committed 3 years ago

900c8356

Muennighoff committed 3 years ago

616cfe86

Efficient loss normalization

Muennighoff committed 3 years ago

992446c8

Muennighoff committed 3 years ago

462efd99

Muennighoff committed 3 years ago

dc8d0abb

Reshape deepspeed checkpoint (#239)

tjruwase committed 3 years ago

Verified 0f23a729

Add prefixlm arg

Muennighoff committed 3 years ago

b15ca2d5

Allow not using torch distributed

Muennighoff committed 3 years ago

b62dcafc

Avoid loading module when not loading optim

Muennighoff committed 3 years ago

cb0313ba

Remove helper scripts

Muennighoff committed 3 years ago

ca740f1e

Remove unnec imports

Muennighoff committed 3 years ago

2dfe5d11

JSON helper scripts

Muennighoff committed 3 years ago

a55d2fb5

Muennighoff committed 3 years ago

fb8ecb8c

Add helpers & set is_causal to true

Muennighoff committed 3 years ago

89460c0a

Update tools/preprocess_data.py

Muennighoff committed 3 years ago

Verified 63daa46f

Add prepend-space arg

Muennighoff committed 3 years ago

0fcb19c1

Swap decoder_is_inputs & segment_ids

Muennighoff committed 3 years ago

abdd7030

Enable loading ckpt for t0 finetuning

Muennighoff committed 3 years ago

90b8f46d

Older