bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

added dirty hack to have a maximum size

TevenLeScao committed 3 years ago

cb7648b0

Adding faster hack version

TevenLeScao committed 3 years ago

ea9051e0

Adding faster hack version

TevenLeScao committed 3 years ago

505e8cac

fixed confusion between row format and column format

TevenLeScao committed 3 years ago

c868bc6c

making sure iterator actually advances

TevenLeScao committed 3 years ago

f30f97cb

preprocessing from arrow file to load an HF dataset

TevenLeScao committed 3 years ago

20f582fe

Sorry, last change was meant to a PR. This reverts commit d0fcf4170def7205426117016d4622c745f33883.

TevenLeScao committed 3 years ago

497aa1bf

preprocessing from arrow file to load an HF dataset

TevenLeScao committed 3 years ago

d0fcf417

switch to MixedFusedLayerNorm (#262)

stas00 committed 3 years ago

Verified b9883f44

allocate embed norm only on pp0 (#261)

stas00 committed 3 years ago

Verified dd06ea32

sync the whole Meg-LM fused_kernels sub-tree (#260)

stas00 committed 3 years ago

Verified 1cb76a64

deploy elastic error handler (#258)

stas00 committed 3 years ago

Verified 8673d464

add `pad-vocab-size-to` argument and tests (#255)

SaulLu committed 3 years ago

Verified 58d92042

make partition_method configurable (#256)

stas00 committed 3 years ago

Verified 00f5f889

[sync] bf16 (#250)

stas00 committed 3 years ago

Verified 49d21af2

Try to read the data path arguments directly from a file (#254)

thomasw21 committed 3 years ago

Verified cf2d23a5

Use padded vocab size in preprocessing scripts (#253)

thomasw21 committed 3 years ago

Verified 99c8fe07

Fix preprocess_data_many_cores to use dtype

thomasw21 committed 3 years ago

b289eb81

Fix preprocess_data_many_cores to use dtype

thomasw21 committed 3 years ago

2c43a0fe

Replace approximate formula with exact one for throughput (#251)

deepakn94 committed 3 years ago

Verified ae6277f5

[apex FusedAdam] crash workaround (#249)

stas00 committed 3 years ago

Verified 541b967f

--abort-on-unmet-fused-kernel-constraints (#247)

stas00 committed 3 years ago

Verified 406022ed

implement kill switch (#245)

stas00 committed 3 years ago

Verified e9c8550e

Alibi Tensor Parallel Fix (#244)

DanielHesslow committed 3 years ago

Verified 40e8b2a0

[TB] log restarts (#234)

stas00 committed 3 years ago

Verified dfb5d687

Fix throughput unit (#241)

janEbert committed 3 years ago

Verified 4c2aea04

fix add_scalar for pt<1.9 (#240)

stas00 committed 3 years ago

Verified 5da3578a

enable new_style for add_scalar function for faster data format (#237)

abodacs committed 3 years ago

Verified e9750c42

[ds report] less noise (#215)

stas00 committed 3 years ago

Verified bbf311ee

fix tests (#232)

stas00 committed 4 years ago

Verified 04c461ed

Older