bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

WIP: conversion script:

thomasw21 committed 3 years ago

e73aa58e

Add TODO for loading previous checkpoint

thomasw21 committed 3 years ago

f557523e

Removing erroneous trailing layers

thomasw21 committed 3 years ago

216a3f5f

thomasw21 committed 3 years ago

8578ab36

More dirty fixes

thomasw21 committed 3 years ago

fc0560ac

thomasw21 committed 3 years ago

26d9dce1

Lambda function can't unpack tuples

thomasw21 committed 3 years ago

b25803bf

Flatten to pass tuples of tensor instead

thomasw21 committed 3 years ago

52c42b25

I think GPT2 tokenizer might be missing <sep> token

thomasw21 committed 3 years ago

12626f61

I think GPT2 tokenizer might be missing <sep> token

thomasw21 committed 3 years ago

07f0ce57

Fix MLM datasets length

thomasw21 committed 3 years ago

9da75889

Hopefully this fixes MLM

thomasw21 committed 3 years ago

c71b3799

Hopefully this fixes MLM

thomasw21 committed 3 years ago

ef205ddf

add --vocab-extra-ids 100

thomasw21 committed 3 years ago

c93b5639

SEP is only defined for HFTokenizer

thomasw21 committed 3 years ago

ed3dbeac

Maybe this is better

thomasw21 committed 3 years ago

90ee34c6

Maybe this is better

thomasw21 committed 3 years ago

ffbe3bf5

DS has poor default

thomasw21 committed 3 years ago

04cdc2e0

Hack my way into fix attn_mask

thomasw21 committed 3 years ago

dcb2d610

thomasw21 committed 3 years ago

50cf6d9c

thomasw21 committed 3 years ago

b5568fba

Shared t5 tests

thomasw21 committed 3 years ago

e55b6031

thomasw21 committed 3 years ago

91a20aa6

thomasw21 committed 3 years ago

fbc1d040

thomasw21 committed 3 years ago

cc6dd1d3

Fixed MLM dataset arguments(#290)

thomasw21 committed 3 years ago

Verified 55f8cf8b

Mlm adaptation (#287)

Lintang Sutawika committed 3 years ago

Verified 9d264312

Fix DS init (#285)

Quentin-Anthony committed 3 years ago

Verified 987663c1

Fix tflops glu computation (#283)

Muennighoff committed 3 years ago

Verified e23393fb

[valid] deadlock workaround (#282)

stas00 committed 3 years ago

Verified cb48bd2c

Older