bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

addressed jareds comments

shoeybi committed 5 years ago

197c132e

added fp16 lm cross entropy to bert

shoeybi committed 5 years ago

78022005

added fp16 cross entropy loss option for gpt2

shoeybi committed 5 years ago

22e3c7e6

added fp16 cross entropy loss option for gpt2

shoeybi committed 5 years ago

acfe848e

Some changes noticed late

Neel Kant committed 5 years ago

c044f59a

Prune changes to only be related to ICT

Neel Kant committed 5 years ago

32bb4edc

Merge branch 'realm-mlm' of https://gitlab-master.nvidia.com/ADLR/megatron-lm into realm-mlm

Neel Kant committed 5 years ago

91158c9b

runs with new log loss but plateaus early

Neel Kant committed 5 years ago

20895f2c

shoeybi committed 5 years ago

2ede8235

Merge branch 'master' into realm-mlm

Neel Kant committed 5 years ago

674814a5

Temporarily modify ict_eval_bm25 to build full wiki index

Neel Kant committed 5 years ago

8b355d93

Neel Kant committed 5 years ago

51204a4d

Add bm25 evaluation code

Neel Kant committed 5 years ago

8409e1c1

Correct indexer seq length

Neel Kant committed 5 years ago

dfb907fe

Merge branch 'realm-mlm' of https://gitlab-master.nvidia.com/ADLR/megatron-lm into realm-mlm

Neel Kant committed 5 years ago

ab5c4f92

Correct CrossEntropyLoss

Neel Kant committed 5 years ago

0e8f4331

Corrected realm example building, misc improvements for async concurrency

Neel Kant committed 5 years ago

2fd4ea6c

Fix token alignment, add mpu checkpointing, misc training code

Neel Kant committed 5 years ago

8e22824e

Use Faiss GPU index and report retrieval utility

Neel Kant committed 5 years ago

8573ab35

Restructure Indexer classes

Neel Kant committed 5 years ago

e59496bf

Fix autoresume by removing code from indexer

Neel Kant committed 5 years ago

e0a1caba

Change sync variable to gloo backend

Neel Kant committed 5 years ago

05ea0cca

Async works for total 8 GPU, indexer debug mode

Neel Kant committed 5 years ago

a670b6c9

Full cycle of communication complete. Also added BasicIndexBuilder

Neel Kant committed 5 years ago

5684f904

Reorganize indexer. Things run up to saving model checkpoint and repeating

Neel Kant committed 5 years ago

d4b00be0

Indexer_async works in theory

Neel Kant committed 5 years ago

e338e311

Add indexer_async

Neel Kant committed 5 years ago

edaf2aab

Able to run REALM with terrible index sync

Neel Kant committed 5 years ago

0f0f60aa

Merge branch 'pip_support' into 'master'

shoeybi committed 5 years ago

5897a790

ekmb committed 5 years ago

faa462fa

Newer Older