Megatron-DeepSpeed
BLOOM Inference via DeepSpeed-Inference, Accelerate and DeepSpeed-ZeRO
#308
Merged
Overview
Commits
58
Changes
View On GitHub

BLOOM Inference via DeepSpeed-Inference, Accelerate and DeepSpeed-ZeRO #308

stas00 merged 58 commits into main from bloom-inference
stas00
stas00 hardcode the dtype depending on the model
efa354dd
stas00 stas00 changed the title hardcode the dtype depending on the model BLOOM Inference via DeepSpeed-Inference 3 years ago
stas00
RezaYazdaniAminabadi
RezaYazdaniAminabadi
RezaYazdaniAminabadi
RezaYazdaniAminabadi
change the mp based on the world_size
cafc3f5d
RezaYazdaniAminabadi
RezaYazdaniAminabadi
stas00
stas00 remove hardcoded world_size
daeb293b
stas00 add bigscience/bigscience-small-testing
7d5f7d46
Merge branch 'bloom-inference' of https://github.com/bigscience-works…
2d3d271b
RezaYazdaniAminabadi
stas00
stas00 fixes
1ff0f698
stas00 add zero-inference script
56b24ed3
stas00 fixes
67aab37c
stas00 fix
328ab0cc
stas00 working script
f2628b03
stas00 renames
195288e5
stas00 fixes
3c7b2cb6
stas00 fix for offline use
6c5c23ba
stas00
stas00 add benchmark
6b192274
stas00 add benchmark
10cbb2d4
stas00 update
494c212e
stas00 cleanup
2b67c0d9
stas00
stas00 update
3853724e
stas00 msecs
18967399
stas00 cleanup
7c9daaf1
stas00 improve
dca2c8f7
pommedeterresautee
stas00
stas00 fix benchmark, add warmup
85580c0b
stas00 update
5ea3dee9
stas00 fix; thanks Michael Wyatt
737c6816
stas00 clarify
6be0cca4
Merge branch 'bloom-inference' of https://github.com/bigscience-works…
fea3902a
add bloom batch-inference script
fc9b458a
removed the names :-)
7b0edef2
stas00 fold the bs functionality from the other script
2120dd2a
stas00 fix
78bcbb7e
stas00 restore do_sample
e7468cd2
stas00 dump generate args
68f5ca6a
stas00 fix
1eca7c50
stas00 fix
8815fc3d
stas00 support any batchsize
034cc6fe
stas00 div by bs
155c3c32
stas00 mul by bs
73a8b7b7
stas00 add cpu_offload; sync scripts
09d74088
stas00 wip
695265d4
thomasw21
thomasw21 commented on 2022-07-14
stas00 improvements
1a7e891b
stas00 fixes
aba4055a
stas00 fixes
5e92d552
stas00 add accelerate script
39921122
stas00 fix
5a7057b0
stas00 stas00 changed the title BLOOM Inference via DeepSpeed-Inference BLOOM Inference via DeepSpeed-Inference, Accelerate and DeepSpeed-ZeRO 3 years ago
stas00 wip
47585312
stas00 wip
7550ee06
stas00 stats
5153c402
jeffra add OnDevice and remove zero-inference (#316)
cb50ea59
stas00 wip
a53fcaa5
stas00 rework generate + benchmark
72528797
stas00 figure out the memory map dynamically
2aa419d4
stas00 bug fix
4bd8ca5b
tjruwase
tjruwase commented on 2022-07-19
stas00 fix ds-zero-inference wrt device
b76e5166
stas00 bug fix
ecfd5771
stas00 update
fd26b9c4
mayank31398
mayank31398 commented on 2022-07-21
stas00
stas00
stas00 update
e2bfe916
mayank31398
stas00
mayank31398
mayank31398
stas00
mayank31398
mayank31398 commented on 2022-07-23
mayank31398
stas00
xuyifan-0731
mayank31398
mayank31398
RezaYazdaniAminabadi
mayank31398
stas00
RezaYazdaniAminabadi
mayank31398
mayank31398
stas00
mayank31398
mayank31398 approved these changes on 2022-07-29
mayank31398
stas00
mayank31398
mayank31398
stas00
stas00
mayank31398
mayank31398
mayank31398
stas00 fix
b9a67ea5
stas00 Merge remote-tracking branch 'origin/main' into bloom-inference
3862ef0f
mayank31398
mayank31398
stas00
stas00
stas00 stas00 merged 3932c749 into main 3 years ago
stas00 stas00 deleted the bloom-inference branch 3 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum