PR #1492 enable/disable moe token dropping.

enable/disable moe token dropping. #1492

conglongli merged 3 commits into master from moe+cl

Add a flag to enable/disable token dropping in moe/top-1 gating.

2d1ce861

fix syntax and formatting.

75175707

awan-10 requested a review from

cli99 4 years ago

awan-10 requested a review from

conglongli 4 years ago

awan-10 requested a review from

eltonzheng 4 years ago

awan-10 requested a review from

jeffra 4 years ago

awan-10 requested a review from

minjiaz 4 years ago

awan-10 requested a review from

niumanar 4 years ago

awan-10 requested a review from

RezaYazdaniAminabadi 4 years ago

awan-10 requested a review from

samyam 4 years ago

awan-10 requested a review from

ShadenSmith 4 years ago

awan-10 requested a review from

tjruwase 4 years ago

conglongli approved these changes on 2021-10-28

conglongli enabled auto-merge (squash) 4 years ago

add the missing drop_tokens entry.

603db144

conglongli merged 56635d5b into master 4 years ago

mrwyattii deleted the moe+cl branch 2 years ago

Reviewers

conglongli

cli99

eltonzheng

jeffra

minjiaz

niumanar

RezaYazdaniAminabadi

samyam

ShadenSmith

tjruwase

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed enable/disable moe token dropping. #1492 Merged

enable/disable moe token dropping. #1492

DeepSpeed
enable/disable moe token dropping.
#1492

Merged