DeepSpeed
Mixtral FastGen Support
#4828
Merged
Overview
Commits
13
Changes
View On GitHub

Mixtral FastGen Support #4828

mrwyattii merged 13 commits into master from cholmes/mixtral-fastgen-support
cmikeh2
cmikeh2 Kernel changes, cleanup still necessary
72212feb
cmikeh2 Add explicit theta to rotary embeddings
62fdf964
cmikeh2 Mixtral model implementation
20823e5b
cmikeh2 Commit the unsaved files
0e7d4e0d
cmikeh2 Minor fixes
6b1761a0
cmikeh2 Missing engine factor, rope theta fixes
28ab6441
cmikeh2 MoE type mismatches
d28c327b
cmikeh2 Misnamed mapping
c1e90a37
cmikeh2 Clear output
880417e8
cmikeh2 cmikeh2 requested a review from mrwyattii mrwyattii 2 years ago
cmikeh2 cmikeh2 requested a review from awan-10 awan-10 2 years ago
cmikeh2 cmikeh2 requested a review from arashb arashb 2 years ago
cmikeh2 cmikeh2 requested a review from tjruwase tjruwase 2 years ago
RezaYazdaniAminabadi
RezaYazdaniAminabadi
RezaYazdaniAminabadi commented on 2023-12-18
mrwyattii
mrwyattii approved these changes on 2023-12-19
cmikeh2 Fix unit test
77b871d5
cmikeh2 cmikeh2 requested a review from loadams loadams 2 years ago
cmikeh2 Fix unit tests
a383b67e
cmikeh2 Clean up top_k support in the C++ code
0255d6bd
mrwyattii Merge branch 'master' into cholmes/mixtral-fastgen-support
e67e15af
mrwyattii mrwyattii merged c00388a2 into master 2 years ago
mrwyattii mrwyattii deleted the cholmes/mixtral-fastgen-support branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum