[Model] Deepseek GGUF support #13167
vllm-bot
merged
23 commits into
vllm-project:main
from
SzymonOzog:gguf-deepseek
huang-junhong
approved these changes
on 2025-02-18
Isotr0py
commented
on 2025-02-20
WIP gguf MoE
d528c245
Manual mapping of deepseek layers
625b6589
Proof of concept GGUF MoE
c455901b
MoE weight initialization for GGUF
a18d44ba
Expert weight initialization for GGUF
8374b373
Add option to override hf config
011fbfb4
Cleanup
0f1c8ee3
Fix gguf weight loading for replicated linear
ec4de414
Make hf_config_path optional
1ff2fd59
fix inheritance
ea9eb0eb
lint
58022e93
add hf config path to parser
ee1dc1f8
loading all experts at once
8a7e61fa
Accept suggestion
da3dff61
Accept suggestion
ae19a8a2
Accept suggestion
bb4af454
Accept suggestion
b92793e0
Incorporate feedback from code review + add assertions and config
0ea9252d
Isotr0py
commented
on 2025-02-24
merge from main
36f73b59
missing parameters
7caa1f74
Isotr0py
commented
on 2025-02-25
Isotr0py
approved these changes
on 2025-02-25
remove TP assettion
a264af34
Fix supported dtypes
28baf157
cjackal
commented
on 2025-02-26
disabled auto-merge 1 year ago
Manually disabled by user
Manually disabled by user
feedback: add activation parameter
e5473941
Reviewers
Assignees
No one assigned
Labels
documentation
structured-output
frontend
speculative-decoding
ready
ci/build
v1
Milestone
No milestone
Login to write a write a comment.
Login via GitHub