vllm
e22ee1e7 - [Kernel] GGUF MoE kernel (#14613)

Commit

106 days ago

[Kernel] GGUF MoE kernel (#14613) Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>

References

#14613 - [Kernel] GGUF MoE kernel

Author

SzymonOzog

SzymonOzog

Parents

Files8

csrc
- ops.h
- quantization/gguf
  - gguf_kernel.cu
  - moe.cuh
- torch_bindings.cpp
tests/kernels
- test_ggml.py
- test_gguf.py
vllm
- _custom_ops.py
- model_executor/layers/quantization
  - gguf.py