llama.cpp
eaf5c319 - deepseek41 : gather multiple tokens per stream

Commit
3 hours ago
deepseek41 : gather multiple tokens per stream Generalize the gather path to up to 32 tokens per stream instead of only one token per stream or a single stream. Select the compressed rows per stream and repeat the window per query within each stream. Ref: https://github.com/ggml-org/llama.cpp/pull/32519#issuecomment-7037816723 Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
Author
Parents
Loading