ggerganov/llama.cpp

Pull Requests Commits
wip
ggerganov committed 1 year ago
Verified 49a483e0
cuda : minor
ggerganov committed 1 year ago
Verified 1846e92a
cuda : fix matrix names
ggerganov committed 1 year ago
Verified ef68fac2
cuda : simplify softmax
ggerganov committed 1 year ago
Verified cfd9732b
cuda : fix -INF block check
ggerganov committed 1 year ago
Verified e04ff391
cuda : unroll Q*K^T loop
ggerganov committed 1 year ago
Verified 5b263dd8
cuda : speed-up reduce part of the kernel
ggerganov committed 1 year ago
Verified 3b1c4e76
cuda : switch to 1 warp for bs > 16
ggerganov committed 1 year ago
Verified a7b47156
cuda : use half2 in softmax
ggerganov committed 1 year ago
Verified b958151e
cuda : avoid __hisinf branches
ggerganov committed 1 year ago
Verified c51f27c0
cuda : unroll some of the loops
ggerganov committed 1 year ago
Verified 92472ea2
cuda : make loops use the same loop values
ggerganov committed 1 year ago
Verified 1f8a5924
cuda : use int instead of int64_t
ggerganov committed 1 year ago
Verified 7c34655b
cuda : avoid warp_reduce for smax
ggerganov committed 1 year ago
Verified b150abe8
cuda : fix __hisinf() result check
ggerganov committed 1 year ago
Verified b68a1122
tests : update dims
ggerganov committed 1 year ago
Verified 12eaa226
cuda : avoid zeroing fragments
ggerganov committed 1 year ago
Verified db1f3c48
tests : minor fix
ggerganov committed 1 year ago
Verified c6769b94
metal : optimize softmax
ggerganov committed 1 year ago
Verified cda5a60a
metal : optimize softmax for C > 32
ggerganov committed 1 year ago
Verified 56e45a23
Merge branch 'master' into gg/flash-attn
ggerganov committed 1 year ago
Verified 41d136b6
cuda : add flash_attn kernel (wip)
ggerganov committed 1 year ago
Verified 5a19a9f6
cuda : fix LLAMA_CUDA_F16 (#5262)
slaren committed 1 year ago
Verified 8ca511ca
make : generate .a library for static linking (#5205)
nehzata committed 1 year ago
Verified d71ac909
cuda : fix soft_max to use correct mask size
ggerganov committed 1 year ago
Verified 2e460137
ggml : fix ggml_soft_max mask requirement
ggerganov committed 1 year ago
Verified 910b15bb
llama : support InternLM2 (#5184)
SolenoidWGT committed 1 year ago
Verified ce320601
Fix broken Vulkan Cmake (properly) (#5230)
netrunnereve committed 1 year ago
Verified 1cfb5372
ggml : switch to padded F16 mask for ggml_soft_max, ggml_flash_attn_ext
ggerganov committed 1 year ago
Verified 8ad92dc1
Merge branch 'master' into gg/flash-attn
ggerganov committed 1 year ago
Verified 2ddc9bbe
Terms Of Use Privacy Policy Refund Policy Impressum