sync : ggml #3071
ggerganov
merged
80 commits into
master
from
sync-ggml-25-04-24
CUDA: don't convert BF16 weights to FP32 (ggml/1174)
b4b7e155
ggml : simplify Arm fp16 CPU logic (ggml/1177)
53506afa
llama : add option to override model tensor buffers (llama/11397)
614c087b
Vulkan: Fix mmq int dot float cache size (llama/12722)
8a6981a7
vulkan: Implement grouped query attention in the coopmat2 FA shader (…
ab9d6087
cmake: remove caching from vulkan coopmat checks (llama/12719)
35027550
vulkan: Implement split_k for coopmat2 flash attention. (llama/12627)
2f935a82
opencl: use `max_alloc_size` in backend ctx instead of querying again…
953a9033
CANN: Fix failed test cases (llama/12708)
ec21d02b
Simplify and improve CUDA graphs through use of indirect copy pointer…
2d0b11bc
CANN: Support operator SIN COS ARGMAX (llama/12709)
ade66e63
fix MUSA compiler warning (llama/12704)
d52ab9b8
vulkan: Fix missing cmake logic for dot product extension (llama/12721)
96a8cc5a
CUDA: Prefer vector flash decoding kernel for Gemma models (llama/12738)
f88fe40f
vulkan: set cmake minimum and project name in vulkan-shaders (llama/1…
4773f07a
vulkan: Hybrid waitForFences/getFenceStatus to reduce fence latency (…
bceee711
cmake: fix ggml-shaders-gen compiler paths containing spaces (llama/1…
7cff350e
sycl: allow ggml-sycl configuration and compilation using Visual Stud…
cb85299c
Vulkan: Tune Vulkan mmq int dot shader for performance (llama/12767)
c5b7ca94
vulkan: Use unclamped loads for flash attention mask (llama/12720)
1426dfdc
vulkan: fix NaN issue in flash attention shader (llama/12776)
bf9d152e
musa: fix compilation warnings in mp_22/31 (llama/12780)
26531963
CANN: Refactor to reduce duplicate code (llama/12731)
1751e806
CANN: fix typo in ggml-cann (llama/12733)
9dcd7ff8
sycl: remove redundant memcopy in function ggml_backend_sycl_buffer_s…
b1306b96
cuda : fix HIP and MUSA BF16 (llama/0)
981432f5
opencl: better identify Adreno GPU (llama/12760)
db561f20
Revert "sycl:remove redundant memcopy in function ggml_backend_sycl_b…
72ff1965
ggml : add more generic custom op, remove deprecated custom ops (ggml…
75749182
ggml : add bilinear upscale support (ggml/1185)
3c4f20e3
ggml: don't include arm_neon.h when using CUDA 12 with ARM Neon (ggml…
8c36c7bd
llama : fix FA when KV cache is not used (i.e. embeddings) (llama/12825)
1f36f271
cuda : add f32 to bf16 copy op (llama/12806)
f96f36f5
vulkan: Use fp16 for the flash attention P*V multiplication (llama/12…
d25499b7
vulkan: In coopmat2 mmq, load q4_k/q5_k scales through shared memory …
0f556ed9
CANN: Support Opt CONV_TRANSPOSE_1D and ELU (llama/12786)
147541f0
ggml-impl.h: fix build on POWER9 (llama/12855)
55e48dfb
ggml-cpu-impl.h: do not redefine bool on POWER9 (llama/12856)
27af59e3
Fixes #12823 (llama/12830)
2b846dd0
CANN: Support more ops (llama/12841)
38a99017
cpu: fix cpu backend's supports-op for GET_ROWS_BACK. fixes a fatal w…
19575cbf
ggml: fix compilation error s390x (llama/12848)
9f66d675
SYCL: Add fp16 type support to unary op kernels (llama/12788)
4eb52944
sycl: Support sycl_ext_oneapi_limited_graph (llama/12873)
963470fe
vulkan: use aligned loads for flash attention mask (llama/12853)
e8fe2868
ggml: disable CUDA graphs for unsupported DUP and CONT node types (ll…
9765632c
ggml: use _mm[512/256]_dpbusd[_avx]_epi32 to directly accumulate into…
31b3ada8
ggml : Depthwise 2D convolution (ggml/1152)
319af9f0
rpc : use ggml_context_ptr (llama/12938)
00cb979a
SYCL: Fix im2col (llama/12910)
d3713ee0
CANN: Optimize CANN buffer pool memory management (llama/12875)
378c551d
CANN: Opt ROPE optimization (llama/12865)
79d78426
ggml : Add AVX512 implementation of GEMM - Q4_Kx8 (llama/12829)
b3449573
SYCL: Add ROPE vision kernel (llama/12887)
6f2a40f7
CUDA/HIP: Share the same unified memory allocation logic. (llama/12934)
752162ba
CANN: Add x86 build ci (llama/12950)
6a2273b3
metal : add FA-vec kernels for head size 96 (llama/12952)
342a49fe
CANN: Add 310P operator support check (llama/12962)
505e57d1
vulkan: enable coopmat2 FA gqa and split_k optimizations more often (…
062375ee
opencl: fix incorrect local_size index in profiling log (llama/12868)
fa69e39f
CANN: Add support for async operator submission (llama/12864)
c941896d
ggml: Re-enable CUDA graphs in presence of CONT and DUP nodes (llama/…
9a2ac85d
graph : make FA compatible with MLA + add initial Metal kernels (llam…
056d903b
rpc : add RPC_CMD_HELLO (llama/12955)
4a7775c7
SYCL: Refactor and enable FP16 in binary broadcast OPs (llama/12975)
ed18c7a9
metal: add neg operator (llama/13029)
f7eb6eb6
vulkan: support noncontiguous rms_norm (llama/13031)
308239db
SYCL: Add non-contiguous support in ROPE (llama/12993)
0612ed67
ggml : add SSE 4.2 and x64 base variant for CPUs without AVX (llama/1…
6d63a738
CUDA: noncont MMVQ + batched bs1 MUL_MAT_ID (llama/13014)
6e92d2ee
vulkan: matmul gcn tuning (llama/13016)
2299308d
metal : fix floating-point range of attention scores in FA kernels (l…
3bd21ed2
CUDA: use switch statements in constexpr functions (llama/13095)
de20f455
ggml : fix trailing whitespaces (llama/0)
fd137aac
opencl: split ggml-opencl.cl into multiple files and cleanup (llama/1…
6f9d3e6b
sync : ggml
775402bb
opencl : remove obsolete files (skip) (ggml/1200)
fc70647d
sync : ggml
00fb6606
cuda : fix unused variable compile warning (#0)
4ae6aa3f
ruby : add cmake option (#0)
475e93eb
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub