ggml : implement REGLU/GEGLU/SWIGLU ops #14158
CISC
merged
23 commits into
master
from
cisc/unary-reglu-geglu-swiglu
ggerganov
commented
on 2025-06-13
CISC
changed the title
ggerganov
approved these changes
on 2025-06-13
ngxson
commented
on 2025-06-13
JohannesGaessler
commented
on 2025-06-13
ngxson
commented
on 2025-06-13
sgeor255
approved these changes
on 2025-06-17
CISC
changed the title
implement unary REGLU/GEGLU/SWIGLU cpu ops
76c9bc17
relax constraints
5a490f07
duplicate shape of source
56c79931
fix ggml_vec_geglu_f16
21c4963b
special case gated ops
bb2fda70
implement unary REGLU/GEGLU/SWIGLU cuda ops
a1a7b6df
tighten constraints again
f8c20809
refactor into GGML_GLU_OP
a341aa3c
metal : add glu kernels
d9ddeb9d
add CUDA_GLU_BLOCK_SIZE [no ci]
cfa9c7a4
more constraints and use 64bit ints
70e8b48e
64bit multiplication [no ci]
f8705a23
implement swapped variants (cpu/cuda)
0b2703fc
update comment [no ci]
d5934297
Vulkan: Add GLU ops and shaders
34d1aeda
SYCL: Implement fused kernel GEGLU, SWIGLU and REGLU for single up+gate
a9aedf46
ggml : implement GLU for split up/gate (#14181)
35dacd1a
GGML: increase OP count in assertion
a234e09f
Refactor: Optimize SYCL element-wise operations with unary function i…
ab46d11d
vulkan: Increase workgroup size for GLU, for performance (#14345)
650d3987
merge fix
8bca3e2a
CISC
changed the title
Merge branch 'master' into cisc/unary-reglu-geglu-swiglu
a6b641e0
ggerganov
commented
on 2025-06-29
metal : add support for split and swap
9054f1a0
ggerganov
approved these changes
on 2025-06-29
Reviewers
Assignees
No one assigned
Labels
help wanted
testing
Nvidia GPU
Vulkan
ggml
SYCL
Apple Metal
Milestone
No milestone
Login to write a write a comment.
Login via GitHub