whisper : Metal and ggml-alloc support #1270
ggerganov
merged
44 commits into
master
from
metal-and-alloc
metal : init
fbc3f803
whisper : factor out graph builds
949ab632
whisper : allocate encoder and decoder using ggml-alloc
bed5ad69
whisper : ggml-alloc is now supported
af6f67b2
whisper : CoreML support ggml-alloc
fa672b46
ggerganov
commented
on 2023-09-10
build : fix ggml-alloc
794e8fe0
ios : update submodule
9a78b722
extra : update sync-ggml.sh script to also sync ggml-alloc
06d1d283
slaren
commented
on 2023-09-10
slaren
commented
on 2023-09-10
ci : see if this is causing the crash
4d9acc60
whisper : refactor ggml-alloc init
2770d46e
whisper.android : try to fix build
4845b9ed
whisper : initial Metal version
d3b2dd49
Merge branch 'master' into metal-and-alloc
de94c783
ci : try to debug vmem issue
3b9979a3
metal : decoder works on GPU!
fbc9ddc5
metal : add multi-decoder support
79a88057
ggml : fix ggml_nbytes (probably temp solution)
9fdd4153
metal : run "cross" step on the GPU
cd476375
ggerganov
commented
on 2023-09-12
whisper : remove ggml_repeat in the encoder
ec9a7db7
whisper : offload the Encoder to Metal
3074a7ff
ggml : use simpler ggml_bytes() implementation
905c9441
ggml-alloc : try to make CI happy by reducing vram to 128GB
b19888cf
whisper : add whisper_allocr to wrap ggml_allocr
254b6872
whisper : factor out alloc init in a function
b6f09669
cmake : update to support Metal build
77f4bf49
whisper : add <functional> header
796f84cd
objc : fix build (no Metal yet)
181bb8cb
ios : add Metal support
257d7942
swiftui : fix build
16db4da3
metal : speed-up KQ multiplication
8e8daa84
metal : sync latest llama.cpp kernels
ecb23fb1
readme : add Metal info
23277d21
ios : update submodule
d37f56e7
coreml : add code to toggle Core ML config (CPU, ANE, GPU)
d863f725
bench : fix timings by running a pre-heat
f408c645
bench : start benching the decoder
e81c67a1
whisper : add ggml_mul_mat_pad
af947cb7
bench : fix uninitialized vars
c46167f8
whisper : add comment for disabling mul-mat padding
f3655438
whisper : add description of ggml_mul_mat_pad
2b4160af
whisper : clean-up ggml_mul_mat_pad
0d5e4cdc
metal : remove the "concurrent" flag
bfcb2a2a
bench : variable n_past
a1664574
ios : update SPM package
3ac05580
Login to write a write a comment.
Login via GitHub