ggerganov/llama.cpp

Pull Requests Commits

metal : rewrite to fit new backend interface correctly (WIP)

ggerganov committed 2 years ago

Verified d45c1631

metal : first working version of the inference without prompt processing

ggerganov committed 2 years ago

Verified cb82adad

metal : map the CPU buffers to Metal buffers (WIP)

ggerganov committed 2 years ago

Verified 290cb700

Merge remote-tracking branch 'origin/ggml-backends' into ggml-backends-metal

ggerganov committed 2 years ago

Verified f38433ef

metal : create backend, mostly reuse CPU backend interface

ggerganov committed 2 years ago

Verified 70c55c17

slaren committed 2 years ago

295f8565

llama : separate compute buffer for metal

ggerganov committed 2 years ago

Verified ed960fa1

ggml : add is_ram_shared to ggml_backend

ggerganov committed 2 years ago

Verified 652c8496

llama : init metal backend as CPU backend for now

ggerganov committed 2 years ago

Verified 90503f15

metal : adapting to ggml_backend (WIP)

ggerganov committed 2 years ago

Verified 0a3861c4

fix double-free with --no-mmap

slaren committed 2 years ago

1102ff56

improve layer backend printing with ranges

slaren committed 2 years ago

4e94af30

only allocate as much memory as is required in each backend for the model

slaren committed 2 years ago

c2beeb8e

rebase to master (except ggml-cuda)

slaren committed 2 years ago

9c72e7e9

fix NVCC version on Makefile, __halves2half2 -> make_half2

slaren committed 2 years ago

33ab185d

slaren committed 2 years ago

24cc6f00

restore simple.cpp for now

slaren committed 2 years ago

5765d7a5

ggml backend interface wip

slaren committed 2 years ago

0d2b66c6

llama : add custom RoPE (#2054)

jxy committed 2 years ago

Verified 6e7cca40

flake : add runHook preInstall/postInstall to installPhase so hooks function (#2224)

ddellacosta committed 2 years ago

Verified a6803cab

make : use pkg-config for OpenBLAS (#2222)

Freed-Wu committed 2 years ago

Verified 7dabc66f

cuda : allocate all temporary ggml_tensor_extra_gpu from a fixed-size buffer (#2220)

bullno1 committed 2 years ago

Verified 7cdd30bf

ggml : fix static_assert with older compilers #2024 (#2218)

evanmiller committed 2 years ago

Verified e8035f14

llama : add functions that work directly on model (#2197)

bullno1 committed 2 years ago

Verified 7513b7b0

build.zig : install config header (#2216)

alichraghi committed 2 years ago

Verified de834242

examples : fixed path typos in embd-input (#2214)

xushangning committed 2 years ago

Verified c48c525f

cuda : support broadcast add & mul (#2192)

li-plus committed 2 years ago

Verified 206e01de

CUDA: mul_mat_vec_q kernels for k-quants (#2203)

JohannesGaessler committed 2 years ago

Verified 4304bd3c

make : fix combination of LLAMA_METAL and LLAMA_MPI (#2208)

magnusviri committed 2 years ago

Verified 229aab35

ggml : sync (ggml_conv_2d, fix mul_mat bug, CUDA GLM rope)

ggerganov committed 2 years ago

Verified 69796668

Older