ggerganov/llama.cpp

Pull Requests Commits

llama : refactor tensor offloading as callback

ggerganov committed 2 years ago

Verified 15267192

llama : move refact in correct place + optimize graph input

ggerganov committed 2 years ago

Verified da936188

llama : try to fix build

ggerganov committed 2 years ago

Verified 739b85c9

llama : fix non-CUDA build

ggerganov committed 2 years ago

Verified 25cfbf67

llama : try to optimize offloading code

ggerganov committed 2 years ago

Verified b4ad03b3

llama : fix res_norm offloading

ggerganov committed 2 years ago

Verified 79617902

llama : do tensor offload only with CUDA

ggerganov committed 2 years ago

Verified e14aa461

llama : factor graph input into a function

ggerganov committed 2 years ago

Verified 0dc05b84

llama : support offloading result_norm + comments

ggerganov committed 2 years ago

Verified 4e98897e

llama : comments

ggerganov committed 2 years ago

Verified 51c4f9ee

llama : update offload log messages to print node index

ggerganov committed 2 years ago

Verified 3af87713

llama : offload rest of the models

ggerganov committed 2 years ago

Verified 83d2c437

llama : factor out tensor offloading outside the build call (wip)

ggerganov committed 2 years ago

Verified 38aca9e1

metal : disable kernel load log

ggerganov committed 2 years ago

Verified 5946d98f

llama : factor out ggml-alloc from graph graph build functions

ggerganov committed 2 years ago

Verified 8b2420d2

flake : update flake.lock for newer transformers version + provide extra dev shell (#3797)

Green-Sky committed 2 years ago

Verified ff3bad83

metal : try cwd for ggml-metal.metal if bundle lookup fails (#3793)

akx committed 2 years ago

Verified 82a6646e

issues : change label from bug to bug-unconfirmed (#3748)

ggerganov committed 2 years ago

Verified ba231e8a

convert : ignore tokens if their IDs are within [0, vocab_size) (#3831)

ggerganov committed 2 years ago

Verified 8a2f2fea

llama : allow quantizing k-quants to fall back when tensor size incompatible (#3747)

KerfuffleV2 committed 2 years ago

Verified bd6d9e20

llama : add option for greedy sampling with probs (#3813)

ggerganov committed 2 years ago

Verified ee1a0ec9

common : print that one line of the syntax help *also* to standard output (#3823)

HenkPoley committed 2 years ago

Verified 17746110

starcoder : add GPU offloading (#3827)

ggerganov committed 2 years ago

Verified fdee152e

speculative : ensure draft and target model vocab matches (#3812)

KerfuffleV2 committed 2 years ago

Verified 41aee4df

llama : correctly report GGUFv3 format (#3818)

cebtenzzre committed 2 years ago

Verified 6d459cbf

simple : fix batch handling (#3803)

tterrasson committed 2 years ago

Verified c8d6a1f3

cuda : improve text-generation and batched decoding performance (#3776)

ggerganov committed 2 years ago

Verified 2f9ec7e2

server : do not release slot on image input (#3798)

ggerganov committed 2 years ago

Verified 34b2a5e1

batched-bench : print params at start

ggerganov committed 2 years ago

Verified 6961c4bd

log : disable pid in log filenames

ggerganov committed 2 years ago

Verified cc448774

Older