huggingface/text-generation-inference

Pull Requests Commits

Putting back header inclusion (seems unused but still)

Narsil committed 2 years ago

f555dabc

Simpler exllama

Narsil committed 2 years ago

5ca0508d

fix per-column quantization

fxmarty committed 2 years ago

6bf7090e

Merge branch 'main' into gptq-cuda-kernels

fxmarty committed 2 years ago

edfbfdfb

docs: Update README.md (#643)

Narsil committed 2 years ago

Verified 5a1512c0

docs: Update README.md (#639)

Narsil committed 2 years ago

Verified 1c81df15

feat(router): ngrok edge (#642)

OlivierDehaene committed 2 years ago

Verified b66b1904

feat(server): auto max_batch_total_tokens for flash att models (#630)

OlivierDehaene committed 2 years ago

Verified fe80f536

fix(server): fix llamav2 config (#635)

OlivierDehaene committed 2 years ago

Verified 5e6ddfd6

OlivierDehaene committed 2 years ago

Verified cf83f9b6

feat(server): add support for llamav2 (#633)

Narsil committed 2 years ago

Verified 211b211e

feat(server): flash attention v2 (#624)

OlivierDehaene committed 2 years ago

Verified 3b71c385

feat(server): Reworking the quantization script so it's still universal (not llama specific) (#587)

Narsil committed 2 years ago

Verified 4d38a1c4

fea(launcher): debug logs (#623)

OlivierDehaene committed 2 years ago

Verified 44acf72a

fix(launcher): Rename `b-float16` to `bfloat16` in the launcher arg (#621)

Narsil committed 2 years ago

Verified bc287324

fix(server): empty_cache when stopped

OlivierDehaene committed 2 years ago

a2cf1bdb

OlivierDehaene committed 2 years ago

Verified c58a0c18

fix(server): blacklist local files (#609)

OlivierDehaene committed 2 years ago

Verified 5b9de4a1

docs: README: Add logo + baseline (#611)

gary149 committed 2 years ago

Verified c8b077be

feat(router): explicit warning if revision is not set (#608)

OlivierDehaene committed 2 years ago

Verified 982ce322

fix the usual merge mess

fxmarty committed 2 years ago

74e6d6e5

Merge branch 'main' into gptq-cuda-kernels

fxmarty committed 2 years ago

9401e102

support all, test llama

fxmarty committed 2 years ago

00360842

feat(launcher): add arg validation and drop subprocess (#595)

OlivierDehaene committed 2 years ago

Verified b7327205

fxmarty committed 2 years ago

2ae65b45

fxmarty committed 2 years ago

38c2be59

GPTQ Env vars: catch correct type of error (#596)

ssmi153 committed 2 years ago

Verified 36285595

Merge branch 'main' into gptq-cuda-kernels

fxmarty committed 2 years ago

faa5b52f

fxmarty committed 2 years ago

8645fd39

support bits different than 4

fxmarty committed 2 years ago

f90c61a3

Older