huggingface/text-generation-inference

Pull Requests Commits

Narsil committed 2 years ago

7ccb8eef

Narsil committed 2 years ago

edc9ce9b

Narsil committed 2 years ago

8d42e1d1

Single place for TP layers + Dropout Layer Norm + FastLinear

Narsil committed 2 years ago

848c4aa4

Lifting check_unitialized. (#325)

Narsil committed 2 years ago

Verified 91e674bb

Hotfixes for santacoder/bigcode. (#294)

Narsil committed 2 years ago

Verified 73d84c6e

fix(docker): use ubuntu20.04

OlivierDehaene committed 2 years ago

22c4fd07

fix(docker): remove quantize default

OlivierDehaene committed 2 years ago

119f7e06

chore(docker): use nvidia base image (#318)

OlivierDehaene committed 2 years ago

Verified 8a8f4341

feat(server): GPTQ quantization (step1) (#277)

Narsil committed 2 years ago

Verified 76a48cd3

fix(server): fix multinomial implem in Sampling

OlivierDehaene committed 2 years ago

4f6d038c

feat(server): use cuda graph in logits warping (#302)

OlivierDehaene committed 2 years ago

Verified a6c18c39

fix(docker): remove CUDA_VERSION

OlivierDehaene committed 2 years ago

35ab6cfc

feat(server): use float16 (#304)

OlivierDehaene committed 2 years ago

Verified 745f596c

feat(server): shard token decode (#303)

OlivierDehaene committed 2 years ago

Verified 68e9d6ab

fix(docker): remove nvidia require cuda env (#310)

OlivierDehaene committed 2 years ago

Verified 15854044

fix(docker): fix nvidia env vars (#305)

OlivierDehaene committed 2 years ago

Verified 49cffad1

feat(server): optim flash causal lm decode_token (#285)

OlivierDehaene committed 2 years ago

Verified ad66f6ef

fix(docker): fix docker build (#299)

OlivierDehaene committed 2 years ago

Verified bc5c0723

feat(docker): add benchmarking tool to docker image (#298)

OlivierDehaene committed 2 years ago

Verified e2502822

feat(router): Adding response schema for compat_generate (#292)

gsaivinay committed 2 years ago

Verified 926fd9a0

fix(dockerfile): fix nvidia env vars (#297)

OlivierDehaene committed 2 years ago

Verified e9b01b34

fea(server): decrease convert RAM requirements (#286)

Narsil committed 2 years ago

Verified b4aa87db

chore: add `flash-attention` to docker ignore (#287)

Narsil committed 2 years ago

Verified 3314a46d

fix(server): fix convert (#284)

Narsil committed 2 years ago

Verified 690fc317

feat(launcher): Improve error message when download process fails. (#276)

Narsil committed 2 years ago

Verified e68509ad

fix(server): Removes the parallelism in file convertion (during download) (#275)

Narsil committed 2 years ago

Verified f08343d4

fix(launcher): handle hub branches (#278)

Narsil committed 2 years ago

Verified b4fe248b

fix(launcher): pass weights cache override to the download process (#274)

OlivierDehaene committed 2 years ago

Verified b67908e0

feat(server): support hf endpoint weight layout (#266)

OlivierDehaene committed 2 years ago

Verified 85aa7e2e

Older