huggingface/text-generation-inference

Pull Requests Commits

Narsil committed 2 years ago

bfa3920a

Update layers.py

flozi00 committed 2 years ago

Verified a1c23f38

Update seq2seq_lm.py

flozi00 committed 2 years ago

Verified 64accc59

Update santacoder.py

flozi00 committed 2 years ago

Verified 780198b9

flozi00 committed 2 years ago

Verified 377c01e2

Update causal_lm.py

flozi00 committed 2 years ago

Verified 198e6179

Update requirements.txt bnb version

flozi00 committed 2 years ago

Verified f2fae6db

feat: better errors for warmup and TP (#575)

OlivierDehaene committed 2 years ago

Verified b4024edd

fix(server): harden the weights choice to save on disk. (#561)

Narsil committed 2 years ago

Verified e943a294

OlivierDehaene committed 2 years ago

Verified 31b36cca

fix(server): decrease memory fragmentation (#557)

OlivierDehaene committed 2 years ago

Verified c4bb5264

feat(router): add argument for hostname in router (#545) (#550)

OlivierDehaene committed 2 years ago

Verified 6f429427

feat(server): use latest flash attention commit (#543)

OlivierDehaene committed 2 years ago

Verified 31e2253a

fix(server): avoid errors for very small top_p values (#544)

njhill committed 2 years ago

Verified e4b26aa1

fix(server): Handle loading from local files for MPT (#534)

Yard1 committed 2 years ago

Verified 2a101207

docs(benchmarker): Adding some help for the options in `text-generation-benchmark`. (#462)

Narsil committed 2 years ago

Verified e6888d0e

fix: Update server/Makefile to include Makefile-vllm (#520)

Yard1 committed 2 years ago

Verified 8405581f

feat(server): Add Non flash MPT. (#514)

Narsil committed 2 years ago

Verified 1da07e85

OlivierDehaene committed 2 years ago

Verified e28a8090

fix(launcher): fix issue where launcher does not properly report shard failures (#522)

OlivierDehaene committed 2 years ago

Verified 2b53d719

feat: Add the option to force another dtype than `f16`. (#513)

Narsil committed 2 years ago

Verified ecf6dc3a

feat(router): arg validation (#519)

OlivierDehaene committed 2 years ago

Verified 3b0c979e

feat(server): add paged attention to flash models (#516)

OlivierDehaene committed 2 years ago

Verified e74bd41e

feat(router): add header option to disable buffering for the generate_stream response (#498)

rkimball committed 2 years ago

Verified 70f485bf

fix(server): Do not init process group if already initialized (#388)

Yard1 committed 2 years ago

Verified ae466a87

feat(server): Add inference support for GPTQ (llama + falcon tested) + Quantization script (#438)

Narsil committed 2 years ago

Verified aefde28b

fix(router): add timeout on flume sends (#488)

OlivierDehaene committed 2 years ago

Verified bd3a9d8e

feat(server): Adding new ignore_rule for conversion. (#485)

Narsil committed 2 years ago

Verified 776d150c

feat(server): Update convert logic. (#483)

Narsil committed 2 years ago

Verified 49b4b33e

fix(server): Fixing T5 in case the names are mixed up. (#475)

Narsil committed 2 years ago

Verified c9c65ab3

Older