huggingface/text-generation-inference

Pull Requests Commits

fix(server): T5 weights names. (#582)

Narsil committed 2 years ago

Verified db4efbf4

chore: migrate ci region for more availability. (#581)

Narsil committed 2 years ago

Verified f063ebde

feat(server): Support for env value for GPTQ_BITS and GPTQ_GROUPSIZE. (#580)

Narsil committed 2 years ago

Verified 5bd2ab65

fix(server): Fixing RW code (it's remote code so the Arch checking doesn't work to see which weights to keep). (#579)

Narsil committed 2 years ago

Verified f0181436

feat: better errors for warmup and TP (#575)

OlivierDehaene committed 2 years ago

Verified b4024edd

fix(server): harden the weights choice to save on disk. (#561)

Narsil committed 2 years ago

Verified e943a294

OlivierDehaene committed 2 years ago

Verified 31b36cca

fix(server): decrease memory fragmentation (#557)

OlivierDehaene committed 2 years ago

Verified c4bb5264

feat(router): add argument for hostname in router (#545) (#550)

OlivierDehaene committed 2 years ago

Verified 6f429427

feat(server): use latest flash attention commit (#543)

OlivierDehaene committed 2 years ago

Verified 31e2253a

fix(server): avoid errors for very small top_p values (#544)

njhill committed 2 years ago

Verified e4b26aa1

fix(server): Handle loading from local files for MPT (#534)

Yard1 committed 2 years ago

Verified 2a101207

docs(benchmarker): Adding some help for the options in `text-generation-benchmark`. (#462)

Narsil committed 2 years ago

Verified e6888d0e

fix: Update server/Makefile to include Makefile-vllm (#520)

Yard1 committed 2 years ago

Verified 8405581f

feat(server): Add Non flash MPT. (#514)

Narsil committed 2 years ago

Verified 1da07e85

OlivierDehaene committed 2 years ago

Verified e28a8090

fix(launcher): fix issue where launcher does not properly report shard failures (#522)

OlivierDehaene committed 2 years ago

Verified 2b53d719

feat: Add the option to force another dtype than `f16`. (#513)

Narsil committed 2 years ago

Verified ecf6dc3a

feat(router): arg validation (#519)

OlivierDehaene committed 2 years ago

Verified 3b0c979e

feat(server): add paged attention to flash models (#516)

OlivierDehaene committed 2 years ago

Verified e74bd41e

feat(router): add header option to disable buffering for the generate_stream response (#498)

rkimball committed 2 years ago

Verified 70f485bf

fix(server): Do not init process group if already initialized (#388)

Yard1 committed 2 years ago

Verified ae466a87

feat(server): Add inference support for GPTQ (llama + falcon tested) + Quantization script (#438)

Narsil committed 2 years ago

Verified aefde28b

fix(router): add timeout on flume sends (#488)

OlivierDehaene committed 2 years ago

Verified bd3a9d8e

feat(server): Adding new ignore_rule for conversion. (#485)

Narsil committed 2 years ago

Verified 776d150c

feat(server): Update convert logic. (#483)

Narsil committed 2 years ago

Verified 49b4b33e

fix(server): Fixing T5 in case the names are mixed up. (#475)

Narsil committed 2 years ago

Verified c9c65ab3

fix(server): fix warpers on CPU (#472)

OlivierDehaene committed 2 years ago

Verified 53aa9194

feat(server): improve flash attention import errors (#465)

OlivierDehaene committed 2 years ago

Verified ece7ffa4

feat(router): add ngrok integration (#453)

OlivierDehaene committed 2 years ago

Verified f59fb8b6

Older