huggingface/text-generation-inference

Pull Requests Commits

OlivierDehaene committed 2 years ago

Verified 31b36cca

fix(server): decrease memory fragmentation (#557)

OlivierDehaene committed 2 years ago

Verified c4bb5264

feat(router): add argument for hostname in router (#545) (#550)

OlivierDehaene committed 2 years ago

Verified 6f429427

feat(server): use latest flash attention commit (#543)

OlivierDehaene committed 2 years ago

Verified 31e2253a

fix(server): avoid errors for very small top_p values (#544)

njhill committed 2 years ago

Verified e4b26aa1

fix(server): Handle loading from local files for MPT (#534)

Yard1 committed 2 years ago

Verified 2a101207

docs(benchmarker): Adding some help for the options in `text-generation-benchmark`. (#462)

Narsil committed 2 years ago

Verified e6888d0e

fix: Update server/Makefile to include Makefile-vllm (#520)

Yard1 committed 2 years ago

Verified 8405581f

feat(server): Add Non flash MPT. (#514)

Narsil committed 2 years ago

Verified 1da07e85

OlivierDehaene committed 2 years ago

Verified e28a8090

fix(launcher): fix issue where launcher does not properly report shard failures (#522)

OlivierDehaene committed 2 years ago

Verified 2b53d719

feat: Add the option to force another dtype than `f16`. (#513)

Narsil committed 2 years ago

Verified ecf6dc3a

feat(router): arg validation (#519)

OlivierDehaene committed 2 years ago

Verified 3b0c979e

feat(server): add paged attention to flash models (#516)

OlivierDehaene committed 2 years ago

Verified e74bd41e

feat(router): add header option to disable buffering for the generate_stream response (#498)

rkimball committed 2 years ago

Verified 70f485bf

fix(server): Do not init process group if already initialized (#388)

Yard1 committed 2 years ago

Verified ae466a87

feat(server): Add inference support for GPTQ (llama + falcon tested) + Quantization script (#438)

Narsil committed 2 years ago

Verified aefde28b

fix(router): add timeout on flume sends (#488)

OlivierDehaene committed 2 years ago

Verified bd3a9d8e

feat(server): Adding new ignore_rule for conversion. (#485)

Narsil committed 2 years ago

Verified 776d150c

feat(server): Update convert logic. (#483)

Narsil committed 2 years ago

Verified 49b4b33e

fix(server): Fixing T5 in case the names are mixed up. (#475)

Narsil committed 2 years ago

Verified c9c65ab3

fix(server): fix warpers on CPU (#472)

OlivierDehaene committed 2 years ago

Verified 53aa9194

feat(server): improve flash attention import errors (#465)

OlivierDehaene committed 2 years ago

Verified ece7ffa4

feat(router): add ngrok integration (#453)

OlivierDehaene committed 2 years ago

Verified f59fb8b6

feat(server): pre-allocate past key values for flash causal LM (#412)

OlivierDehaene committed 2 years ago

Verified 5ce89059

fix(makefile): Fix typo and use POSIX comparison in the makefile (#443)

piratos committed 2 years ago

Verified ca650e5b

docs(launcher): fix CUDA_VISIBLE_DEVICES helper comment (#441)

antferdom committed 2 years ago

Verified d4eb60f4

feat(server): optimize dist ops (#434)

OlivierDehaene committed 2 years ago

Verified e496c9ba

feat(server): Rework model loading (#344)

Narsil committed 2 years ago

Verified abd58ff8

chore: update openapi schema

OlivierDehaene committed 2 years ago

19c41824

Newer Older