huggingface/text-generation-inference

Pull Requests Commits

add logic to queue

OlivierDehaene committed 2 years ago

a9634953

feat(server): improve max tokens calculation

OlivierDehaene committed 2 years ago

4f460e5b

fix(benchmarking): fix benchmarking tool

OlivierDehaene committed 2 years ago

7de8a377

Starting some routing tests. (#233)

Narsil committed 2 years ago

Verified 45344244

fix(python-client): add auth headers to is supported requests (#234)

OlivierDehaene committed 2 years ago

Verified 323546df

chore(server): update safetensors version (#235)

OlivierDehaene committed 2 years ago

Verified 37b64a5c

feat(router): add endpoint info to /info route (#228)

OlivierDehaene committed 2 years ago

Verified 8b182eb9

feat(router): use number of tokens in batch as input for dynamic batching (#226)

OlivierDehaene committed 2 years ago

Verified ebc74d56

chore(server): update huggingface-hub (#227)

OlivierDehaene committed 2 years ago

Verified 98a3e0d1

feat(server): reduce memory requirement (#214)

njhill committed 2 years ago

Verified 4a7dd408

OlivierDehaene committed 2 years ago

Verified 6ded76a4

misc: update to rust 1.69 (#221)

OlivierDehaene committed 2 years ago

Verified 97df0c7b

fix(server): fix flash batch filtering (#220)

OlivierDehaene committed 2 years ago

Verified 4b460e72

fix(server): fix flash causal (#219)

OlivierDehaene committed 2 years ago

Verified 1ffea36e

fix(server): fix flash causal (#218)

OlivierDehaene committed 2 years ago

Verified 86bca365

fix(server): cleanup new flash past_key_values logic (#217)

OlivierDehaene committed 2 years ago

Verified afc5b999

fix(server): fix past key values logic (#216)

OlivierDehaene committed 2 years ago

Verified db4cb5e4

feat(router): add device and dtype info (#215)

OlivierDehaene committed 2 years ago

Verified 343437c7

feat(server): flash attention past key value optimizations (#213)

njhill committed 2 years ago

Verified ac8c0f6f

fix(ci): fix sha in docker image (#212)

OlivierDehaene committed 2 years ago

Verified 274513e6

feat(router): drop requests when client closes the channel (#202)

OlivierDehaene committed 2 years ago

Verified 709d8936

feat(router): add git sha to info route (#208)

OlivierDehaene committed 2 years ago

Verified b6ee0ec7

fix(router): add auth token to get model info (#207)

OlivierDehaene committed 2 years ago

Verified 252f42c1

fix(docker): remove unused dependencies (#205)

OlivierDehaene committed 2 years ago

Verified 6837b2eb

fix(server): fix hf_transfer issue with private repos (#203)

OlivierDehaene committed 2 years ago

Verified 5d27f525

feat(server): check cuda capability when importing flash models (#201)

OlivierDehaene committed 2 years ago

Verified a88c54bb

feat(server): support quantization for flash models (#200)

OlivierDehaene committed 2 years ago

Verified e14ae3b5

feat(router): add info route (#196)

OlivierDehaene committed 2 years ago

Verified 2475aede

feat(python-client): get list of currently deployed tgi models using the inference API (#191)

OlivierDehaene committed 2 years ago

Verified b927244e

fix(router): fix truncation (#190)

OlivierDehaene committed 2 years ago

Verified c13b9d87

Older