huggingface/text-generation-inference

Pull Requests Commits

chore(github): add templates (#264)

Narsil committed 2 years ago

Verified 411b0d4e

Adding docs on how dynamic batching works. (#258)

Narsil committed 2 years ago

Verified e86cca97

feat(benchmark): add support for private tokenizers (#262)

OlivierDehaene committed 2 years ago

Verified 0e9d249b

doc(launcher): add more docs to the `launcher` itself and link in the README (#257)

Narsil committed 2 years ago

Verified b0b97fd9

feat(docker): add nvidia env vars (#255)

OlivierDehaene committed 2 years ago

Verified 593a5634

feat(server): add watermarking tests (#248)

ehsanmok committed 2 years ago

Verified f092ba9b

chore(server): update transformers (#250)

OlivierDehaene committed 2 years ago

Verified b9ae7e5d

fix(server): Small tidy of code from recent changes (#251)

njhill committed 2 years ago

Verified 34bca0b8

fix(server): fix reshaping of bloom past_key_values in concatenate() (#252)

njhill committed 2 years ago

Verified b4cf832c

feat(router): new healthcheck that skips the queue (#244)

Narsil committed 2 years ago

Verified db2b4e07

feat(router): add tests to validation (#237)

Narsil committed 2 years ago

Verified c4fb09f2

chore(launcher): refactor logic (#242)

Narsil committed 2 years ago

Verified 77758f60

fix(benchmarking): fix benchmarking tool

OlivierDehaene committed 2 years ago

7de8a377

Starting some routing tests. (#233)

Narsil committed 2 years ago

Verified 45344244

fix(python-client): add auth headers to is supported requests (#234)

OlivierDehaene committed 2 years ago

Verified 323546df

chore(server): update safetensors version (#235)

OlivierDehaene committed 2 years ago

Verified 37b64a5c

feat(router): add endpoint info to /info route (#228)

OlivierDehaene committed 2 years ago

Verified 8b182eb9

feat(router): use number of tokens in batch as input for dynamic batching (#226)

OlivierDehaene committed 2 years ago

Verified ebc74d56

chore(server): update huggingface-hub (#227)

OlivierDehaene committed 2 years ago

Verified 98a3e0d1

feat(server): reduce memory requirement (#214)

njhill committed 2 years ago

Verified 4a7dd408

OlivierDehaene committed 2 years ago

Verified 6ded76a4

misc: update to rust 1.69 (#221)

OlivierDehaene committed 2 years ago

Verified 97df0c7b

fix(server): fix flash batch filtering (#220)

OlivierDehaene committed 2 years ago

Verified 4b460e72

fix(server): fix flash causal (#219)

OlivierDehaene committed 2 years ago

Verified 1ffea36e

fix(server): fix flash causal (#218)

OlivierDehaene committed 2 years ago

Verified 86bca365

fix(server): cleanup new flash past_key_values logic (#217)

OlivierDehaene committed 2 years ago

Verified afc5b999

fix(server): fix past key values logic (#216)

OlivierDehaene committed 2 years ago

Verified db4cb5e4

feat(router): add device and dtype info (#215)

OlivierDehaene committed 2 years ago

Verified 343437c7

feat(server): flash attention past key value optimizations (#213)

njhill committed 2 years ago

Verified ac8c0f6f

fix(ci): fix sha in docker image (#212)

OlivierDehaene committed 2 years ago

Verified 274513e6

Newer Older