huggingface/text-generation-inference

Pull Requests Commits

allow to fix paged attention num blocks

fxmarty committed 1 year ago

bb37321b

Narsil committed 1 year ago

9a59ebce

Fixing GPTQ imports. (#1994)

Narsil committed 1 year ago

Verified 9add5d0a

Narsil committed 1 year ago

799a193b

single char ` addition for docs (#1989)

nbroad1881 committed 1 year ago

Verified 08b3eac2

Fixing exl2 scratch buffer. (#1990)

Narsil committed 1 year ago

Verified 5ab4cef6

Purely refactors paged/attention into `layers/attention` and make hardware differences more obvious with 1 file per hardware. (#1986)

Narsil committed 1 year ago

Verified 06edde94

Update documentation version to 2.0.4 (#1980)

fxmarty committed 1 year ago

Verified 659bd67f

Gemma GPTQ checks: skip logprob checks

danieldk committed 1 year ago

967ced2f

Add support for exl2 quantization

danieldk committed 1 year ago

36dd1601

feat: adjust attn weight loading logic (#1975)

drbh committed 1 year ago

Verified cbced7f0

Fixing the text part from tokenizer endpoint. (#1967)

Narsil committed 1 year ago

Verified 612bc483

Fix (non-container) pytest stdout buffering-related lock-up

danieldk committed 1 year ago

f20463e4

Upgrade to Axum 0.7 and Hyper 1.0 (Breaking change: disabled ngrok tunneling). (#1959)

Narsil committed 1 year ago

Verified e76b9824

fix small typo and broken link (#1958)

MoritzLaurer committed 1 year ago

Verified b7ffa287

Processor config chat template (#1954)

drbh committed 1 year ago

Verified 0732b9d2

Fix GPTQ for models which do not have float16 at the default dtype (simpler) (#1953)

danieldk committed 1 year ago

Verified a401c83c

Fix (flash) Gemma prefix and enable tests

danieldk committed 1 year ago

9231098f

Fix seeded output. (#1949)

Narsil committed 1 year ago

Verified d32e33bd

Fixing codellama loads by using purely `AutoTokenizer`. (#1947)

Narsil committed 1 year ago

Verified cff472ba

Improving the logging system. (#1938)

Narsil committed 1 year ago

Verified 95465346

Add completion route to client and add stop parameter where it's missing (#1869)

thomas-schillaci committed 1 year ago

Verified 629047cb

Fixing some legacy behavior (big swapout of serverless on legacy stuff). (#1937)

Narsil committed 1 year ago

Verified f4a073ae

reenable xpu for tgi (#1939)

sywangyi committed 1 year ago

Verified f41d644a

feat: add train medusa head tutorial (#1934)

drbh committed 1 year ago

Verified a103e3e9

fix: use path inside of speculator config (#1935)

drbh committed 1 year ago

Verified efb73fcb

Creating doc automatically for supported models. (#1929)

Narsil committed 1 year ago

Verified 2f243a1a

feat: include token in client test like server tests (#1932)

drbh committed 1 year ago

Verified fc0eaffc

docs: Fix grafana dashboard url (#1925)

edwardzjl committed 1 year ago

Verified 904ff369

ROCm: make CK FA2 default instead of Triton (#1924)

fxmarty committed 1 year ago

Verified 293b8125

Older