huggingface/text-generation-inference

Pull Requests Commits

feat: adjust to load weights

drbh committed 2 years ago

cf8fdef9

feat: move allocation logic to rust (#1835)

OlivierDehaene committed 2 years ago

Verified 8aece3bd

Do not initialize scratch space when there are no ExLlamaV2 layers (#2015)

danieldk committed 2 years ago

Verified 9ffe1f1e

Hotfixing `make install`. (#2008)

Narsil committed 2 years ago

Verified 824edf28

Making `make install` work better by default. (#2004)

Narsil committed 2 years ago

Verified 8390e251

Support GPTQ models with column-packed up/gate tensor (#2006)

danieldk committed 2 years ago

Verified d14eaaca

feat: add SchedulerV3 (#1996)

OlivierDehaene committed 2 years ago

Verified 757223b3

fix: update triton implementation reference (#2002)

emmanuel-ferdman committed 2 years ago

Verified fec0167a

Fix Phi-2 with `tp>1` (#2003)

danieldk committed 2 years ago

Verified 9b52f0e2

router: send the input as chunks to the backend

danieldk committed 2 years ago

df71aafd

reable xpu, broken by gptq and setuptool upgrade (#1988)

sywangyi committed 2 years ago

Verified d1d724b0

Narsil committed 2 years ago

9a59ebce

Fixing GPTQ imports. (#1994)

Narsil committed 2 years ago

Verified 9add5d0a

Narsil committed 2 years ago

799a193b

single char ` addition for docs (#1989)

nbroad1881 committed 2 years ago

Verified 08b3eac2

Fixing exl2 scratch buffer. (#1990)

Narsil committed 2 years ago

Verified 5ab4cef6

Purely refactors paged/attention into `layers/attention` and make hardware differences more obvious with 1 file per hardware. (#1986)

Narsil committed 2 years ago

Verified 06edde94

Update documentation version to 2.0.4 (#1980)

fxmarty committed 2 years ago

Verified 659bd67f

Gemma GPTQ checks: skip logprob checks

danieldk committed 2 years ago

967ced2f

Add support for exl2 quantization

danieldk committed 2 years ago

36dd1601

feat: adjust attn weight loading logic (#1975)

drbh committed 2 years ago

Verified cbced7f0

Fixing the text part from tokenizer endpoint. (#1967)

Narsil committed 2 years ago

Verified 612bc483

Fix (non-container) pytest stdout buffering-related lock-up

danieldk committed 2 years ago

f20463e4

Upgrade to Axum 0.7 and Hyper 1.0 (Breaking change: disabled ngrok tunneling). (#1959)

Narsil committed 2 years ago

Verified e76b9824

fix small typo and broken link (#1958)

MoritzLaurer committed 2 years ago

Verified b7ffa287

Processor config chat template (#1954)

drbh committed 2 years ago

Verified 0732b9d2

Fix GPTQ for models which do not have float16 at the default dtype (simpler) (#1953)

danieldk committed 2 years ago

Verified a401c83c

Fix (flash) Gemma prefix and enable tests

danieldk committed 2 years ago

9231098f

Fix seeded output. (#1949)

Narsil committed 2 years ago

Verified d32e33bd

Fixing codellama loads by using purely `AutoTokenizer`. (#1947)

Narsil committed 2 years ago

Verified cff472ba

Older