huggingface/text-generation-inference

Pull Requests Commits

marlin: support tp>1 when group_size==-1

danieldk committed 1 year ago

0d96468e

Add support for Marlin-quantized models

danieldk committed 1 year ago

4594e6fa

Revert "Less cache misses on cargo build."

Narsil committed 1 year ago

cf0d459a

Less cache misses on cargo build.

Narsil committed 1 year ago

5aec4154

Update __version__ on __init__.py to 0.7.0 (#2017)

andimarafioti committed 1 year ago

Verified 2a48a100

Fix GPTQWeight import (#2020)

danieldk committed 1 year ago

Verified 3f4bcf97

Fixing rocm. (#2021)

Narsil committed 1 year ago

Verified 0a94fad7

feat: move allocation logic to rust (#1835)

OlivierDehaene committed 1 year ago

Verified 8aece3bd

Do not initialize scratch space when there are no ExLlamaV2 layers (#2015)

danieldk committed 1 year ago

Verified 9ffe1f1e

Hotfixing `make install`. (#2008)

Narsil committed 1 year ago

Verified 824edf28

Making `make install` work better by default. (#2004)

Narsil committed 1 year ago

Verified 8390e251

Support GPTQ models with column-packed up/gate tensor (#2006)

danieldk committed 1 year ago

Verified d14eaaca

feat: add SchedulerV3 (#1996)

OlivierDehaene committed 1 year ago

Verified 757223b3

fix: update triton implementation reference (#2002)

emmanuel-ferdman committed 1 year ago

Verified fec0167a

Fix Phi-2 with `tp>1` (#2003)

danieldk committed 1 year ago

Verified 9b52f0e2

router: send the input as chunks to the backend

danieldk committed 1 year ago

df71aafd

reable xpu, broken by gptq and setuptool upgrade (#1988)

sywangyi committed 1 year ago

Verified d1d724b0

Narsil committed 1 year ago

9a59ebce

Fixing GPTQ imports. (#1994)

Narsil committed 1 year ago

Verified 9add5d0a

Narsil committed 1 year ago

799a193b

single char ` addition for docs (#1989)

nbroad1881 committed 1 year ago

Verified 08b3eac2

Fixing exl2 scratch buffer. (#1990)

Narsil committed 1 year ago

Verified 5ab4cef6

Purely refactors paged/attention into `layers/attention` and make hardware differences more obvious with 1 file per hardware. (#1986)

Narsil committed 1 year ago

Verified 06edde94

Update documentation version to 2.0.4 (#1980)

fxmarty committed 1 year ago

Verified 659bd67f

Gemma GPTQ checks: skip logprob checks

danieldk committed 1 year ago

967ced2f

Add support for exl2 quantization

danieldk committed 1 year ago

36dd1601

feat: adjust attn weight loading logic (#1975)

drbh committed 1 year ago

Verified cbced7f0

Fixing the text part from tokenizer endpoint. (#1967)

Narsil committed 1 year ago

Verified 612bc483

Fix (non-container) pytest stdout buffering-related lock-up

danieldk committed 1 year ago

f20463e4

Upgrade to Axum 0.7 and Hyper 1.0 (Breaking change: disabled ngrok tunneling). (#1959)

Narsil committed 1 year ago

Verified e76b9824

Newer Older