huggingface/text-generation-inference

Pull Requests Commits

Add Phi-3 medium support (#2039)

danieldk committed 1 year ago

Verified 85dfc392

ROCm and sliding windows fixes (#2033)

fxmarty committed 1 year ago

Verified 9b3674d9

server: use chunked inputs

danieldk committed 1 year ago

bf3c8137

Xpu gqa (#2013)

sywangyi committed 1 year ago

Verified 4dabddb7

Revert "Enabling CI for AMD with new runner.."

Narsil committed 1 year ago

97656582

Enabling CI for AMD with new runner..

Narsil committed 1 year ago

101ac9a7

Internal runner ? (#2023)

Narsil committed 1 year ago

Verified ed1cfde0

marlin: improve build

danieldk committed 1 year ago

51621439

marlin: support tp>1 when group_size==-1

danieldk committed 1 year ago

0d96468e

Add support for Marlin-quantized models

danieldk committed 1 year ago

4594e6fa

Revert "Less cache misses on cargo build."

Narsil committed 1 year ago

cf0d459a

Less cache misses on cargo build.

Narsil committed 1 year ago

5aec4154

Update __version__ on __init__.py to 0.7.0 (#2017)

andimarafioti committed 1 year ago

Verified 2a48a100

Fix GPTQWeight import (#2020)

danieldk committed 1 year ago

Verified 3f4bcf97

Fixing rocm. (#2021)

Narsil committed 1 year ago

Verified 0a94fad7

feat: move allocation logic to rust (#1835)

OlivierDehaene committed 1 year ago

Verified 8aece3bd

Do not initialize scratch space when there are no ExLlamaV2 layers (#2015)

danieldk committed 1 year ago

Verified 9ffe1f1e

Hotfixing `make install`. (#2008)

Narsil committed 1 year ago

Verified 824edf28

Making `make install` work better by default. (#2004)

Narsil committed 1 year ago

Verified 8390e251

Support GPTQ models with column-packed up/gate tensor (#2006)

danieldk committed 1 year ago

Verified d14eaaca

feat: add SchedulerV3 (#1996)

OlivierDehaene committed 1 year ago

Verified 757223b3

fix: update triton implementation reference (#2002)

emmanuel-ferdman committed 1 year ago

Verified fec0167a

Fix Phi-2 with `tp>1` (#2003)

danieldk committed 1 year ago

Verified 9b52f0e2

router: send the input as chunks to the backend

danieldk committed 1 year ago

df71aafd

reable xpu, broken by gptq and setuptool upgrade (#1988)

sywangyi committed 1 year ago

Verified d1d724b0

Narsil committed 1 year ago

9a59ebce

Fixing GPTQ imports. (#1994)

Narsil committed 1 year ago

Verified 9add5d0a

Narsil committed 1 year ago

799a193b

single char ` addition for docs (#1989)

nbroad1881 committed 1 year ago

Verified 08b3eac2

Fixing exl2 scratch buffer. (#1990)

Narsil committed 1 year ago

Verified 5ab4cef6

Newer Older