huggingface/text-generation-inference

Pull Requests Commits

Fixup residual, initial block attention config

danieldk committed 2 years ago

5d2b93ba

WIP, many bits are still missing...

danieldk committed 2 years ago

4ed551ab

Factor out sharding of packed tensors

danieldk committed 2 years ago

c0f201c9

Support chat response format (#2046)

drbh committed 2 years ago

Verified 376a0b7a

Update LLMM1 bound (#2050)

fxmarty committed 2 years ago

Verified a6e4d63c

fix(ci): remove unnecessary permissions (#2045)

McPatate committed 2 years ago

Verified dfca1dfc

feat(ci): add trufflehog secrets detection (#2038)

McPatate committed 2 years ago

Verified 4e74ec09

Add Phi-3 medium support (#2039)

danieldk committed 2 years ago

Verified 85dfc392

ROCm and sliding windows fixes (#2033)

fxmarty committed 2 years ago

Verified 9b3674d9

server: use chunked inputs

danieldk committed 2 years ago

bf3c8137

Xpu gqa (#2013)

sywangyi committed 2 years ago

Verified 4dabddb7

Revert "Enabling CI for AMD with new runner.."

Narsil committed 2 years ago

97656582

Enabling CI for AMD with new runner..

Narsil committed 2 years ago

101ac9a7

Internal runner ? (#2023)

Narsil committed 2 years ago

Verified ed1cfde0

marlin: improve build

danieldk committed 2 years ago

51621439

marlin: support tp>1 when group_size==-1

danieldk committed 2 years ago

0d96468e

Add support for Marlin-quantized models

danieldk committed 2 years ago

4594e6fa

Revert "Less cache misses on cargo build."

Narsil committed 2 years ago

cf0d459a

Less cache misses on cargo build.

Narsil committed 2 years ago

5aec4154

Update __version__ on __init__.py to 0.7.0 (#2017)

andimarafioti committed 2 years ago

Verified 2a48a100

Fix GPTQWeight import (#2020)

danieldk committed 2 years ago

Verified 3f4bcf97

Fixing rocm. (#2021)

Narsil committed 2 years ago

Verified 0a94fad7

feat: move allocation logic to rust (#1835)

OlivierDehaene committed 2 years ago

Verified 8aece3bd

Do not initialize scratch space when there are no ExLlamaV2 layers (#2015)

danieldk committed 2 years ago

Verified 9ffe1f1e

Hotfixing `make install`. (#2008)

Narsil committed 2 years ago

Verified 824edf28

Making `make install` work better by default. (#2004)

Narsil committed 2 years ago

Verified 8390e251

Support GPTQ models with column-packed up/gate tensor (#2006)

danieldk committed 2 years ago

Verified d14eaaca

feat: add SchedulerV3 (#1996)

OlivierDehaene committed 2 years ago

Verified 757223b3

fix: update triton implementation reference (#2002)

emmanuel-ferdman committed 2 years ago

Verified fec0167a

Fix Phi-2 with `tp>1` (#2003)

danieldk committed 2 years ago

Verified 9b52f0e2

Older