huggingface/text-generation-inference

Pull Requests Commits

feat: add test to view batch speedup amount

drbh committed 1 year ago

cb232a35

Hotfixing qwen2 and starcoder2 (which also get clamping). (#2167)

Narsil committed 1 year ago

Verified 0759ec49

Ci test (#2124)

glegendre01 committed 1 year ago

Verified 963b6c6f

Fixing rocm. (#2164)

Narsil committed 1 year ago

Verified dea9c0dc

fix: use the base layers weight in mistral rocm (#2155)

drbh committed 1 year ago

Verified b966bc0d

fix FlashDecoding change's regression in intel platform (#2161)

sywangyi committed 1 year ago

Verified 5d97e0c4

Fixing graph capture for flash decoding. (#2163)

Narsil committed 1 year ago

Verified 022f6515

[Major Change][Undecided yet] Move to FlashDecoding instead of PagedAttention kernel. (#1940)

Narsil committed 1 year ago

Verified 4327210e

Fixing baichuan override. (#2158)

Narsil committed 1 year ago

Verified 4f55f158

GH router. (#2153)

Narsil committed 1 year ago

Verified d0225b10

Fixing test. (#2152)

Narsil committed 1 year ago

Verified 17cebc45

fix: prefer serde structs over custom functions (#2127)

drbh committed 1 year ago

Verified 9eefb2f6

refine get xpu free memory/enable Qwen2/gemma2/gemma/phi in intel platform (#2132)

sywangyi committed 1 year ago

Verified 5da4cfab

fix AttributeError: 'MixtralLayer' object has no attribute 'mlp' (#2123)

icyxp committed 1 year ago

Verified 9d0ca503

Use GPTQ-Marlin for supported GPTQ configurations (#2111)

danieldk committed 1 year ago

Verified 2ce80194

feat: download lora adapter weights from launcher (#2140)

drbh committed 1 year ago

Verified 0d97a93c

fix: use weights from base_layer (#2141)

drbh committed 1 year ago

Verified 25f57e2e

Fixing clippy. (#2149)

Narsil committed 1 year ago

Verified b4552f9d

fix microsoft/Phi-3-mini-4k-instruct crash in batch.slots[batch.slot_… (#2148)

sywangyi committed 1 year ago

Verified 6ea570dd

Fixing the CI to also run in release when it's a tag ? (#2138)

Narsil committed 1 year ago

Verified fb98ab27

fix: refactor post_processor logic and add test (#2137)

drbh committed 1 year ago

Verified 74b0231b

Fixing gemma2. (#2135)

Narsil committed 1 year ago

Verified 3ea8259a

Fixing malformed rust tokenizers (#2134)

Narsil committed 1 year ago

Verified 0e4ab6d3

Idefics2: sync added image tokens with transformers (#2080)

danieldk committed 1 year ago

Verified dd2d91b0

Bumping to 2.1 (#2131)

Narsil committed 1 year ago

Verified b53b21c6

Fixing prom leak by upgrading. (#2129)

Narsil committed 1 year ago

Verified bcfcd474

fix: simplify kserve endpoint and fix imports (#2119)

drbh committed 1 year ago

Verified be2d3803

Add support for Marlin 2:4 sparsity (#2102)

danieldk committed 1 year ago

Verified f1f98e36

Support AWQ quantization with bias (#2117)

danieldk committed 1 year ago

Verified 14980df2

Enable multiple LoRa adapters (#2010)

drbh committed 1 year ago

Verified 04e1af94

Older