huggingface/text-generation-inference

Pull Requests Commits

Falcon/DBRX: get correct number of key-value heads (#2205)

danieldk committed 1 year ago

Verified 5c7c9f13

Fix incorrect cache allocation with multi-query (#2203)

danieldk committed 1 year ago

Verified 153fcf77

hotfix: Fix number of KV heads (#2202)

danieldk committed 1 year ago

Verified cce475a9

fix dbrx & opt model prefix bug (#2201)

icyxp committed 1 year ago

Verified 521d0d99

Consistently take `prefix` in model constructors (#2191)

danieldk committed 1 year ago

Verified 05c094fc

GPTQ CI improvements (#2151)

danieldk committed 1 year ago

Verified 67ef0649

Fix Starcoder2 after refactor (#2189)

danieldk committed 1 year ago

Verified b67d4633

Hotfixing after refactor.

Narsil committed 1 year ago

853d4eb9

Refactor dead code - Removing all `flash_xxx.py` files. (#2166)

Narsil committed 1 year ago

Verified fb2f74e2

Adding "longrope" for Phi-3 (#2172) (#2179)

amihalik committed 1 year ago

Verified c6bcadf8

Preparing patch release. (#2186)

Narsil committed 1 year ago

Verified 245d3de9

Fixing missing `object` field for regular completions. (#2175)

Narsil committed 1 year ago

Verified 5ad41aa2

Fixing the dockerfile warnings. (#2173)

Narsil committed 1 year ago

Verified 2b3bd1e0

Revert "Fixing missing `object` field for regular completions."

Narsil committed 1 year ago

be4a4c47

Fixing missing `object` field for regular completions.

Narsil committed 1 year ago

2bbb7fa4

feat: improve update_docs for openapi schema (#2169)

drbh committed 1 year ago

Verified 571530dd

Hotfixing qwen2 and starcoder2 (which also get clamping). (#2167)

Narsil committed 1 year ago

Verified 0759ec49

Ci test (#2124)

glegendre01 committed 1 year ago

Verified 963b6c6f

Fixing rocm. (#2164)

Narsil committed 1 year ago

Verified dea9c0dc

fix: use the base layers weight in mistral rocm (#2155)

drbh committed 1 year ago

Verified b966bc0d

fix FlashDecoding change's regression in intel platform (#2161)

sywangyi committed 1 year ago

Verified 5d97e0c4

Fixing graph capture for flash decoding. (#2163)

Narsil committed 1 year ago

Verified 022f6515

[Major Change][Undecided yet] Move to FlashDecoding instead of PagedAttention kernel. (#1940)

Narsil committed 1 year ago

Verified 4327210e

Fixing baichuan override. (#2158)

Narsil committed 1 year ago

Verified 4f55f158

GH router. (#2153)

Narsil committed 1 year ago

Verified d0225b10

Fixing test. (#2152)

Narsil committed 1 year ago

Verified 17cebc45

fix: prefer serde structs over custom functions (#2127)

drbh committed 1 year ago

Verified 9eefb2f6

refine get xpu free memory/enable Qwen2/gemma2/gemma/phi in intel platform (#2132)

sywangyi committed 1 year ago

Verified 5da4cfab

fix AttributeError: 'MixtralLayer' object has no attribute 'mlp' (#2123)

icyxp committed 1 year ago

Verified 9d0ca503

Use GPTQ-Marlin for supported GPTQ configurations (#2111)

danieldk committed 1 year ago

Verified 2ce80194

Newer Older