huggingface/text-generation-inference

Pull Requests Commits

feat(router): drop permit after batching

OlivierDehaene committed 1 year ago

Verified 344427b6

chore: update to torch 2.4 (#2259)

OlivierDehaene committed 1 year ago

Verified e7e3aa6c

hotfix: pin numpy (#2289)

danieldk committed 1 year ago

Verified bc9593a5

Add support for Llama 3 rotary embeddings (#2286)

danieldk committed 1 year ago

Verified 4ab41737

Preparing for release. (#2285)

Narsil committed 1 year ago

Verified 5d121a97

[WIP] Add support for Mistral-Nemo by supporting head_dim through config (#2254)

shaltielshmid committed 1 year ago

Verified 3961e323

Add support for repacking AWQ weights for GPTQ-Marlin (#2278)

danieldk committed 1 year ago

Verified 9935720c

fix(l4): fix fp8 logic on l4 (#2277)

OlivierDehaene committed 1 year ago

Verified 5fca30ee

Fixing mistral nemo. (#2276)

Narsil committed 1 year ago

Verified abc32537

use proper name for ci (#2274)

XciD committed 1 year ago

Verified 47004651

Softcapping for gemma2. (#2273)

Narsil committed 1 year ago

Verified 6aeb6690

fix(server): fix fp8 weight loading (#2268)

OlivierDehaene committed 1 year ago

Verified 4844ff79

fix(ci): test new instances (#2272)

XciD committed 1 year ago

Verified 6aebf44f

legacy warning on text_generation client (#2271)

ErikKaum committed 1 year ago

Verified 07441f5a

Hotfix: fix of use of unquantized weights in Mixtral GQA loading (#2269)

icyxp committed 1 year ago

Verified 4e420722

fix(server): fix deepseekv2 loading (#2266)

OlivierDehaene committed 1 year ago

Verified f3435bab

feat(fp8): use fbgemm kernels and load fp8 weights directly (#2248)

OlivierDehaene committed 1 year ago

Verified 53ec0b79

Add FP8 release test (#2261)

danieldk committed 1 year ago

Verified e5c1d6d6

re-push to internal registry (#2242)

XciD committed 1 year ago

Verified 11123a8e

Add support for Deepseek V2 (#2224)

danieldk committed 1 year ago

Verified e52be9bb

fix: adjust default tool choice (#2244)

drbh committed 1 year ago

Verified 68a9685f

add usage stats to toctree (#2260)

ErikKaum committed 1 year ago

Verified 40f5dc3e

usage stats and crash reports (#2220)

ErikKaum committed 1 year ago

Verified 4c19593a

Hotfix: pass through model revision in `VlmCausalLM` (#2258)

danieldk committed 1 year ago

Verified 3f37a667

Hotfix: fix MPT after recent refactor (#2257)

danieldk committed 1 year ago

Verified 3b41e93a

Hotfix: various GPT-based model fixes (#2256)

danieldk committed 1 year ago

Verified 18db78f2

Hotfix: fix of use of unquantized weights in Gemma GQA loading (#2255)

danieldk committed 1 year ago

Verified 80adb5be

Improve the handling of quantized weights (#2250)

danieldk committed 1 year ago

Verified ba291dad

fix(server): fix cohere (#2249)

OlivierDehaene committed 1 year ago

Verified 1d1b1efa

Remove stray `quantize` argument in `get_weights_col_packed_qkv` (#2237)

danieldk committed 1 year ago

Verified da82c63a

Older