huggingface/text-generation-inference

Pull Requests Commits

fix: run pre commit lints

drbh committed 1 year ago

a604bfe4

Fix prefix caching + speculative decoding

tgaddair committed 1 year ago

Verified 3bafa0eb

Support qwen2 vl (#2689)

drbh committed 1 year ago

Verified befd9f67

add xpu triton in dockerfile, or will show "Could not import Flash At… (#2702)

sywangyi committed 1 year ago

Verified 46aeb086

Monkey patching as a desperate measure. (#2704)

Narsil committed 1 year ago

Verified 98330df6

More timeout on docker start ? (#2701)

Narsil committed 1 year ago

Verified 513d19b9

Fixing auto bloom test. (#2699)

Narsil committed 1 year ago

Verified 3a9cdc32

Update poetry lock. (#2698)

Narsil committed 1 year ago

Verified 78ce618c

We can have a tokenizer anywhere. (#2527)

Narsil committed 1 year ago

Verified 90b226db

Choosing input/total tokens automatically based on available VRAM? (#2673)

Narsil committed 1 year ago

Verified 0c9b6cdd

Green main (#2697)

Narsil committed 1 year ago

Verified 2e4f4ba1

Avoiding timeout for bloom tests. (#2693)

Narsil committed 1 year ago

Verified 8a8794a6

chore: prepare 2.4.0 release (#2695)

OlivierDehaene committed 1 year ago

Verified a6b02da9

feat: add triton kernels to decrease latency of large batches (#2687)

OlivierDehaene committed 1 year ago

Verified 6f88bd93

Switch from fbgemm-gpu w8a8 scaled matmul to vLLM/marlin-kernels (#2688)

danieldk committed 1 year ago

Verified 0f346a32

Add support for stop words in TRTLLM (#2678)

mfuntowicz committed 1 year ago

Verified ba5fc7d9

Fixing mt0 test. (#2692)

Narsil committed 1 year ago

Verified db68bd05

Fixing rocm gptq by using triton code too (renamed cuda into triton). (#2691)

Narsil committed 1 year ago

Verified cece8635

[TENSORRT-LLM] - Implement new looper thread based backend (#2357)

mfuntowicz committed 1 year ago

Verified 43df056e

Fixing "deadlock" when python prompts for trust_remote_code by always (#2664)

Narsil committed 1 year ago

Verified ed87b464

Add support for FP8 KV cache scales (#2628)

danieldk committed 1 year ago

Verified eab07f74

Fix Phi 3.5 MoE tests (#2684)

danieldk committed 1 year ago

Verified 14a0df3a

flashinfer: reminder to remove contiguous call in the future (#2685)

danieldk committed 1 year ago

Verified 1b914f37

feat: allow any supported payload on /invocations (#2683)

OlivierDehaene committed 1 year ago

Verified 41c26237

hotfix: fix flashllama

OlivierDehaene committed 1 year ago

Verified 27ff1871

feat: natively support Granite models (#2682)

OlivierDehaene committed 1 year ago

Verified 03c9388b

Make moe-kernels and marlin-kernels mandatory in CUDA installs (#2632)

danieldk committed 1 year ago

Verified f58eb70e

Add `impureWithCuda` dev shell (#2677)

danieldk committed 1 year ago

Verified 9c9ef37c

break when there's nothing to read (#2582)

sywangyi committed 1 year ago

Verified 058d3061

Test Marlin MoE with `desc_act=true` (#2622)

danieldk committed 1 year ago

Verified 7f54b733

Older