huggingface/text-generation-inference

Pull Requests Commits

Fix integration mt0 (transformers update).

Narsil committed 1 year ago

Verified e3db5259

Simple updates.

Narsil committed 1 year ago

Verified 199973cc

Revert doc text.

Narsil committed 1 year ago

Verified cacaba64

Updating logic + non flash.

Narsil committed 1 year ago

Verified 6994fa12

Much simpler logic after the overhead.

Narsil committed 1 year ago

Verified 10534511

QuantLinear is rocm compatible.

Narsil committed 1 year ago

Verified 849d8821

Narsil committed 1 year ago

Verified 82a6cb82

Narsil committed 1 year ago

Verified 5c3efbc7

Trying to fix non chunking targets.

Narsil committed 1 year ago

Verified 0a01dde9

Remove generated files.

Narsil committed 1 year ago

Verified a31db047

Narsil committed 1 year ago

Verified 79469f5f

Choosing input/total tokens automatically based on available VRAM?

Narsil committed 1 year ago

Verified a1aac784

Test Marlin MoE with `desc_act=true` (#2622)

danieldk committed 1 year ago

Verified 7f54b733

Make handling of FP8 scales more consisent (#2666)

danieldk committed 1 year ago

Verified 5e0fb468

CI job. Gpt awq 4 (#2665)

Narsil committed 1 year ago

Verified 153ff374

Break cycle between the attention implementations and KV cache (#2627)

danieldk committed 1 year ago

Verified 8ec57558

fix: prefer inplace softmax to avoid copy (#2661)

drbh committed 1 year ago

Verified 5f32dea1

fix tgi-entrypoint wrapper in docker file: exec instead of spawning a child process (#2663)

oOraph committed 1 year ago

Verified 1b97e084

Simplify the `attention` function (#2609)

danieldk committed 1 year ago

Verified 59ea38cb

Support `e4m3fn` KV cache (#2655)

danieldk committed 1 year ago

Verified 5bbe1ce0

feat: prefill chunking (#2600)

OlivierDehaene committed 1 year ago

Verified a6a0c97e

Fp8 e4m3_fnuz support for rocm (#2588)

mht-sharma committed 1 year ago

Verified 704a58c8

Rollback to `ChatRequest` for Vertex AI Chat instead of `VertexChat` (#2651)

alvarobartt committed 1 year ago

Verified ffe05ccd

Use flashinfer for Gemma 2.

danieldk committed 1 year ago

ce7e3565

Fixing linters. (#2650)

Narsil committed 1 year ago

Verified cf04a43f

feat: enable pytorch xpu support for non-attention models (#2561)

dvrogozh committed 1 year ago

Verified 58848cb4

update ipex to fix incorrect output of mllama in cpu (#2640)

sywangyi committed 1 year ago

Verified 7a82ddcb

Clarify gated description and quicktour (#2631)

osanseviero committed 1 year ago

Verified 51f54018

Cpu perf (#2596)

Narsil committed 1 year ago

Verified 3ea82d00

Small fixes for supported models (#2471)

osanseviero committed 1 year ago

Verified ce28ee88

Older