huggingface/text-generation-inference

Pull Requests Commits

Clean both threads.

Narsil committed 1 year ago

Verified fe8d55db

Fixing performance degradation on Intel.

Narsil committed 1 year ago

Verified 009c4e0b

Test Marlin MoE with `desc_act=true` (#2622)

danieldk committed 1 year ago

Verified 7f54b733

Make handling of FP8 scales more consisent (#2666)

danieldk committed 1 year ago

Verified 5e0fb468

CI job. Gpt awq 4 (#2665)

Narsil committed 1 year ago

Verified 153ff374

Break cycle between the attention implementations and KV cache (#2627)

danieldk committed 1 year ago

Verified 8ec57558

fix: prefer inplace softmax to avoid copy (#2661)

drbh committed 1 year ago

Verified 5f32dea1

fix tgi-entrypoint wrapper in docker file: exec instead of spawning a child process (#2663)

oOraph committed 1 year ago

Verified 1b97e084

Simplify the `attention` function (#2609)

danieldk committed 1 year ago

Verified 59ea38cb

Support `e4m3fn` KV cache (#2655)

danieldk committed 1 year ago

Verified 5bbe1ce0

feat: prefill chunking (#2600)

OlivierDehaene committed 1 year ago

Verified a6a0c97e

Fp8 e4m3_fnuz support for rocm (#2588)

mht-sharma committed 1 year ago

Verified 704a58c8

Rollback to `ChatRequest` for Vertex AI Chat instead of `VertexChat` (#2651)

alvarobartt committed 1 year ago

Verified ffe05ccd

Use flashinfer for Gemma 2.

danieldk committed 1 year ago

ce7e3565

Fixing linters. (#2650)

Narsil committed 1 year ago

Verified cf04a43f

feat: enable pytorch xpu support for non-attention models (#2561)

dvrogozh committed 1 year ago

Verified 58848cb4

update ipex to fix incorrect output of mllama in cpu (#2640)

sywangyi committed 1 year ago

Verified 7a82ddcb

Clarify gated description and quicktour (#2631)

osanseviero committed 1 year ago

Verified 51f54018

Cpu perf (#2596)

Narsil committed 1 year ago

Verified 3ea82d00

Small fixes for supported models (#2471)

osanseviero committed 1 year ago

Verified ce28ee88

Fixing intel Supports windowing. (#2637)

Narsil committed 1 year ago

Verified 0c478846

Intel ci (#2630)

Narsil committed 1 year ago

Verified 3dbdf63e

Update documentation to most recent stable version of TGI. (#2625)

Vaibhavs10 committed 1 year ago

Verified d912f0bf

feat: allow tool calling to respond without a tool (#2614)

drbh committed 1 year ago

Verified e36dfaa8

Narsil committed 1 year ago

Verified 43f39f68

nix: add black and isort to the closure (#2619)

danieldk committed 1 year ago

Verified 9ed0c85f

CI (2599): Update ToolType input schema (#2601)

drbh committed 1 year ago

Verified 8ad20daf

nix: move back to the tgi-nix main branch (#2620)

danieldk committed 1 year ago

Verified 6db3bcb7

Add support for fused MoE Marlin for AWQ (#2616)

danieldk committed 1 year ago

Verified 64142489

Upgrade minor rust version (Fixes rust build compilation cache) (#2617)

Narsil committed 1 year ago

Verified 8b295aa4

Older