huggingface/text-generation-inference

Pull Requests Commits

Apply suggestions from code review

sywangyi committed 1 year ago

Verified 53b6f6e6

Merge branch 'main' into moe

sywangyi committed 1 year ago

Verified e0e39fa0

Remove vLLM dependency for CUDA (#2751)

danieldk committed 1 year ago

Verified 52e48739

feat: return streaming errors as an event formatted for openai's client (#2668)

drbh committed 1 year ago

Verified 6489f852

Upgrading our deps. (#2750)

Narsil committed 1 year ago

Verified 34a3bded

Upgrade outlines to 0.1.1 (#2742)

aW3st committed 1 year ago

Verified 4580ced0

fix response type of document for Text Generation Inference (#2743)

jitokim committed 1 year ago

Verified 003eaec0

Fix: Change embeddings to embedding (#2738)

mokeddembillel committed 1 year ago

Verified 4f4857a4

Fix: Change model_type from ssm to mamba (#2740)

mokeddembillel committed 1 year ago

Verified f9ee46f7

benchmark: fix prefill throughput (#2741)

danieldk committed 1 year ago

Verified 8442f1ac

nix: update nixpkgs (#2746)

danieldk committed 1 year ago

Verified ca4f46dd

Add initial support for compressed-tensors checkpoints (#2732)

danieldk committed 1 year ago

Verified a7850008

fix oneapi basekit version

sywangyi committed 1 year ago

1639152c

add trust_remote_code in tokenizer to fix baichuan issue (#2725)

sywangyi committed 1 year ago

Verified 97f7a22f

torch has xpu support in 2.5

sywangyi committed 1 year ago

6becab5d

Merge branch 'main' into moe

sywangyi committed 1 year ago

3d4c50f0

fix incorrect output of Qwen2-7B-Instruct-GPTQ-Int4 and Qwen2-7B-Inst… (#2717)

sywangyi committed 1 year ago

Verified b1f9044d

nix: move to tgi-nix `main` (#2718)

danieldk committed 1 year ago

Verified 5eedb2ec

Fixing linting on main. (#2719)

Narsil committed 1 year ago

Verified 9fde5666

Fix prefix caching + speculative decoding (#2711)

tgaddair committed 1 year ago

Verified aadc9cb4

Hotfixing auto length (warmup max_s was wrong). (#2716)

Narsil committed 1 year ago

Verified a5593ba8

update to ipex xpu 2.5

sywangyi committed 1 year ago

11080516

fix: add chat_tokenize endpoint to api docs (#2710)

drbh committed 1 year ago

Verified 08c4184e

Merge branch 'main' into moe

sywangyi committed 1 year ago

d2f28687

fix: create position ids for text only input (#2714)

drbh committed 1 year ago

Verified 6e322052

fix cuda graphs for qwen2-vl (#2708)

drbh committed 1 year ago

Verified 01dacf8e

Support qwen2 vl (#2689)

drbh committed 1 year ago

Verified befd9f67

add xpu triton in dockerfile, or will show "Could not import Flash At… (#2702)

sywangyi committed 1 year ago

Verified 46aeb086

add ipex moe implementation to support Mixtral and PhiMoe

sywangyi committed 1 year ago

d9a8bbc1

Monkey patching as a desperate measure. (#2704)

Narsil committed 1 year ago

Verified 98330df6

Older