huggingface/text-generation-inference

Pull Requests Commits

feat: return streaming errors as an event formatted for openai's client (#2668)

drbh committed 1 year ago

Verified 6489f852

Upgrading our deps. (#2750)

Narsil committed 1 year ago

Verified 34a3bded

Upgrade outlines to 0.1.1 (#2742)

aW3st committed 1 year ago

Verified 4580ced0

fix response type of document for Text Generation Inference (#2743)

jitokim committed 1 year ago

Verified 003eaec0

Fix: Change embeddings to embedding (#2738)

mokeddembillel committed 1 year ago

Verified 4f4857a4

Fix: Change model_type from ssm to mamba (#2740)

mokeddembillel committed 1 year ago

Verified f9ee46f7

benchmark: fix prefill throughput (#2741)

danieldk committed 1 year ago

Verified 8442f1ac

nix: update nixpkgs (#2746)

danieldk committed 1 year ago

Verified ca4f46dd

Add initial support for compressed-tensors checkpoints (#2732)

danieldk committed 1 year ago

Verified a7850008

add trust_remote_code in tokenizer to fix baichuan issue (#2725)

sywangyi committed 1 year ago

Verified 97f7a22f

fix incorrect output of Qwen2-7B-Instruct-GPTQ-Int4 and Qwen2-7B-Inst… (#2717)

sywangyi committed 1 year ago

Verified b1f9044d

nix: move to tgi-nix `main` (#2718)

danieldk committed 1 year ago

Verified 5eedb2ec

Fixing linting on main. (#2719)

Narsil committed 1 year ago

Verified 9fde5666

Fix prefix caching + speculative decoding (#2711)

tgaddair committed 1 year ago

Verified aadc9cb4

Hotfixing auto length (warmup max_s was wrong). (#2716)

Narsil committed 1 year ago

Verified a5593ba8

fix: add chat_tokenize endpoint to api docs (#2710)

drbh committed 1 year ago

Verified 08c4184e

fix: create position ids for text only input (#2714)

drbh committed 1 year ago

Verified 6e322052

fix cuda graphs for qwen2-vl (#2708)

drbh committed 1 year ago

Verified 01dacf8e

Support qwen2 vl (#2689)

drbh committed 1 year ago

Verified befd9f67

add xpu triton in dockerfile, or will show "Could not import Flash At… (#2702)

sywangyi committed 1 year ago

Verified 46aeb086

Monkey patching as a desperate measure. (#2704)

Narsil committed 1 year ago

Verified 98330df6

More timeout on docker start ? (#2701)

Narsil committed 1 year ago

Verified 513d19b9

Fixing auto bloom test. (#2699)

Narsil committed 1 year ago

Verified 3a9cdc32

Update poetry lock. (#2698)

Narsil committed 1 year ago

Verified 78ce618c

We can have a tokenizer anywhere. (#2527)

Narsil committed 1 year ago

Verified 90b226db

Choosing input/total tokens automatically based on available VRAM? (#2673)

Narsil committed 1 year ago

Verified 0c9b6cdd

Green main (#2697)

Narsil committed 1 year ago

Verified 2e4f4ba1

Avoiding timeout for bloom tests. (#2693)

Narsil committed 1 year ago

Verified 8a8794a6

chore: prepare 2.4.0 release (#2695)

OlivierDehaene committed 1 year ago

Verified a6b02da9

feat: add triton kernels to decrease latency of large batches (#2687)

OlivierDehaene committed 1 year ago

Verified 6f88bd93

Newer Older