huggingface/text-generation-inference

Pull Requests Commits

backend(vllm): disable metrics for now

mfuntowicz committed 1 year ago

5452c129

backend(vllm): map ResultOutput to InferStreamResponse to stream back to the client

mfuntowicz committed 1 year ago

003163a2

backend(vllm): expose FFI for CompletionOutput and RequestOutput on Rust side

mfuntowicz committed 1 year ago

32dffcff

backend(vllm): make v1 the default

mfuntowicz committed 1 year ago

7028f5bc

backend(vllm): remove python print stmt

mfuntowicz committed 1 year ago

dc5addae

backend(vllm): submit new request to vLLM engine

mfuntowicz committed 1 year ago

a7c2a470

backend(vllm): plug in the tokio server and CLI

mfuntowicz committed 1 year ago

02e4b9ab

backend(vllm): statically allocate LLMEngine

mfuntowicz committed 1 year ago

bd2ec03d

backend(vllm): initial commit

mfuntowicz committed 1 year ago

cfd22726

Run `pre-commit run --all-files` to fix CI (#2933)

alvarobartt committed 1 year ago

Verified 64a33c1f

Trying to avoid the random timeout. (#2929)

Narsil committed 1 year ago

Verified bdb3e488

Give TensorRT-LLMa proper CI/CD 😍 (#2886)

mfuntowicz committed 1 year ago

Verified 17367438

Flash Transformers modeling backend support (#2913)

Cyrilvallez committed 1 year ago

Verified b980848a

Fixing TRTLLM dockerfile. (#2922)

Narsil committed 1 year ago

Verified 447a5b2f

flashinfer: switch to plan API (#2904)

danieldk committed 1 year ago

Verified 630f1986

Revert "feat: improve qwen2-vl startup " (#2924)

drbh committed 1 year ago

Verified 8f6146f1

feat: improve qwen2-vl startup (#2802)

drbh committed 1 year ago

Verified eecca271

fix the crash of meta-llama/Llama-3.2-1B (#2918)

sywangyi committed 1 year ago

Verified 6e982f43

Add fp8 kv cache for ROCm (#2856)

mht-sharma committed 1 year ago

Verified c20025db

Moving to `uv` instead of `poetry`. (#2919)

Narsil committed 1 year ago

Verified de19e7e8

nix: update to PyTorch 2.5.1 (#2921)

danieldk committed 1 year ago

Verified d61f14f2

Flash decoding kernel adding and prefill-chunking and prefix caching enabling in intel cpu/xpu (#2815)

sywangyi committed 1 year ago

Verified 88514416

feat: improve star coder to support multi lora layers (#2883)

drbh committed 1 year ago

Verified 82f6ea1b

Do not convert weight scale to e4m3fnuz on CUDA (#2917)

danieldk committed 1 year ago

Verified 5f78ec32

Upgrading bitsandbytes. (#2910)

Narsil committed 1 year ago

Verified 922cc38f

Removing the github runner. (#2912)

Narsil committed 1 year ago

Verified 120bd3e3

Fix typo in TPU docs (#2911)

baptistecolle committed 1 year ago

Verified 1470aec9

Upgrading our rustc version. (#2908)

Narsil committed 1 year ago

Verified 203cade2

:memo: add guide on using TPU with TGI in the docs (#2907)

baptistecolle committed 1 year ago

Verified 46994b34

Fix `docker run` in `README.md` (#2861)

alvarobartt committed 1 year ago

Verified dc9b8e98

Older