huggingface/text-generation-inference

Pull Requests Commits

mht-sharma committed 353 days ago

b0b855fe

update dockerfile

mht-sharma committed 354 days ago

0863dd15

mht-sharma committed 354 days ago

05d0aa67

mht-sharma committed 354 days ago

57de05b0

Add fp8 support moe models

mht-sharma committed 362 days ago

16162602

flashinfer: switch to plan API (#2904)

danieldk committed 1 year ago

Verified 630f1986

Revert "feat: improve qwen2-vl startup " (#2924)

drbh committed 1 year ago

Verified 8f6146f1

feat: improve qwen2-vl startup (#2802)

drbh committed 1 year ago

Verified eecca271

fix the crash of meta-llama/Llama-3.2-1B (#2918)

sywangyi committed 1 year ago

Verified 6e982f43

Add fp8 kv cache for ROCm (#2856)

mht-sharma committed 1 year ago

Verified c20025db

Moving to `uv` instead of `poetry`. (#2919)

Narsil committed 1 year ago

Verified de19e7e8

nix: update to PyTorch 2.5.1 (#2921)

danieldk committed 1 year ago

Verified d61f14f2

Flash decoding kernel adding and prefill-chunking and prefix caching enabling in intel cpu/xpu (#2815)

sywangyi committed 1 year ago

Verified 88514416

feat: improve star coder to support multi lora layers (#2883)

drbh committed 1 year ago

Verified 82f6ea1b

Do not convert weight scale to e4m3fnuz on CUDA (#2917)

danieldk committed 1 year ago

Verified 5f78ec32

Upgrading bitsandbytes. (#2910)

Narsil committed 1 year ago

Verified 922cc38f

Removing the github runner. (#2912)

Narsil committed 1 year ago

Verified 120bd3e3

Fix typo in TPU docs (#2911)

baptistecolle committed 1 year ago

Verified 1470aec9

Upgrading our rustc version. (#2908)

Narsil committed 1 year ago

Verified 203cade2

:memo: add guide on using TPU with TGI in the docs (#2907)

baptistecolle committed 1 year ago

Verified 46994b34

Fix `docker run` in `README.md` (#2861)

alvarobartt committed 1 year ago

Verified dc9b8e98

docs(conceptual/speculation): available links Train Medusa (#2863)

guspan-tanadi committed 1 year ago

Verified 3c7ae48f

Baichuan2-13B does not have max_position_embeddings in config (#2903)

sywangyi committed 1 year ago

Verified cc8b9650

Enable FP8 Per-Tensor Scales and Integrate Marlin/MoE Kernels Repo for ROCm (#2825)

mht-sharma committed 1 year ago

Verified e07acc7f

Add Flash decoding kernel ROCm (#2855)

mht-sharma committed 1 year ago

Verified 880ab9c2

fix crash in torch2.6 if TP=1 (#2885)

sywangyi committed 1 year ago

Verified 1660154a

Update using_guidance.md (#2901)

nbroad1881 committed 1 year ago

Verified 2e22164d

Add possible variants for A100 and H100 GPUs for auto-detecting flops (#2837)

lazariv committed 1 year ago

Verified 83624a07

chore: Update jsonschema to 0.28.0 (#2870)

Stranger6667 committed 1 year ago

Verified 01067f8b

Update to marlin-kernels 0.3.7 (#2882)

danieldk committed 1 year ago

Verified 4f7e00f4

Older