huggingface/text-generation-inference

Pull Requests Commits
fix moe in quantization path (#2935)
sywangyi committed 333 days ago
Verified 1d3c9beb
Upgrading the deps to have transformers==4.48.0 necessary (#2937)
Narsil committed 333 days ago
Verified 2dfe3b3e
Run `pre-commit run --all-files` to fix CI (#2933)
alvarobartt committed 334 days ago
Verified 64a33c1f
Trying to avoid the random timeout. (#2929)
Narsil committed 334 days ago
Verified bdb3e488
Give TensorRT-LLMa proper CI/CD 😍 (#2886)
mfuntowicz committed 334 days ago
Verified 17367438
Flash Transformers modeling backend support (#2913)
Cyrilvallez committed 334 days ago
Verified b980848a
Fixing TRTLLM dockerfile. (#2922)
Narsil committed 335 days ago
Verified 447a5b2f
flashinfer: switch to plan API (#2904)
danieldk committed 338 days ago
Verified 630f1986
Revert "feat: improve qwen2-vl startup " (#2924)
drbh committed 338 days ago
Verified 8f6146f1
feat: improve qwen2-vl startup (#2802)
drbh committed 338 days ago
Verified eecca271
fix the crash of meta-llama/Llama-3.2-1B (#2918)
sywangyi committed 338 days ago
Verified 6e982f43
Add fp8 kv cache for ROCm (#2856)
mht-sharma committed 338 days ago
Verified c20025db
Moving to `uv` instead of `poetry`. (#2919)
Narsil committed 338 days ago
Verified de19e7e8
nix: update to PyTorch 2.5.1 (#2921)
danieldk committed 338 days ago
Verified d61f14f2
Flash decoding kernel adding and prefill-chunking and prefix caching enabling in intel cpu/xpu (#2815)
sywangyi committed 338 days ago
Verified 88514416
feat: improve star coder to support multi lora layers (#2883)
drbh committed 339 days ago
Verified 82f6ea1b
Do not convert weight scale to e4m3fnuz on CUDA (#2917)
danieldk committed 339 days ago
Verified 5f78ec32
Upgrading bitsandbytes. (#2910)
Narsil committed 340 days ago
Verified 922cc38f
Removing the github runner. (#2912)
Narsil committed 340 days ago
Verified 120bd3e3
Fix typo in TPU docs (#2911)
baptistecolle committed 340 days ago
Verified 1470aec9
Upgrading our rustc version. (#2908)
Narsil committed 340 days ago
Verified 203cade2
:memo: add guide on using TPU with TGI in the docs (#2907)
baptistecolle committed 340 days ago
Verified 46994b34
Fix `docker run` in `README.md` (#2861)
alvarobartt committed 340 days ago
Verified dc9b8e98
docs(conceptual/speculation): available links Train Medusa (#2863)
guspan-tanadi committed 340 days ago
Verified 3c7ae48f
Baichuan2-13B does not have max_position_embeddings in config (#2903)
sywangyi committed 340 days ago
Verified cc8b9650
Enable FP8 Per-Tensor Scales and Integrate Marlin/MoE Kernels Repo for ROCm (#2825)
mht-sharma committed 340 days ago
Verified e07acc7f
Add Flash decoding kernel ROCm (#2855)
mht-sharma committed 342 days ago
Verified 880ab9c2
fix crash in torch2.6 if TP=1 (#2885)
sywangyi committed 342 days ago
Verified 1660154a
Update using_guidance.md (#2901)
nbroad1881 committed 342 days ago
Verified 2e22164d
Add possible variants for A100 and H100 GPUs for auto-detecting flops (#2837)
lazariv committed 345 days ago
Verified 83624a07
Terms Of Use Privacy Policy Refund Policy Impressum