huggingface/text-generation-inference

Pull Requests Commits

More logs in the allocator.

Narsil committed 330 days ago

Verified c871d74b

Attempt to remove AWS S3 flaky cache for sccache (#2953)

mfuntowicz committed 331 days ago

Verified 40b00275

Revert "Remove AWS credentials?"

Narsil committed 334 days ago

Verified 6cb41a80

Remove AWS credentials?

Narsil committed 334 days ago

Verified d2ff68e9

Trying to put back the archlist (to fix the oom). (#2947)

Narsil committed 334 days ago

Verified d9dda117

Fixing cargo lock.

Narsil committed 335 days ago

Verified d937eb64

Transformers backend TP fix (#2945)

Cyrilvallez committed 335 days ago

Verified 18c4607d

Tmp tp transformers (#2942)

Narsil committed 335 days ago

Verified 29a0893b

[TRTLLM] Expose finish reason (#2841)

mfuntowicz committed 335 days ago

Verified 0a899026

Add NVIDIA A40 to known cards (#2941)

kldzj committed 335 days ago

Verified 4e172028

Set `alias` for `max_completion_tokens` in `ChatRequest` (#2932)

alvarobartt committed 335 days ago

Verified 6ab02931

Bump TensorRT-LLM backend dependency to v0.16.0 (#2931)

mfuntowicz committed 335 days ago

Verified cc212154

Clarify FP8-Marlin use on capability 8.9 (#2940)

danieldk committed 336 days ago

Verified 1dd34666

fix moe in quantization path (#2935)

sywangyi committed 336 days ago

Verified 1d3c9beb

Upgrading the deps to have transformers==4.48.0 necessary (#2937)

Narsil committed 336 days ago

Verified 2dfe3b3e

Run `pre-commit run --all-files` to fix CI (#2933)

alvarobartt committed 337 days ago

Verified 64a33c1f

Trying to avoid the random timeout. (#2929)

Narsil committed 337 days ago

Verified bdb3e488

Give TensorRT-LLMa proper CI/CD 😍 (#2886)

mfuntowicz committed 337 days ago

Verified 17367438

Flash Transformers modeling backend support (#2913)

Cyrilvallez committed 337 days ago

Verified b980848a

Fixing TRTLLM dockerfile. (#2922)

Narsil committed 338 days ago

Verified 447a5b2f

flashinfer: switch to plan API (#2904)

danieldk committed 341 days ago

Verified 630f1986

Revert "feat: improve qwen2-vl startup " (#2924)

drbh committed 341 days ago

Verified 8f6146f1

feat: improve qwen2-vl startup (#2802)

drbh committed 341 days ago

Verified eecca271

fix the crash of meta-llama/Llama-3.2-1B (#2918)

sywangyi committed 341 days ago

Verified 6e982f43

Add fp8 kv cache for ROCm (#2856)

mht-sharma committed 341 days ago

Verified c20025db

Moving to `uv` instead of `poetry`. (#2919)

Narsil committed 341 days ago

Verified de19e7e8

nix: update to PyTorch 2.5.1 (#2921)

danieldk committed 341 days ago

Verified d61f14f2

Flash decoding kernel adding and prefill-chunking and prefix caching enabling in intel cpu/xpu (#2815)

sywangyi committed 341 days ago

Verified 88514416

feat: improve star coder to support multi lora layers (#2883)

drbh committed 342 days ago

Verified 82f6ea1b

Do not convert weight scale to e4m3fnuz on CUDA (#2917)

danieldk committed 342 days ago

Verified 5f78ec32

Older