huggingface/text-generation-inference

Pull Requests Commits

Update transformers_flash_causal_lm.py

Cyrilvallez committed 354 days ago

Verified bafbd067

tp monkey patch

Cyrilvallez committed 354 days ago

Verified de83178b

Revert "Remove AWS credentials?"

Narsil committed 354 days ago

Verified 6cb41a80

Remove AWS credentials?

Narsil committed 354 days ago

Verified d2ff68e9

Trying to put back the archlist (to fix the oom). (#2947)

Narsil committed 354 days ago

Verified d9dda117

Fixing cargo lock.

Narsil committed 355 days ago

Verified d937eb64

Transformers backend TP fix (#2945)

Cyrilvallez committed 355 days ago

Verified 18c4607d

Tmp tp transformers (#2942)

Narsil committed 355 days ago

Verified 29a0893b

[TRTLLM] Expose finish reason (#2841)

mfuntowicz committed 355 days ago

Verified 0a899026

Add NVIDIA A40 to known cards (#2941)

kldzj committed 355 days ago

Verified 4e172028

Set `alias` for `max_completion_tokens` in `ChatRequest` (#2932)

alvarobartt committed 355 days ago

Verified 6ab02931

Bump TensorRT-LLM backend dependency to v0.16.0 (#2931)

mfuntowicz committed 355 days ago

Verified cc212154

Clarify FP8-Marlin use on capability 8.9 (#2940)

danieldk committed 356 days ago

Verified 1dd34666

fix moe in quantization path (#2935)

sywangyi committed 356 days ago

Verified 1d3c9beb

Upgrading the deps to have transformers==4.48.0 necessary (#2937)

Narsil committed 356 days ago

Verified 2dfe3b3e

Run `pre-commit run --all-files` to fix CI (#2933)

alvarobartt committed 357 days ago

Verified 64a33c1f

Trying to avoid the random timeout. (#2929)

Narsil committed 357 days ago

Verified bdb3e488

Give TensorRT-LLMa proper CI/CD 😍 (#2886)

mfuntowicz committed 357 days ago

Verified 17367438

Flash Transformers modeling backend support (#2913)

Cyrilvallez committed 357 days ago

Verified b980848a

Fixing TRTLLM dockerfile. (#2922)

Narsil committed 358 days ago

Verified 447a5b2f

flashinfer: switch to plan API (#2904)

danieldk committed 361 days ago

Verified 630f1986

Revert "feat: improve qwen2-vl startup " (#2924)

drbh committed 361 days ago

Verified 8f6146f1

feat: improve qwen2-vl startup (#2802)

drbh committed 361 days ago

Verified eecca271

fix the crash of meta-llama/Llama-3.2-1B (#2918)

sywangyi committed 361 days ago

Verified 6e982f43

Add fp8 kv cache for ROCm (#2856)

mht-sharma committed 361 days ago

Verified c20025db

Moving to `uv` instead of `poetry`. (#2919)

Narsil committed 361 days ago

Verified de19e7e8

nix: update to PyTorch 2.5.1 (#2921)

danieldk committed 361 days ago

Verified d61f14f2

Flash decoding kernel adding and prefill-chunking and prefix caching enabling in intel cpu/xpu (#2815)

sywangyi committed 361 days ago

Verified 88514416

feat: improve star coder to support multi lora layers (#2883)

drbh committed 362 days ago

Verified 82f6ea1b

Do not convert weight scale to e4m3fnuz on CUDA (#2917)

danieldk committed 362 days ago

Verified 5f78ec32

Older