huggingface/text-generation-inference

Pull Requests Commits

Update transformers_flash_causal_lm.py

Cyrilvallez committed 1 year ago

Verified bafbd067

tp monkey patch

Cyrilvallez committed 1 year ago

Verified de83178b

Revert "Remove AWS credentials?"

Narsil committed 1 year ago

Verified 6cb41a80

Remove AWS credentials?

Narsil committed 1 year ago

Verified d2ff68e9

Trying to put back the archlist (to fix the oom). (#2947)

Narsil committed 1 year ago

Verified d9dda117

Fixing cargo lock.

Narsil committed 1 year ago

Verified d937eb64

Transformers backend TP fix (#2945)

Cyrilvallez committed 1 year ago

Verified 18c4607d

Tmp tp transformers (#2942)

Narsil committed 1 year ago

Verified 29a0893b

[TRTLLM] Expose finish reason (#2841)

mfuntowicz committed 1 year ago

Verified 0a899026

Add NVIDIA A40 to known cards (#2941)

kldzj committed 1 year ago

Verified 4e172028

Set `alias` for `max_completion_tokens` in `ChatRequest` (#2932)

alvarobartt committed 1 year ago

Verified 6ab02931

Bump TensorRT-LLM backend dependency to v0.16.0 (#2931)

mfuntowicz committed 1 year ago

Verified cc212154

Clarify FP8-Marlin use on capability 8.9 (#2940)

danieldk committed 1 year ago

Verified 1dd34666

fix moe in quantization path (#2935)

sywangyi committed 1 year ago

Verified 1d3c9beb

Upgrading the deps to have transformers==4.48.0 necessary (#2937)

Narsil committed 1 year ago

Verified 2dfe3b3e

Run `pre-commit run --all-files` to fix CI (#2933)

alvarobartt committed 1 year ago

Verified 64a33c1f

Trying to avoid the random timeout. (#2929)

Narsil committed 1 year ago

Verified bdb3e488

Give TensorRT-LLMa proper CI/CD 😍 (#2886)

mfuntowicz committed 1 year ago

Verified 17367438

Flash Transformers modeling backend support (#2913)

Cyrilvallez committed 1 year ago

Verified b980848a

Fixing TRTLLM dockerfile. (#2922)

Narsil committed 1 year ago

Verified 447a5b2f

flashinfer: switch to plan API (#2904)

danieldk committed 1 year ago

Verified 630f1986

Revert "feat: improve qwen2-vl startup " (#2924)

drbh committed 1 year ago

Verified 8f6146f1

feat: improve qwen2-vl startup (#2802)

drbh committed 1 year ago

Verified eecca271

fix the crash of meta-llama/Llama-3.2-1B (#2918)

sywangyi committed 1 year ago

Verified 6e982f43

Add fp8 kv cache for ROCm (#2856)

mht-sharma committed 1 year ago

Verified c20025db

Moving to `uv` instead of `poetry`. (#2919)

Narsil committed 1 year ago

Verified de19e7e8

nix: update to PyTorch 2.5.1 (#2921)

danieldk committed 1 year ago

Verified d61f14f2

Flash decoding kernel adding and prefill-chunking and prefix caching enabling in intel cpu/xpu (#2815)

sywangyi committed 1 year ago

Verified 88514416

feat: improve star coder to support multi lora layers (#2883)

drbh committed 1 year ago

Verified 82f6ea1b

Do not convert weight scale to e4m3fnuz on CUDA (#2917)

danieldk committed 1 year ago

Verified 5f78ec32

Older