huggingface/text-generation-inference

Pull Requests Commits

Narsil committed 231 days ago

Verified 338cdc2e

Fixing the Trie in case of exact prefix match split.

Narsil committed 231 days ago

Verified 3aa85646

CI must be green.

Narsil committed 320 days ago

Verified 50c8ebde

Cleaner version.

Narsil committed 320 days ago

Verified 57fa04ad

Adding kvrouter to the workspace.

Narsil committed 320 days ago

Verified 1932c5b9

Remove kvrouter from default members.

Narsil committed 320 days ago

Verified 914b1637

Updating the kvrouter to support roundrobin

Narsil committed 320 days ago

Verified 0a495ad1

Adding Dummy kvrouter.

Narsil committed 320 days ago

Verified 6a88063c

More logs in the allocator.

Narsil committed 320 days ago

Verified 7ef8b89e

doc: Update TRTLLM deployment doc. (#2960)

Hugoch committed 321 days ago

Verified 065aabb1

Add deepseekv3 (#2968)

Narsil committed 321 days ago

Verified cb747b33

Hotfixing intel-cpu (not sure how it was working before). (#2967)

Narsil committed 322 days ago

Verified 80e7d98f

Update to moe-kernels 0.8.0 (#2966)

danieldk committed 322 days ago

Verified ee0dffcd

Add fp8 support moe models (#2928)

mht-sharma committed 322 days ago

Verified 4ef2e045

Add backend name to telemetry (#2962)

Hugoch committed 323 days ago

Verified 73b7cf83

Fixing the oom maybe with 2.5.1 change. (#2958)

Narsil committed 323 days ago

Verified eb3df0f4

fix: Telemetry (#2957)

Hugoch committed 323 days ago

Verified c690da59

Update to attention-kernels 0.2.0 (#2950)

danieldk committed 324 days ago

Verified db922eb7

Attempt to remove AWS S3 flaky cache for sccache (#2953)

mfuntowicz committed 324 days ago

Verified 40b00275

Revert "Remove AWS credentials?"

Narsil committed 327 days ago

Verified 6cb41a80

Remove AWS credentials?

Narsil committed 327 days ago

Verified d2ff68e9

Trying to put back the archlist (to fix the oom). (#2947)

Narsil committed 327 days ago

Verified d9dda117

Fixing cargo lock.

Narsil committed 328 days ago

Verified d937eb64

Transformers backend TP fix (#2945)

Cyrilvallez committed 328 days ago

Verified 18c4607d

Tmp tp transformers (#2942)

Narsil committed 328 days ago

Verified 29a0893b

[TRTLLM] Expose finish reason (#2841)

mfuntowicz committed 328 days ago

Verified 0a899026

Add NVIDIA A40 to known cards (#2941)

kldzj committed 328 days ago

Verified 4e172028

Set `alias` for `max_completion_tokens` in `ChatRequest` (#2932)

alvarobartt committed 328 days ago

Verified 6ab02931

Bump TensorRT-LLM backend dependency to v0.16.0 (#2931)

mfuntowicz committed 328 days ago

Verified cc212154

Clarify FP8-Marlin use on capability 8.9 (#2940)

danieldk committed 329 days ago

Verified 1dd34666

Older