huggingface/text-generation-inference

Pull Requests Commits

Fixing the updating logic of backends.

Narsil committed 333 days ago

Verified 73d0876f

Fixing add/remove/set backends.

Narsil committed 333 days ago

Verified 18cb4a42

remove useless rwlock

co42 committed 335 days ago

5831ff6e

add Dockerfile for kvrouter

co42 committed 336 days ago

40b2011b

CI must be green.

Narsil committed 1 year ago

Verified 50c8ebde

Cleaner version.

Narsil committed 1 year ago

Verified 57fa04ad

Adding kvrouter to the workspace.

Narsil committed 1 year ago

Verified 1932c5b9

Remove kvrouter from default members.

Narsil committed 1 year ago

Verified 914b1637

Updating the kvrouter to support roundrobin

Narsil committed 1 year ago

Verified 0a495ad1

Adding Dummy kvrouter.

Narsil committed 1 year ago

Verified 6a88063c

More logs in the allocator.

Narsil committed 1 year ago

Verified 7ef8b89e

doc: Update TRTLLM deployment doc. (#2960)

Hugoch committed 1 year ago

Verified 065aabb1

Add deepseekv3 (#2968)

Narsil committed 1 year ago

Verified cb747b33

Hotfixing intel-cpu (not sure how it was working before). (#2967)

Narsil committed 1 year ago

Verified 80e7d98f

Update to moe-kernels 0.8.0 (#2966)

danieldk committed 1 year ago

Verified ee0dffcd

Add fp8 support moe models (#2928)

mht-sharma committed 1 year ago

Verified 4ef2e045

Add backend name to telemetry (#2962)

Hugoch committed 1 year ago

Verified 73b7cf83

Fixing the oom maybe with 2.5.1 change. (#2958)

Narsil committed 1 year ago

Verified eb3df0f4

fix: Telemetry (#2957)

Hugoch committed 1 year ago

Verified c690da59

Update to attention-kernels 0.2.0 (#2950)

danieldk committed 1 year ago

Verified db922eb7

Attempt to remove AWS S3 flaky cache for sccache (#2953)

mfuntowicz committed 1 year ago

Verified 40b00275

Revert "Remove AWS credentials?"

Narsil committed 1 year ago

Verified 6cb41a80

Remove AWS credentials?

Narsil committed 1 year ago

Verified d2ff68e9

Trying to put back the archlist (to fix the oom). (#2947)

Narsil committed 1 year ago

Verified d9dda117

Fixing cargo lock.

Narsil committed 1 year ago

Verified d937eb64

Transformers backend TP fix (#2945)

Cyrilvallez committed 1 year ago

Verified 18c4607d

Tmp tp transformers (#2942)

Narsil committed 1 year ago

Verified 29a0893b

[TRTLLM] Expose finish reason (#2841)

mfuntowicz committed 1 year ago

Verified 0a899026

Add NVIDIA A40 to known cards (#2941)

kldzj committed 1 year ago

Verified 4e172028

Set `alias` for `max_completion_tokens` in `ChatRequest` (#2932)

alvarobartt committed 1 year ago

Verified 6ab02931

Older