huggingface/text-generation-inference

Pull Requests Commits

feat(metrics): remove ngrok mandatory feature for backendv3 crate

mfuntowicz committed 1 year ago

f72547c9

feat(metrics): dispatch internal engine state event from queuing/batching tasks

mfuntowicz committed 1 year ago

712199c7

feat(metrics): update Cargo.lock

mfuntowicz committed 1 year ago

1a9c5dec

feat(metrics): exposes the engine state as an endpoint

mfuntowicz committed 1 year ago

8de41f63

feat(metrics): exposes queue size as tokens along with individual requests count

mfuntowicz committed 1 year ago

bb8f5963

Avoid running neuron integration tests twice (#3054)

dacorvo committed 1 year ago

Verified 5eec3a8b

fix: run linters and fix formatting (#3057)

drbh committed 1 year ago

Verified b0069e04

some minor fix (#3048)

sywangyi committed 1 year ago

Verified d7a24c03

You need to seek apparently. (#3049)

Narsil committed 1 year ago

Verified cea9dbc9

Add Neuron backend (#3033)

dacorvo committed 1 year ago

Verified c00add9c

Use `rotary` kernel from the Hub (#3041)

danieldk committed 1 year ago

Verified 97c5f7e6

Improve tool call message processing (#3036)

drbh committed 1 year ago

Verified 1cae3197

Update Gradio ChatInterface configuration in consuming_tgi.md (#3042)

angt committed 1 year ago

Verified 3498f608

Simplify logs2. (#3045)

Narsil committed 1 year ago

Verified 142a49a8

fix qwen2 vl crash in continous batching (#3004)

sywangyi committed 1 year ago

Verified 06dfe9ab

flashinfer 0.2.0.post1 -> post2 (#3040)

danieldk committed 1 year ago

Verified ed96ba65

update ipex and torch to 2.6 for cpu (#3039)

sywangyi committed 1 year ago

Verified feaa2477

feat: Add the parsing of HF_HUB_USER_AGENT_ORIGIN environment variable for telemetry (#3027)

Hugoch committed 1 year ago

Verified 230aa256

Having less logs in case of failure for checking CI more easily. (#3037)

Narsil committed 1 year ago

Verified 9c89d007

Using public external registry (to use external runners for CI). (#3031)

Narsil committed 1 year ago

Verified fde3234c

feat: add initial qwen2.5-vl model and test (#2971)

drbh committed 1 year ago

Verified d6a0c67e

Improve Transformers support (#2970)

Cyrilvallez committed 1 year ago

Verified a7448661

It's find in some machine. using hf_hub::api::sync::Api to download c… (#3030)

Narsil committed 1 year ago

Verified 5543fdc7

Pinning trufflehog. (#3032)

Narsil committed 1 year ago

Verified b8a4928d

Add `loop_controls` feature to `minijinja` to handle `{% break %}` (#2998)

alvarobartt committed 1 year ago

Verified 8a1cfd61

Update README.md (#3024)

celsowm committed 1 year ago

Verified 794ec58b

Use eetq kernel from the hub (#3029)

danieldk committed 1 year ago

Verified f0ed7658

[Backend] Add Llamacpp backend (#2975)

angt committed 1 year ago

Verified cfd4fbb4

Support sigmoid scoring function in GPTQ-MoE (#3017)

danieldk committed 1 year ago

Verified 6df0fc0b

Putting back the NCCL forced upgrade. (#2999)

Narsil committed 1 year ago

Verified d6881c37

Older