text-generation-inference
Disable `decoder_input_details` on OpenAI-compatible chat streaming, pass temp and top-k from API
#1470

Merged

Disable `decoder_input_details` on OpenAI-compatible chat streaming, pass temp and top-k from API #1470

drbh merged 2 commits into huggingface:main from fix-oai-stream

Disable decoder_input_details for streaming requests

43479601

Transparently pass through temp and top_p

d805612c

drbh approved these changes on 2024-01-23

drbh merged 82f87ada into main 2 years ago

EndlessReform deleted the fix-oai-stream branch 2 years ago

Reviewers

drbh

Assignees

No one assigned

Labels

None yet

Milestone

No milestone