transformers
8636b309 - Fix chat CLI GPU loading and request_id validation issues (#40230) (#40232)

Commit
253 days ago
Fix chat CLI GPU loading and request_id validation issues (#40230) (#40232) * Fix chat CLI GPU loading and request_id validation issues (#40230) This commit addresses two critical bugs in the transformers chat CLI: 1. **GPU Loading Issue**: Changed default device from "cpu" to "auto" in ChatArguments - Chat CLI now automatically uses GPU when available instead of defaulting to CPU - Matches the behavior of the underlying serving infrastructure 2. **Request ID Validation Error**: Added request_id field to TransformersCompletionCreateParamsStreaming schema - Fixes "Unexpected keys in the request: {'request_id'}" error on second message - Allows request_id to be properly sent and validated by the server Both fixes target the exact root causes identified in issue #40230: - Users will now get GPU acceleration by default when available - Chat sessions will no longer break after the second message * Remove unrelated request_id field from TransformersCompletionCreateParamsStreaming
Author
Parents
Loading