lighteval
932e1f2f - Store provider credentials as SecretStr in model configs (#1326)

Commit
68 days ago
Store provider credentials as SecretStr in model configs (#1326) * Store provider credentials as SecretStr in model configs LiteLLMModelConfig.api_key and TGIModelConfig.inference_server_auth were plain str fields, which meant they were retained in plaintext wherever a model config gets serialized (e.g. EvaluationTracker.results). Switch both to pydantic SecretStr, which masks the value in reprs and default serialization, and additionally exclude them explicitly when building the results dict as a second layer. The real value is still unwrapped via get_secret_value() at the specific call sites that need it for the actual outgoing request. Added regression tests asserting the credential never appears in the serialized results dict. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Store JudgeLM credentials as SecretStr JudgeLM.api_key was a plain str consumed directly by several backend clients (OpenAI, AsyncInferenceClient, litellm). Wrap it in SecretStr on assignment and unwrap via get_secret_value() at each usage site, for consistency with the other model configs and to remove any reliance on incidental string formatting to keep it out of logs or serialized output. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Fix pre-existing ruff format drift in README and docs Unrelated cleanup so CI's Quality check is green on this branch. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Nathan Habib <nathan_habib@Mac.lan> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Author
Parents
Loading