Add reasoning effort configuration for capable models (#2252)
* feat(reasoning): add per-model thinking-effort selector
Add support for the OpenAI-compatible `reasoning_effort` parameter so
users can control how deeply reasoning-capable models think (low /
medium / high) on every chat. The HF router proxies the param to
underlying providers (gpt-oss, Kimi-K2-Thinking, GLM-4.5+, DeepSeek
R1/V3.1+, Qwen3-*-Thinking, MiniMax M-series, etc.) which honor or
silently ignore it.
- New `supportsReasoning` flag on the model config (Zod) + YAML
annotations on each reasoning-capable entry in chart/env/{prod,dev}.yaml.
- New per-model user setting `reasoningEffortOverrides`. Omitting a
value falls back to the provider default (no `reasoning_effort` is
sent).
- New `ThinkingEffortChip` rendered in the chat footer (opposite the
Model chip) when the active model supports reasoning. Dropdown writes
to settings and the choice persists per model.
- Threaded `reasoningEffort` through `TextGenerationContext` ->
`EndpointParameters` -> `endpointOai` and the MCP completion path.
- Cohere clamp: `command-a-reasoning` only accepts `low`, so the helper
`cohereSafeEffort` downgrades medium/high when the resolved provider
is `cohere`.
https://claude.ai/code/session_01VNC3Kh8bzmuZJ9bLNo1SJW
* fix(reasoning): clamp Cohere effort by model id; thread effort into MCP flow
Address Codex review on #2252:
- `cohereSafeEffort` now clamps to `low` for `command-a-reasoning-08-2025`
by model id, not just when `provider === "cohere"`. The provider field
is undefined / "auto" in the common case, so the previous check left
medium/high to 422 against Cohere.
- `runMcpFlow` was given a `reasoningEffort` parameter but the caller in
`textGeneration/index.ts` never passed `ctx.reasoningEffort`. Forward
it now so MCP-backed generations honor the user's effort selection.
* refactor(reasoning): drop Cohere clamp; remove command-a-reasoning from list
Cohere's command-a-reasoning only accepts reasoning_effort=low and 422s
on medium/high. Rather than maintain a per-model clamp helper, just
don't expose the model as reasoning-capable. Removes:
- src/lib/server/textGeneration/reasoningEffort.ts (cohereSafeEffort)
- supportsReasoning flag on CohereLabs/command-a-reasoning-08-2025 in
prod.yaml and dev.yaml
reasoning_effort is now sent verbatim by both endpointOai and the MCP
flow.
* fix(reasoning): persist effort immediately to avoid 300ms debounce race
settings.update goes through a 300ms debounce before posting to the
server, while the conversation route reads the value from MongoDB on
every send. Switching the chip to settings.instantSet awaits the POST
so a quick "toggle effort then send" flow uses the chosen value.
* chore(skill): teach add-model-descriptions to flag reasoning models
Updates the skill so when new models are pulled from the HF router it
also decides whether to set "supportsReasoning": true on each new entry,
based on the underlying providers' chat-completions documentation.
Existing entries are explicitly out of scope to avoid overwriting hand
tuning.
* style: prettier formatting after merge
* chore(reasoning): trim supportsReasoning to verified-working models
After auditing each currently-flagged model with low vs high
reasoning_effort on two reasoning-heavy prompts, only ~12 actually
scale their chain-of-thought depth as the OpenAI-style param requests.
The rest either ignore the param, behave inversely, or no longer have
a routable reasoning provider on the HF router.
Keep the flag on:
openai/gpt-oss-{120b,20b,safeguard-20b}
MiniMaxAI/MiniMax-M2 / M2.5 / M2.7
moonshotai/Kimi-K2-Thinking
zai-org/GLM-4.7 / GLM-4.7-FP8 / GLM-4.7-Flash
zai-org/GLM-5.1 / GLM-5.1-FP8
And add it to deepseek-ai/DeepSeek-V4-Pro (clearly scales).
Removes the flag from 33 entries (DeepSeek V3.x, R1 + R1 distills,
Qwen3-Thinking variants, MiMo-V2, MiniMax-M1/M2.1, all GLM-4.5*/4.6*,
GLM-5, cogito*, DictaLM-Thinking, Prover-V2, QwQ-32B). The chip stops
appearing for them, which matches what users were already observing —
the dropdown was visible but had no perceptible effect.
* feat(reasoning): add per-model reasoning toggle in model settings
Add a "Reasoning effort" capability toggle on /settings/<model> that
mirrors the existing Tool calling / Multimodal pattern. Backed by a new
per-model reasoningOverrides setting that falls back to the model's
configured supportsReasoning flag.
- Self-hosted operators can now flip the toggle to expose the chat-
footer effort dropdown for any model (or hide it on a flagged one).
- HuggingChat keeps the YAML config as the source of truth: the toggle
is disabled in the UI and the POST handlers strip any reasoningOverrides
to match the existing tools/multimodal HuggingChat behavior.
- ChatWindow's chip and the conversation POST route both gate on
(override ?? supportsReasoning), so visibility and the actual
reasoning_effort send go through the same predicate.
---------
Co-authored-by: Claude <noreply@anthropic.com>
