feat(langchain_v1): on_model_call middleware (#33328)
Introduces a generator-based `on_model_call` hook that allows middleware
to intercept model calls with support for retry logic, error handling,
response transformation, and request modification.
## Overview
Middleware can now implement `on_model_call()` using a generator
protocol that:
- **Yields** `ModelRequest` to execute the model
- **Receives** `AIMessage` via `.send()` on success, or exception via
`.throw()` on error
- **Yields again** to retry or transform responses
- Uses **implicit last-yield semantics** (no return values from
generators)
## Usage Examples
### Basic Retry on Error
```python
from langchain.agents.middleware.types import AgentMiddleware
class RetryMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
for attempt in range(3):
try:
yield request # Execute model
break # Success
except Exception:
if attempt == 2:
raise # Max retries exceeded
```
### Response Transformation
```python
class UppercaseMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
result = yield request
modified = AIMessage(content=result.content.upper())
yield modified # Return transformed response
```
### Error Recovery
```python
class FallbackMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
try:
yield request
except Exception:
fallback = AIMessage(content="Service unavailable")
yield fallback # Convert error to fallback response
```
### Caching / Short-Circuit
```python
class CacheMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
if cached := get_cache(request):
yield cached # Skip model execution
else:
result = yield request
save_cache(request, result)
```
### Request Modification
```python
class SystemPromptMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
modified_request = ModelRequest(
model=request.model,
system_prompt="You are a helpful assistant.",
messages=request.messages,
tools=request.tools,
)
yield modified_request
```
### Function Decorator
```python
from langchain.agents.middleware.types import on_model_call
@on_model_call
def retry_three_times(request, state, runtime):
for attempt in range(3):
try:
yield request
break
except Exception:
if attempt == 2:
raise
agent = create_agent(model="openai:gpt-4o", middleware=[retry_three_times])
```
## Middleware Composition
Middleware compose with first in list as outermost layer:
```python
agent = create_agent(
model="openai:gpt-4o",
middleware=[
RetryMiddleware(), # Outer - wraps others
LoggingMiddleware(), # Middle
UppercaseMiddleware(), # Inner - closest to model
]
)
```
