next.js

5ff5ae21 - feat(trace-server): add query CLI and MCP API to turbopack-trace-server (#92030)

feat(trace-server): add query CLI and MCP API to turbopack-trace-server (#92030) ### What? Adds a [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) HTTP server and a companion CLI to the Turbopack trace server (`next internal trace`), so AI agents and developers can explore trace data from a build or dev session incrementally through a structured tool API. Two new user-facing features: 1. **`--mcp-port` flag on `next internal trace`** — starts an MCP-over-HTTP endpoint (at `/mcp`) alongside the existing WebSocket trace viewer. 2. **`next internal query-trace`** — a standalone CLI command that queries a running trace server from the terminal, using the same MCP endpoint under the hood. ### Why? Trace files from Turbopack builds can be very large. An agent querying raw trace data would be overwhelmed by tens of thousands of spans. The MCP API exposes the data incrementally: - **Pagination** (20 spans per page) prevents context overflow. - **Aggregation** groups repeated spans by name (same logic as the WebSocket viewer's graph), so agents see one entry per span type with count/total/avg statistics rather than thousands of identical entries. - **Drill-down** lets agents pick a span ID from one response and fetch its children in the next call. - **Search** filters spans by name/args using `SpanRef::search` (full subtree search index, same as the WebSocket viewer). - **Sort** orders by corrected (wall-clock) duration to surface the slowest spans first. - **Output format** — `outputType: "json"` for machine-readable structured data, `"markdown"` (default) for human-readable rendering. ### How? #### Rust (`turbopack/crates/turbopack-trace-server/src/lib.rs`) - `start_turbopack_trace_server_non_blocking(path, port) -> Arc<StoreContainer>` — starts the reader and WebSocket server on background threads and returns the store handle immediately (previously the function blocked forever). - `QueryOptions` — struct with `parent`, `aggregated`, `sort`, `search`, `page` fields. - `query_spans(store, opts) -> QueryResult` — queries up to 20 spans per page. In aggregated mode it uses the existing `SpanGraphRef` graph logic (the same grouping the WebSocket viewer uses). Waits up to 10s for the store to finish initial data loading on the first call. - Helper functions extracted: `paginate()`, `format_span_name()`, `build_span_id()` to avoid duplication between aggregated and raw code paths. #### Span ID format IDs encode both the span type and the navigation path: | Span kind | Leaf format | Example full path | |---|---|---| | Raw span | `<index>` (decimal) | `a1-a5-20` | | Aggregated span | `a<first-span-index>` | `a1-a5-a34` | Segments are joined by `-` as the caller drills deeper. `resolve_span_by_id` only needs the last segment to look up the underlying store index. #### NAPI bridge (`crates/next-napi-bindings/src/turbo_trace_server.rs`) - `TraceServerHandle` — opaque NAPI class wrapping `Arc<StoreContainer>`. - `startTurbopackTraceServerHandle(path, port)` — calls the non-blocking Rust function. - `queryTraceSpans(handle, options)` — calls `query_spans` and returns a plain JS object. - Doc comments on `SpanInfo` fields clarify that `cpu_duration`/`corrected_duration` hold the first example span's values for aggregated groups (while `total_*` fields hold group totals). #### TypeScript (`packages/next/src/...`) - `generated-native.d.ts` / `types.ts` / `index.ts` — type declarations, native wrappers, and WASM stubs. - `next.ts` — adds `--mcp-port <port>` to `next internal trace` (defaults to `5748`); registers `next internal query-trace` subcommand with `--json` flag. - `turbo-trace-server.ts` — rewrites the CLI handler to: 1. Start the WebSocket trace viewer server non-blocking (was blocking). 2. Start an HTTP server at `/mcp` running MCP `StreamableHTTPServerTransport` (stateless — one transport per request, `sessionIdGenerator: undefined`). 3. `renderSpanMarkdown()` — extracted helper that renders a single span as markdown. 4. Error handling: `server.on('error')` for `EADDRINUSE`, `try/catch` around `loadBindings()` and `startTurbopackTraceServerHandle()`. - `query-trace.ts` — new CLI command that POSTs JSON-RPC to the MCP endpoint and prints the response. On connection failure, shows instructions to start `next internal trace` first. Supports `--json` flag for structured output. #### `next internal query-trace` CLI ``` Usage: next internal query-trace [options] Options: --port <port> MCP port of the running trace server. Defaults to 5748. --parent <parent> Span ID to enumerate children of. Omit for root level. --no-aggregated Disable aggregation of spans by name (aggregated by default). --sort Sort results by corrected duration descending (default: false). --search <search> Substring filter on span name/category. --json Output as JSON instead of markdown. --page <page> Page number (1-based, default 1). -h, --help Displays this message. ``` #### Startup output When `next internal trace` starts with `--mcp-port`: ``` Turbopack trace server started. View trace at https://trace.nextjs.org?port=5747 Query this trace from the command line: next internal query-trace --help Alternatively, connect an MCP client to http://127.0.0.1:5748/mcp ``` #### Markdown output format (per span) ```markdown ### `<name>` (ID: `<id>`) - CPU Duration: … - Corrected Duration: … - Start (relative to parent): … - End (relative to parent): … **Attributes:** - `key`: value ``` For aggregated spans with count > 1, totals and averages are shown first, then one example span's raw data. ### Tests **E2E test (`test/e2e/turbopack-trace-server-query/turbopack-trace-server.test.ts`)** Uses `nextTestSetup` with `env: { NEXT_TURBOPACK_TRACING: '1' }` to produce a real trace file. Spawns `next internal trace <file> --mcp-port <port>`, waits for the MCP server to be ready, then runs both MCP HTTP and CLI queries: - Root listing (markdown format) - Aggregation mode - Pagination - Drill-down by span ID (using JSON output to extract IDs) - Search with a real span name match and a non-matching term - Sort by duration - JSON output format (`outputType: 'json'`) - CLI: `--sort`, `--search`, `--no-aggregated`, `--parent`, `--json` - Error path: connection failure message with instructions Turbopack-only (skips via `if (!isTurbopack) return` at describe level). The error-path test runs regardless of bundler. ### Usage ```bash # Build with tracing NEXT_TURBOPACK_TRACING=1 next build # Start the trace viewer + MCP server next internal trace .next-profiles/trace-turbopack --mcp-port 5748 # Query from CLI next internal query-trace --sort next internal query-trace --parent a1 --sort next internal query-trace --search "turbo_tasks::function" --page 2 next internal query-trace --no-aggregated next internal query-trace --json # Or connect an MCP client to http://127.0.0.1:5748/mcp ``` <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>