Gap Analysis — TanStack AI adapter audit

You are auditing TanStack AI's provider adapters against each provider's upstream documentation. This is a maintainer tool. Your only output is a markdown report under .agent/gap-analysis/. Do not edit source files.

Invocation

Workflow

1. Parse scope. If missing, AskUserQuestion with the four options above.
2. Load the truth files, then read the per-scope inputs you need:
   • Matrix: testing/e2e/src/lib/feature-support.ts
   • Types: testing/e2e/src/lib/types.ts (Provider + Feature unions, ALL_PROVIDERS, ALL_FEATURES)
   • Adapter index: packages/ai-<provider>/src/index.ts
   • Model meta: packages/ai-<provider>/src/model-meta.ts
   • Core types: packages/ai/src/types.ts (Modality, ContentPart, ToolCall)
3. Research upstream. Use WebFetch against the curated URLs in references/provider-doc-urls.md. When a doc page has moved, fall back to WebSearch. For SDK API surface details use the context7 MCP server (mcp__plugin_context7_context7__resolve-library-id then mcp__plugin_context7_context7__query-docs).
4. Walk the audit dimensions in references/audit-checklist.md:
   1. New models
   2. Cross-adapter feature parity
   3. Untracked features
   4. Capability-flag drift
   5. Telemetry / observability parity (usage tokens, cache/reasoning counts, request ids, logging asymmetry)
   6. Activity coverage (which of the 7 core activity kinds each provider ships an adapter for vs. what upstream supports) — this is the only dimension for the activities scope; it's also rolled into --all.
5. Fan out for --all: launch one Explore subagent per provider, max 3 in parallel. Each subagent returns the multi-dimension findings for its provider; you synthesise into the combined report. The activities scope does not fan out — derive the provider×activity matrix centrally from the adapter files (see dimension 6), since it's a fast mechanical diff.
6. Write the report to .agent/gap-analysis/YYYY-MM-DD-<scope>.md using references/report-template.md. Date is today's ISO date. <scope> is openai / feature-tts / models / activities / all.
7. Print the report path and a 5-line summary to the user.

Critical rules

1. Never edit feature-support.ts or any model-meta.ts. The report is read-only — the maintainer applies changes.
2. Always reference line numbers when citing exclusions (e.g., feature-support.ts:57) so the maintainer can jump to them.
3. Distinguish three gap classes in the report:
   • Real gap — upstream supports it, TanStack AI doesn't, no exclusion comment.
   • Tested gap — TanStack AI doesn't list it but there's an exclusion comment in feature-support.ts (e.g., aimock format limitation). Not actionable code-wise; surface in "Out-of-scope" section.
   • Stale capability flag — model-meta.ts declares a capability the model no longer has, or omits one it now has.
4. Cite sources. Every claim "upstream supports X" must link the upstream doc page you read. No claims from training data.
5. Use today's date from the system context (currentDate). Never invent.
6. Quote the relevant snippet from feature-support.ts when flagging a parity gap, so the report is self-contained.

Known providers

openai, anthropic, gemini, ollama, grok, groq, openrouter, bedrock (@tanstack/ai-bedrock; three-API surface — Converse default (adapter name bedrock-converse), Chat Completions opt-in (api: 'chat', adapter name bedrock), Responses opt-in (api: 'responses', adapter name bedrock-responses)), fal (media-only), elevenlabs (TTS-only). The feature matrix tracks openai, anthropic, gemini, ollama, grok, groq, openrouter, bedrock, bedrock-converse, and bedrock-responses; fal and elevenlabs only appear in model/media audits.

Known features (19)

Canonical list is ALL_FEATURES in testing/e2e/src/lib/types.ts — always re-read it; this list is a snapshot:

chat, one-shot-text, reasoning, multi-turn, tool-calling, parallel-tool-calls, tool-approval, text-tool-text, structured-output, structured-output-stream, agentic-structured, multimodal-image, multimodal-structured, summarize, summarize-stream, image-gen, tts, transcription, video-gen.

Known activities (7)

Features (above) are matrix rows about behaviours within an activity. Activities are the coarser-grained core capability kinds in @tanstack/ai — each has a Base<Kind>Adapter and a provider "supports" one only if its package ships an adapter of that kind. Canonical list is the AdapterKind union in packages/ai/src/activities/index.ts — always re-read it:

text, summarize, image, audio, video, tts, transcription.

A provider's activity surface is derived mechanically from its adapter files: packages/ai-<provider>/src/adapters/. Filename → activity-kind map:

(cost.ts is a helper, not an activity adapter.)

Verification before finishing

Before printing the summary:

• Report file exists and is non-empty.
• git status shows only new files under .agent/gap-analysis/ — nothing under packages/ or testing/ should have been modified. Run git status and confirm.
• Every "real gap" entry has an upstream doc URL.

Note: This skill requires familiarity with ai-core and ai-core/chat-experience. Code Mode is always used on top of a chat experience.

Setup

Complete Code Mode setup with Node.js isolate driver:

import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { createCodeModeTool } from '@tanstack/ai-code-mode'
import { createNodeIsolateDriver } from '@tanstack/ai-isolate-node'
import { toolDefinition } from '@tanstack/ai'
import { z } from 'zod'

// Define a tool that code can call
const fetchWeather = toolDefinition({
  name: 'fetchWeather',
  description: 'Get current weather for a city',
  inputSchema: z.object({ city: z.string() }),
  outputSchema: z.object({ temp: z.number(), condition: z.string() }),
}).server(async ({ city }) => {
  const res = await fetch(`https://api.weather.com/${city}`)
  return res.json()
})

// Create code mode tool with Node isolate
const codeModeTool = createCodeModeTool({
  driver: createNodeIsolateDriver({
    memoryLimit: 128,
    timeout: 30000,
  }),
  tools: [fetchWeather],
})

// Use in chat
const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  tools: [codeModeTool],
})

return toServerSentEventsResponse(stream)

The recommended higher-level entry point is createCodeMode(), which returns both the tool and a matching system prompt:

import { chat } from '@tanstack/ai'
import { createCodeMode } from '@tanstack/ai-code-mode'
import { createNodeIsolateDriver } from '@tanstack/ai-isolate-node'
import { openaiText } from '@tanstack/ai-openai'

const { tool, systemPrompt } = createCodeMode({
  driver: createNodeIsolateDriver(),
  tools: [fetchWeather],
  timeout: 30_000,
})

const stream = chat({
  adapter: openaiText('gpt-4o'),
  systemPrompts: ['You are a helpful assistant.', systemPrompt],
  tools: [tool],
  messages,
})

createCodeMode calls createCodeModeTool and createCodeModeSystemPrompt internally. The system prompt includes generated TypeScript type stubs for each tool so the LLM writes correct calls.

Core Patterns

1. Choosing an Isolate Driver

Three drivers implement the IsolateDriver interface. All are interchangeable.

Node.js (createNodeIsolateDriver) -- Full V8 with JIT. Fastest option. Requires isolated-vm native C++ addon.

import { createNodeIsolateDriver } from '@tanstack/ai-isolate-node'

const driver = createNodeIsolateDriver({
  memoryLimit: 128, // MB, default 128
  timeout: 30_000, // ms, default 30000
  // skipProbe: false -- set true only after verifying compatibility
})

QuickJS (createQuickJSIsolateDriver) -- WASM-based, no native deps. Works in Node.js, browsers, Deno, Bun, and edge runtimes. Slower (interpreted, no JIT). Limited stdlib (no File I/O).

import { createQuickJSIsolateDriver } from '@tanstack/ai-isolate-quickjs'

const driver = createQuickJSIsolateDriver({
  memoryLimit: 128, // MB, default 128
  timeout: 30_000, // ms, default 30000
  maxStackSize: 524288, // bytes, default 512 KiB
})

Cloudflare (createCloudflareIsolateDriver) -- Edge execution via a deployed Cloudflare Worker. Requires a workerUrl pointing to your deployed worker. Network latency on each tool call.

import { createCloudflareIsolateDriver } from '@tanstack/ai-isolate-cloudflare'

const driver = createCloudflareIsolateDriver({
  workerUrl: 'https://my-code-mode-worker.my-account.workers.dev',
  authorization: process.env.CODE_MODE_WORKER_SECRET,
  timeout: 30_000, // ms, default 30000
  maxToolRounds: 10, // max tool-call/result cycles, default 10
})

2. Adding Persistent Skills with codeModeWithSkills()

Skills let the LLM save reusable code snippets. On future requests, relevant skills are loaded and exposed as callable tools.

import { chat, maxIterations } from '@tanstack/ai'
import { createNodeIsolateDriver } from '@tanstack/ai-isolate-node'
import { codeModeWithSkills } from '@tanstack/ai-code-mode-skills'
import { createFileSkillStorage } from '@tanstack/ai-code-mode-skills/storage'
import {
  createDefaultTrustStrategy,
  createAlwaysTrustedStrategy,
  createCustomTrustStrategy,
} from '@tanstack/ai-code-mode-skills'
import { openaiText } from '@tanstack/ai-openai'

// Trust strategies control how skills earn trust through executions
// Default: untrusted -> provisional (10+ runs, >=90%) -> trusted (100+ runs, >=95%)
// Relaxed: untrusted -> provisional (3+ runs, >=80%) -> trusted (10+ runs, >=90%)
// Always trusted: immediately trusted (dev/testing)
// Custom: configurable thresholds
const trustStrategy = createDefaultTrustStrategy()

// Storage options: file system (production) or memory (testing)
const storage = createFileSkillStorage({
  directory: './.skills',
  trustStrategy,
})

const driver = createNodeIsolateDriver()

// High-level API: automatic LLM-based skill selection
const { toolsRegistry, systemPrompt, selectedSkills } =
  await codeModeWithSkills({
    config: {
      driver,
      tools: [myTool1, myTool2],
      timeout: 60_000,
      memoryLimit: 128,
    },
    adapter: openaiText('gpt-4o-mini'), // cheap model for skill selection
    skills: {
      storage,
      maxSkillsInContext: 5,
    },
    messages,
  })

const stream = chat({
  adapter: openaiText('gpt-4o'),
  tools: toolsRegistry.getTools(),
  messages,
  systemPrompts: ['You are a helpful assistant.', systemPrompt],
  agentLoopStrategy: maxIterations(15),
})

The registry includes: execute_typescript, search_skills, get_skill, register_skill, and one tool per selected skill.

Custom trust strategy example:

const strategy = createCustomTrustStrategy({
  initialLevel: 'untrusted',
  provisionalThreshold: { executions: 5, successRate: 0.85 },
  trustedThreshold: { executions: 50, successRate: 0.95 },
})

Storage implementations:

// File storage (production) -- persists skills as files on disk
import { createFileSkillStorage } from '@tanstack/ai-code-mode-skills/storage'
const fileStorage = createFileSkillStorage({ directory: './.skills' })

// Memory storage (testing) -- in-memory, lost on restart
import { createMemorySkillStorage } from '@tanstack/ai-code-mode-skills/storage'
const memStorage = createMemorySkillStorage()

3. Client-Side Execution Progress Display

Code Mode emits custom events during sandbox execution. Handle them in useChat via onCustomEvent.

Events emitted:

import { useCallback, useRef, useState } from 'react'
import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'

interface VMEvent {
  id: string
  eventType: string
  data: unknown
  timestamp: number
}

export function CodeModeChat() {
  const [toolCallEvents, setToolCallEvents] = useState<
    Map<string, Array<VMEvent>>
  >(new Map())
  const eventIdCounter = useRef(0)

  const handleCustomEvent = useCallback(
    (
      eventType: string,
      data: unknown,
      context: { toolCallId?: string },
    ) => {
      const { toolCallId } = context
      if (!toolCallId) return

      const event: VMEvent = {
        id: `event-${eventIdCounter.current++}`,
        eventType,
        data,
        timestamp: Date.now(),
      }

      setToolCallEvents((prev) => {
        const next = new Map(prev)
        const events = next.get(toolCallId) || []
        next.set(toolCallId, [...events, event])
        return next
      })
    },
    [],
  )

  const { messages, sendMessage, isLoading } = useChat({
    connection: fetchServerSentEvents('/api/chat'),
    onCustomEvent: handleCustomEvent,
  })

  return (
    <div>
      {messages.map((message) => (
        <div key={message.id}>
          {message.parts.map((part) => {
            if (part.type === 'text') {
              return <p key={part.id}>{part.content}</p>
            }
            if (
              part.type === 'tool-call' &&
              part.name === 'execute_typescript'
            ) {
              const events = toolCallEvents.get(part.id) || []
              return (
                <div key={part.id}>
                  <pre>{JSON.parse(part.arguments)?.typescriptCode}</pre>
                  {events.map((evt) => (
                    <div key={evt.id}>
                      {evt.eventType}: {JSON.stringify(evt.data)}
                    </div>
                  ))}
                  {part.output && (
                    <pre>{JSON.stringify(part.output, null, 2)}</pre>
                  )}
                </div>
              )
            }
            return null
          })}
        </div>
      ))}
    </div>
  )
}

The onCustomEvent callback signature is identical across all framework integrations (@tanstack/ai-react, @tanstack/ai-solid, @tanstack/ai-vue, @tanstack/ai-svelte):

(eventType: string, data: unknown, context: { toolCallId?: string }) => void

Skill-specific events (when using codeModeWithSkills):

4. Lazy Tools

When a large tool catalog would bloat the execute_typescript system prompt, mark low-priority tools lazy: true. Lazy tools are kept out of the full type-stub documentation and listed in a compact "Discoverable APIs" catalog instead. All sandbox bindings are always injected — lazy defers documentation, not callability.

Marking a tool lazy:

import { toolDefinition } from '@tanstack/ai'
import { z } from 'zod'

const rarelyUsedTool = toolDefinition({
  name: 'fetchStocks',
  description: 'Get stock prices for a ticker. Returns a price quote.',
  inputSchema: z.object({ ticker: z.string() }),
  outputSchema: z.object({ price: z.number() }),
  lazy: true, // <-- opt out of full system-prompt documentation
}).server(async ({ ticker }) => {
  // ...
  return { price: 0 }
})

createCodeMode return shape:

createCodeMode() returns { tool, discoveryTool, tools, systemPrompt }. When lazy tools are present discoveryTool is a discover_tools server tool; otherwise it is null. Always spread tools (not just tool) into chat() so the discovery tool is registered:

import { chat } from '@tanstack/ai'
import { createCodeMode } from '@tanstack/ai-code-mode'
import { createNodeIsolateDriver } from '@tanstack/ai-isolate-node'
import { openaiText } from '@tanstack/ai-openai'

const { tools, systemPrompt } = createCodeMode({
  driver: createNodeIsolateDriver(),
  tools: [eagerTool, rarelyUsedTool], // rarelyUsedTool has lazy: true
})

const stream = chat({
  adapter: openaiText('gpt-5.5'),
  systemPrompts: ['You are a helpful assistant.', systemPrompt],
  tools: [...tools, ...otherTools], // spread tools, not just tool
  messages,
})

tools equals [tool] when there are no lazy tools (backward compatible) and [tool, discoveryTool] when lazy tools exist.

discover_tools flow:

When the model encounters a lazy tool it has not seen before, it calls discover_tools with the bare name (no external_ prefix). The tool returns each requested tool's TypeScript type stub and description. The model then writes correctly-typed external_<name> calls inside execute_typescript.

Model sees: "Discoverable APIs: external_fetchStocks"
Model calls: discover_tools({ toolNames: ["fetchStocks"] })
Response:    { tools: [{ name: "external_fetchStocks", description: "...", typeStub: "declare function external_fetchStocks(...)" }] }
Model writes inside execute_typescript: const result = await external_fetchStocks({ ticker: "AAPL" })

lazyToolsConfig.includeDescription:

Control how much of each lazy tool's description appears in the Discoverable APIs catalog (the pre-discovery list):

const { tools, systemPrompt } = createCodeMode({
  driver: createNodeIsolateDriver(),
  tools: [eagerTool, rarelyUsedTool],
  lazyToolsConfig: { includeDescription: 'first-sentence' },
})

The same lazyToolsConfig option is accepted by plain chat() for its own lazy-tool discovery catalog (see ai-core/tool-calling/SKILL.md).

Common Mistakes

CRITICAL: Passing API keys or secrets to the sandbox environment

Code Mode executes LLM-generated code. Any secrets available in the sandbox context are accessible to generated code, which could exfiltrate them via tool calls. Never pass API keys, database credentials, or tokens into the sandbox. Keep secrets in your tool server implementations, which run in the host process outside the sandbox.

Wrong:

const codeModeTool = createCodeModeTool({
  driver,
  tools: [
    toolDefinition({
      name: 'callApi',
      inputSchema: z.object({ url: z.string(), apiKey: z.string() }),
      outputSchema: z.any(),
    }).server(async ({ url, apiKey }) =>
      fetch(url, {
        headers: { Authorization: apiKey },
      }),
    ),
  ],
})

Right:

const codeModeTool = createCodeModeTool({
  driver,
  tools: [
    toolDefinition({
      name: 'callApi',
      inputSchema: z.object({ url: z.string() }),
      outputSchema: z.any(),
    }).server(async ({ url }) =>
      fetch(url, {
        headers: { Authorization: process.env.API_KEY }, // secret stays in host
      }),
    ),
  ],
})

Source: docs/code-mode/code-mode.md

HIGH: Not setting timeout for code execution

LLM-generated code may contain infinite loops. The default timeout is 30s, but developers may override to 0 (no timeout). Always set an explicit, finite timeout.

Wrong:

const driver = createNodeIsolateDriver({ timeout: 0 })

Right:

const driver = createNodeIsolateDriver({ timeout: 30_000 })

Source: ai-code-mode source (default timeout in CodeModeToolConfig)

HIGH: Using Node isolated-vm driver without checking platform compatibility

isolated-vm requires native module compilation. An incompatible build (wrong Node.js version, missing build tools) causes segfaults that no JS error handling can catch. The driver runs a subprocess probe by default. Never set skipProbe: true unless you have independently verified compatibility. Use probeIsolatedVm() to check before creating the driver.

import {
  createNodeIsolateDriver,
  probeIsolatedVm,
} from '@tanstack/ai-isolate-node'

const probe = probeIsolatedVm()
if (!probe.compatible) {
  console.error('isolated-vm not compatible:', probe.error)
  // Fall back to QuickJS
}

// Never do this unless you verified compatibility yourself:
// const driver = createNodeIsolateDriver({ skipProbe: true })

Source: ai-isolate-node source (probeIsolatedVm implementation)

MEDIUM: Expecting identical behavior across isolate drivers

The three drivers have different capabilities. Same code may work in Node but fail elsewhere.

• Node: Full V8 support, JIT compilation, configurable memory limit
• QuickJS: Interpreted, limited stdlib (no File I/O), configurable stack size, asyncified execution (serialized through global queue)
• Cloudflare: Network latency per tool call round-trip, maxToolRounds limit (default 10), requires deployed worker with UNSAFE_EVAL or eval unsafe binding

Test generated code against your target driver. If you need portability, target QuickJS's subset.

Source: docs/code-mode/code-mode-isolates.md

Cross-References

• See also: ai-core/tool-calling/SKILL.md -- Code Mode is an alternative to standard tool calling for complex multi-step operations
• See also: ai-core/chat-experience/SKILL.md -- Code Mode requires handling custom events in useChat

@tanstack/ai-mcp

This skill covers the @tanstack/ai-mcp package. Read ai-core/tool-calling/SKILL.md first — MCP tools flow into chat() the same way hand-written tools do.

When to use this package

Use @tanstack/ai-mcp when:

• A third-party MCP server exposes tools you want an agent or chat loop to call.
• You want to read MCP server resources (files, text, data) or prompts into a chat() message list.
• You want generated TypeScript types for an external MCP server's tool signatures (via the bundled generate CLI).
• You are running tool execution on the server side and want to connect to MCP servers with HTTP (Streamable HTTP or SSE) or stdio transports.

Do NOT use this package for browser/client-side code — MCP connections are server-side only.

Install

pnpm add @tanstack/ai-mcp

The package has two subpath exports:

• . — main client API (createMCPClient, createMCPClients, converters, types)
• ./stdio — Node-only stdio transport factory (stdioTransport); import it separately so edge bundles stay clean

createMCPClient — single server

import { createMCPClient } from '@tanstack/ai-mcp'

const client = await createMCPClient({
  transport: { type: 'http', url: 'https://mcp.example.com/mcp' },
  prefix: 'weather', // optional: prefixes all tool names (e.g. 'weather_get_forecast')
  name: 'my-app', // optional: client identity sent to the server
})

createMCPClient connects immediately and returns an MCPClient. Throws MCPConnectionError if the connection fails.

Transports

Streamable HTTP (default for internet-facing servers)

const client = await createMCPClient({
  transport: {
    type: 'http',
    url: 'https://mcp.example.com/mcp',
    headers: { Authorization: 'Bearer sk-...' },
  },
})

SSE

const client = await createMCPClient({
  transport: {
    type: 'sse',
    url: 'https://mcp.example.com/sse',
    headers: { Authorization: 'Bearer sk-...' },
  },
})

stdio (Node-only — import from /stdio subpath)

import { createMCPClient } from '@tanstack/ai-mcp'
import { stdioTransport } from '@tanstack/ai-mcp/stdio'

const client = await createMCPClient({
  transport: stdioTransport({
    command: 'npx',
    args: ['-y', 'my-mcp-server'],
    env: { API_KEY: process.env.API_KEY ?? '' },
  }),
})

Custom transport (escape hatch)

Pass any SDK Transport instance directly:

import { createMCPClient } from '@tanstack/ai-mcp'
import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js'

const [clientTransport] = InMemoryTransport.createLinkedPair()
const client = await createMCPClient({ transport: clientTransport })

Authentication

Two levels:

• Static tokens — pass headers on the http/sse config (sent with every request): headers: { Authorization: 'Bearer ...' }.
• OAuth 2.1 (MCP authorization spec) — pass authProvider on the http/sse config. It accepts any OAuthClientProvider from @modelcontextprotocol/sdk/client/auth.js; the SDK transport attaches tokens, refreshes them, and retries on 401.

import { createMCPClient } from '@tanstack/ai-mcp'
import type { OAuthClientProvider } from '@modelcontextprotocol/sdk/client/auth.js'

declare const myOAuthProvider: OAuthClientProvider // backed by stored tokens

const client = await createMCPClient({
  transport: {
    type: 'http',
    url: 'https://mcp.example.com/mcp',
    authProvider: myOAuthProvider,
  },
})

Caveat: interactive authorization-code flows need transport.finishAuth(code), and createMCPClient does not expose its internal transport. For redirect flows, construct the StreamableHTTPClientTransport yourself with the authProvider, keep a reference, call finishAuth(code) in the OAuth callback route, then pass the transport via the escape hatch above. For server-side providers backed by pre-provisioned/refreshable tokens, the config form is sufficient.

Three type-safety modes

Mode 1 — Auto-discovery (no types needed)

client.tools() lists every tool the server exposes. Args are typed unknown at compile time but the tool's JSON Schema is forwarded to the LLM.

const tools = await client.tools()
// tools: ServerTool[]  (args unknown)

const stream = chat({
  adapter: openaiText('gpt-5.5'),
  messages,
  tools,
})

Use { lazy: true } to defer schema sending via the existing LazyToolManager:

const tools = await client.tools({ lazy: true })

Mode 2 — Typed via toolDefinition instances

Pass bare toolDefinition() instances (no .server() call) to client.tools([...]). The MCP client binds a callTool proxy as the execute function while input/output validation and TypeScript types come from the definitions' Zod schemas. Only the named tools are returned (allowlist = the definitions' names). Throws MCPToolNotFoundError if the server does not expose a tool with that name.

import { toolDefinition } from '@tanstack/ai'
import { createMCPClient } from '@tanstack/ai-mcp'
import { z } from 'zod'

const getWeatherDef = toolDefinition({
  name: 'get_weather',
  description: 'Current weather for a city',
  inputSchema: z.object({ city: z.string() }),
  outputSchema: z.object({ temperature: z.number(), conditions: z.string() }),
})

const client = await createMCPClient({
  transport: { type: 'http', url: 'https://mcp.example.com/mcp' },
})

// Returns MappedServerTools<typeof defs> — fully typed per definition.
const tools = await client.tools([getWeatherDef])

Mode 3 — Generated types (via generate CLI)

Run npx @tanstack/ai-mcp generate to introspect live servers and emit a ServerDescriptor interface per server. Pass the generated interface as the generic to createMCPClient<WeatherServer>(...) to narrow discovered tool names to the server's literals (args stay untyped — use Mode 2 for typed args).

See the "Codegen CLI" section below for details.

Lifecycle

The caller owns the lifecycle. chat() never closes the client.

Tools execute lazily while the response stream is consumed — close only after the stream is drained. In a streaming route handler, try/finally around the return (or await using at function scope) closes the client before the body streams; use a middleware terminal hook there instead (see Common Mistakes below).

// Option 1: middleware terminal hooks (streaming route handlers)
const client = await createMCPClient({
  transport: { type: 'http', url: '...' },
})
const stream = chat({
  adapter: openaiText('gpt-5.5'),
  messages,
  tools: await client.tools(),
  middleware: [
    {
      name: 'mcp-close',
      onFinish: () => client.close(),
      onAbort: () => client.close(),
      onError: () => client.close(),
    },
  ],
})
return toServerSentEventsResponse(stream)

// Option 2: explicit close after in-scope consumption
const client = await createMCPClient({
  transport: { type: 'http', url: '...' },
})
try {
  const stream = chat({
    adapter: openaiText('gpt-5.5'),
    messages,
    tools: await client.tools(),
  })
  for await (const chunk of stream) {
    // stream fully consumed inside this block
  }
} finally {
  await client.close()
}

// Option 3: await using (TypeScript 5.2+ with Symbol.asyncDispose) —
// same rule: consume the stream before the scope exits.
await using client = await createMCPClient({
  transport: { type: 'http', url: '...' },
})
// ... consume the stream in this scope; close() runs at scope exit

chat({ mcp }) — discovery + lifecycle in one prop

Rather than calling client.tools() and client.close() yourself, pass the mcp option to chat() and let it manage the full lifecycle.

// ChatMCPOptions shape:
// mcp: {
//   clients: Array<MCPClient | MCPClients>,
//   connection?: 'close' | 'keep-alive',  // default: 'close'
//   lazyTools?: boolean,
//   onDiscoveryError?: (error: unknown, source) => void,
// }

Behavior:

• chat() calls .tools() on every entry in clients at run start and merges all results into the tool list.
• lazyTools: true is forwarded to tools({ lazy: true }).
• connection: 'close' (default) — each client is closed when the run ends (after the agent loop completes and the stream is drained). With 'keep-alive', chat() never closes the clients — the caller owns their lifecycle (keep connections warm across requests).
• onDiscoveryError: throw (or re-throw) to abort the entire call; return normally to skip that source and continue. Omitting the handler re-throws (fail-fast).

When to use mcp vs. the tools spread:

Server-side example:

import { createFileRoute } from '@tanstack/react-router'
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { createMCPClient } from '@tanstack/ai-mcp'

export const Route = createFileRoute('/api/chat')({
  server: {
    handlers: {
      POST: async ({ request }) => {
        const { messages } = await request.json()

        const mcpClient = await createMCPClient({
          transport: { type: 'http', url: 'https://mcp.example.com/mcp' },
        })

        const stream = chat({
          adapter: openaiText('gpt-5.5'),
          messages,
          mcp: {
            clients: [mcpClient],
            connection: 'keep-alive', // chat() won't close it — reuse across requests
            onDiscoveryError: (err, source) => {
              console.warn('MCP discovery failed for source, skipping:', err)
              // returning skips this source; throw to fail the whole call fast
            },
          },
        })

        return toServerSentEventsResponse(stream)
        // connection: 'keep-alive' — chat() never closes mcpClient; it stays warm for the next request.
      },
    },
  },
})

You can also pass an MCPClients pool directly:

const pool = await createMCPClients({
  github: { transport: { type: 'http', url: 'https://mcp.github.com/mcp' } },
  linear: { transport: { type: 'http', url: 'https://mcp.linear.app/mcp' } },
})

const stream = chat({
  adapter: openaiText('gpt-5.5'),
  messages,
  mcp: { clients: [pool], connection: 'keep-alive' },
})

createMCPClients — multiple servers

Connect to many MCP servers in parallel. Each config key becomes the default prefix for that server's tools, preventing name collisions across servers.

import { createMCPClients } from '@tanstack/ai-mcp'

await using pool = await createMCPClients({
  github: { transport: { type: 'http', url: 'https://mcp.github.com/mcp' } },
  linear: { transport: { type: 'http', url: 'https://mcp.linear.app/mcp' } },
})

// Tool names auto-prefixed: 'github_search_repos', 'linear_create_issue', etc.
const tools = await pool.tools()

// Forward lazy flag to every server:
const lazyTools = await pool.tools({ lazy: true })

// Per-server typed access:
const githubTools = await pool.clients.github.tools()

createMCPClients connects in parallel, closes already-connected clients if any connection fails (no leaks), and throws MCPConnectionError naming the failed server(s).

Override or disable prefixing:

await using pool = await createMCPClients({
  github: { transport: { ... }, prefix: 'gh' },    // 'gh_search_repos'
  linear: { transport: { ... }, prefix: '' },        // 'create_issue' (no prefix)
})

Abort signal — cancelling in-flight MCP calls

MCP tool calls are automatically cancelled when the chat run's AbortController fires (e.g. client disconnect, server abort). The abortSignal is threaded through ToolExecutionContext into every callTool call with no extra code.

You can also read it in a hand-written server tool that wraps an MCP call:

const myTool = myDef.server(async (args, ctx) => {
  // Forward to any async work that accepts an AbortSignal.
  const result = await fetch('https://slow.api/data', {
    signal: ctx?.abortSignal,
  })
  return result.json()
})

Resources

// List all resources the server exposes.
const resources = await client.resources()

// Read a specific resource by URI.
const resource = await client.readResource(resources[0].uri)

// Convert one content block to a TanStack ContentPart.
import { mcpResourceToContentPart } from '@tanstack/ai-mcp'

const part = mcpResourceToContentPart(resource.contents[0])
// part: ContentPart  (type: 'text' always for v1)

Inject resources into a chat turn:

import { chat } from '@tanstack/ai'
import { createMCPClient, mcpResourceToContentPart } from '@tanstack/ai-mcp'

const client = await createMCPClient({
  transport: { type: 'http', url: '...' },
})
const resource = await client.readResource('file:///project/README.md')
const parts = resource.contents.map(mcpResourceToContentPart)

const stream = chat({
  adapter: openaiText('gpt-5.5'),
  messages: [
    {
      role: 'user',
      content: [
        ...parts,
        { type: 'text', content: 'Summarize this document.' },
      ],
    },
  ],
})

Prompts

// List prompts the server exposes.
const prompts = await client.prompts()

// Get a prompt (with optional arguments).
const prompt = await client.getPrompt('review_code', { language: 'TypeScript' })

// Convert to TanStack ModelMessage[] for use in chat().
import { mcpPromptToMessages } from '@tanstack/ai-mcp'

const messages = mcpPromptToMessages(prompt)
// messages: ModelMessage[]  (role: 'user' | 'assistant')

const stream = chat({
  adapter: openaiText('gpt-5.5'),
  messages: [...messages, ...userMessages],
})

MCP Apps

MCP Apps let an MCP tool surface a UI widget (static or interactive) on the client. Two variants exist. See docs/mcp/apps.md for the full guide.

Static widgets — UIResourcePart

When an MCP tool result carries a ui:// resource, TanStack AI emits a UIResourcePart on the assistant UIMessage, alongside the normal ToolCallPart / ToolResultPart. It is purely presentational — it never enters model input. The resource is read eagerly during the chat() run; if the read fails the tool result still flows to the model and the widget is simply absent (fail-soft). Static widgets require the MCP source to expose readResource — both createMCPClient and a createMCPClients pool do.

import type { UIResourcePart } from '@tanstack/ai'

// UIResourcePart shape (on the assistant UIMessage):
// {
//   type: 'ui-resource'
//   resource: { uri: string; mimeType: string; text?: string; blob?: string }
//   serverId?: string     // pool prefix / config key — routes interactive calls
//   toolCallId: string    // links to the originating tool call
//   toolName: string      // server-native MCP tool name whose UI this renders
//   meta?: Record<string, unknown>  // reserved — currently always undefined
// }

Interactive apps — createMcpAppCallHandler

For interactive apps (the widget iframe posts tool-call / prompt / link actions back), mount createMcpAppCallHandler from @tanstack/ai-mcp/apps at a POST route. Pass the MCP client(s) you already created — a single MCPClient, an MCPClients pool, or an array of either. The handler reads each client's transport descriptor via client.getInfo() / pool.getServers() (pure config, not a live socket) and reconnects per-call (stateless / serverless-safe). It matches the widget-supplied native (unprefixed) tool name against the server's unprefixed tool names, enforces a same-server allowlist, and returns { ok: true, result } or { ok: false, error }.

For a pool, the serverId on the UIResourcePart is the config key (the tool prefix); for a single client it is the client's prefix (or the sole default when serverId is absent and there is exactly one client).

import { createMCPClients } from '@tanstack/ai-mcp'
import {
  createMcpAppCallHandler,
  inMemoryMcpSessionStore,
} from '@tanstack/ai-mcp/apps'

// Reuse the same pool you pass to chat({ mcp: { clients: [mcp] } }).
const mcp = await createMCPClients({
  weather: {
    transport: { type: 'http', url: 'https://mcp-app.example.com/mcp' },
  },
})

// Minimal — reconnect-per-call via getServers() descriptor.
const handler = createMcpAppCallHandler({ clients: mcp })

// Options:
// clients   — MCPClient | MCPClients | Array<MCPClient | MCPClients> (required).
//             The handler reads transport descriptors via client.getInfo() /
//             pool.getServers() — the client does not need a live connection.
// store     — optional dynamic/stateful session store (e.g.
//             inMemoryMcpSessionStore()); used alongside clients.
// allowTool — optional authorizer receiving the WHOLE request:
//             (req: McpAppCallRequest) => boolean | Promise<boolean>.
//             The server-exposure check is ALWAYS enforced (the handler
//             rejects any tool the server does not expose). `allowTool`
//             is an ADDITIONAL restriction AND-ed on top: a request must
//             satisfy BOTH the server-exposure check and allowTool.
const handlerWithStore = createMcpAppCallHandler({
  clients: mcp,
  store: inMemoryMcpSessionStore(),
  allowTool: (req) => req.toolName === 'place_order',
})

The handler invokes the server (body: { threadId, serverId?, toolName, args?, messageId? }):

const result = await handler(body)
// { ok: true; result: unknown } | { ok: false; error: string }

Client side — useMcpAppBridge + MCPAppResource

In React/Preact, create the bridge with the useMcpAppBridge hook (from @tanstack/ai-react / @tanstack/ai-preact) — it returns a stable bridge per threadId/callEndpoint and always calls your latest sendMessage/onLink, so the bridge isn't recreated on every render (no useMemo / exhaustive-deps by hand). It's a thin wrapper over the framework-agnostic createMcpAppBridge from @tanstack/ai-client (use that directly outside React/Preact). Render resources with MCPAppResource from @tanstack/ai-react/mcp-apps (also @tanstack/ai-preact/mcp-apps, which requires a preact/compat alias). MCPAppResource uses @mcp-ui/client's AppRenderer under the hood — React only. Solid, Vue, Svelte, and Angular renderers are deferred.

The bridge exposes { callTool, sendPrompt, openLink } and routes the iframe's actions: tool → POST to callEndpoint; prompt → chat.sendMessage; link → onLink(url) if provided, otherwise the link is dropped (with a console warning) and openLink returns { isError: true } — it does NOT hang. toolName is read from part.toolName; it is not a prop. Omit bridge for display-only (inert) rendering.

import { useChat, useMcpAppBridge } from '@tanstack/ai-react'
import { fetchServerSentEvents } from '@tanstack/ai-client'
import { MCPAppResource } from '@tanstack/ai-react/mcp-apps'

function ChatPage() {
  const threadId = 'weather-chat'
  const { messages, sendMessage } = useChat({
    connection: fetchServerSentEvents('/api/chat'),
  })

  const bridge = useMcpAppBridge({
    threadId,
    callEndpoint: '/api/mcp-app/call',
    chat: { sendMessage: async (content) => void sendMessage({ content }) },
    // Opt in to link navigation — absent means links are dropped.
    onLink: (url) => window.open(url, '_blank', 'noopener'),
  })

  return (
    <div>
      {messages.map((msg) =>
        msg.parts.map((part, i) => {
          if (part.type === 'text') return <p key={i}>{part.content}</p>
          if (part.type === 'ui-resource') {
            return (
              <MCPAppResource
                key={i}
                part={part}
                bridge={bridge}
                sandbox={{ url: new URL('https://sandbox.example.com') }}
                // toolInput is optional; toolName comes from part.toolName.
              />
            )
          }
          return null
        }),
      )}
    </div>
  )
}

Codegen CLI

Generate TypeScript types (typed tool names and pool keys) by introspecting live MCP servers.

1. Create mcp.config.ts at your project root:

import { defineConfig } from '@tanstack/ai-mcp'

export default defineConfig({
  servers: {
    github: {
      transport: { type: 'http', url: 'https://mcp.github.com/mcp' },
      // prefix must match the runtime createMCPClient({ prefix }) value
    },
  },
  outFile: './src/mcp-types.generated.ts',
})

2. Run the generator:

npx @tanstack/ai-mcp generate

This connects to each server, lists its tools/resources/prompts, converts JSON Schemas to TypeScript, and writes one interface <Name>Server extends ServerDescriptor per server plus a combined interface MCPServers for pool typing.

3. Use the generated types:

// Single server — narrows tools() return to descriptor-keyed tool names.
import type { GithubServer } from './src/mcp-types.generated'
import { createMCPClient } from '@tanstack/ai-mcp'

const client = await createMCPClient<GithubServer>({
  transport: { type: 'http', url: 'https://mcp.github.com/mcp' },
})
const tools = await client.tools() // typed to GithubServer's tool names

// Multiple servers via the generated MCPServers map.
import type { MCPServers } from './src/mcp-types.generated'

const pool = await createMCPClients<MCPServers>({
  github: { transport: { type: 'http', url: 'https://mcp.github.com/mcp' } },
})
// pool.clients.github is MCPClient<GithubServer>
// missing/extra keys are a compile error

Codegen deps (json-schema-to-typescript, jiti) are bundled into the CLI bin and do NOT appear in the library's runtime dependency graph.

Error classes

• MCPConnectionError — thrown when a server connection fails or when calling methods after close().
• MCPToolNotFoundError — thrown from client.tools([defs]) when a definition's name is not exposed by the server.
• MCPTaskRequiredToolError — thrown from client.tools([defs]) when the named tool declares execution.taskSupport: 'required' (experimental MCP tasks). Such tools only run via the SDK's tasks/callToolStream flow, which @tanstack/ai-mcp does not support yet; they are silently excluded from tools() auto-discovery for the same reason.
• DuplicateToolNameError — thrown by a single pool's own tools() when two tools within that pool share the same name (same server or pool clients with no prefix). Exported from @tanstack/ai-mcp.
• MCPDuplicateToolNameError — thrown by chat() when tools from separate mcp.clients entries collide after merging. Exported from @tanstack/ai (not @tanstack/ai-mcp), so users can instanceof it at the chat() call site.

import {
  MCPConnectionError,
  MCPToolNotFoundError,
  MCPTaskRequiredToolError,
  DuplicateToolNameError,
} from '@tanstack/ai-mcp'

import { MCPDuplicateToolNameError } from '@tanstack/ai'

Complete server-route example

// src/routes/api.chat.ts
import { createFileRoute } from '@tanstack/react-router'
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { createMCPClients } from '@tanstack/ai-mcp'

export const Route = createFileRoute('/api/chat')({
  server: {
    handlers: {
      POST: async ({ request }) => {
        const { messages } = await request.json()

        const pool = await createMCPClients({
          github: {
            transport: { type: 'http', url: 'https://mcp.github.com/mcp' },
          },
          linear: {
            transport: {
              type: 'http',
              url: 'https://mcp.linear.app/mcp',
              headers: {
                Authorization: `Bearer ${process.env.LINEAR_KEY ?? ''}`,
              },
            },
          },
        })

        const stream = chat({
          adapter: openaiText('gpt-5.5'),
          messages,
          tools: await pool.tools(),
          // Close after the run ends — tools execute while the response streams,
          // so `await using` / try-finally would close the pool too early here.
          middleware: [
            {
              name: 'mcp-close',
              onFinish: () => pool.close(),
              onAbort: () => pool.close(),
              onError: () => pool.close(),
            },
          ],
        })

        return toServerSentEventsResponse(stream)
      },
    },
  },
})

Common Mistakes

a. HIGH: closing the client before the stream finishes

chat() executes tools lazily as the model calls them during streaming. If you close the MCP client before the response stream is fully consumed, in-flight tool calls will fail.

Wrong:

const tools = await client.tools()
const stream = chat({ adapter, messages, tools })
await client.close() // closes before the stream runs tools
return toServerSentEventsResponse(stream)

This includes try/finally around the return, and await using at function scope — both close before the returned Response body streams.

Correct — close in middleware terminal hooks (exactly one of onFinish/onAbort/onError fires per run), or consume the stream in scope before closing:

import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { createMCPClient } from '@tanstack/ai-mcp'

const client = await createMCPClient({
  transport: { type: 'http', url: '...' },
})

const stream = chat({
  adapter: openaiText('gpt-5.5'),
  messages,
  tools: await client.tools(),
  middleware: [
    {
      name: 'mcp-close',
      onFinish: () => client.close(),
      onAbort: () => client.close(),
      onError: () => client.close(),
    },
  ],
})
return toServerSentEventsResponse(stream)

b. HIGH: importing stdioTransport from the main entry point

stdioTransport is only available from @tanstack/ai-mcp/stdio. Importing it from @tanstack/ai-mcp will fail with a module-not-found error and would bundle Node.js child-process code into edge bundles.

Wrong:

import { stdioTransport } from '@tanstack/ai-mcp' // does not exist here

Correct:

import { stdioTransport } from '@tanstack/ai-mcp/stdio'

c. MEDIUM: using client.tools([defs]) without matching names

The name field on each toolDefinition must exactly match the tool name the MCP server exposes. Mismatches throw MCPToolNotFoundError at call time, not at type-check time (unless generated types are in use).

d. MEDIUM: not setting a prefix when multiple servers share tool names

Two different errors can arise depending on where the collision is detected:

• Within a single createMCPClients pool — calling pool.tools() throws DuplicateToolNameError (from @tanstack/ai-mcp) when two servers in that pool expose the same name with no prefix to separate them.
• Across separate mcp.clients entries in chat() — chat() throws MCPDuplicateToolNameError (from @tanstack/ai) after merging discovered tools from all mcp.clients entries.

In both cases, the fix is the same: use createMCPClients (which auto-prefixes by config key) or set an explicit prefix on each createMCPClient call.

Cross-References

• See also: ai-core/tool-calling/SKILL.md — MCP tools are ServerTools; all tool patterns (approval, lazy, client-side) apply.
• See also: ai-core/chat-experience/SKILL.md — wiring tools into chat().

AG-UI Protocol

This skill builds on ai-core. Read it first for critical rules.

Setup — Server Endpoint Producing AG-UI Events via SSE

import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

export async function POST(request: Request) {
  const { messages } = await request.json()
  const stream = chat({
    adapter: openaiText('gpt-5.2'),
    messages,
  })
  return toServerSentEventsResponse(stream)
}

chat() returns an AsyncIterable<StreamChunk>. Each StreamChunk is a typed AG-UI event (discriminated union on type). The toServerSentEventsResponse() helper encodes that iterable into an SSE-formatted Response with correct headers.

Setup — Receiving AG-UI RunAgentInput on the Server

import {
  chat,
  chatParamsFromRequestBody,
  mergeAgentTools,
  toServerSentEventsResponse,
} from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai/adapters'
import { serverTools } from './tools'

export async function POST(req: Request) {
  let params
  try {
    params = await chatParamsFromRequestBody(await req.json())
  } catch (error) {
    return new Response(
      error instanceof Error ? error.message : 'Bad request',
      { status: 400 },
    )
  }

  const stream = chat({
    adapter: openaiText('gpt-4o'),
    messages: params.messages,
    tools: mergeAgentTools(serverTools, params.tools),
  })

  return toServerSentEventsResponse(stream)
}

chatParamsFromRequestBody validates the body against RunAgentInputSchema from @ag-ui/core. mergeAgentTools merges the server's tool registry with client-declared tools (server wins on collision; client-only tools become no-execute stubs that flow through the runtime's ClientToolRequest path).

params.messages is a mixed array of TanStack UIMessage anchors (with parts) and AG-UI fan-out duplicates ({role:'tool',...}, {role:'reasoning',...}). The existing convertMessagesToModelMessages (called inside chat()) handles dedup automatically.

Wire shape (POST body): AG-UI RunAgentInput — {threadId, runId, parentRunId?, state, messages, tools, context, forwardedProps}. The messages array carries TanStack UIMessage anchors with their canonical parts plus AG-UI mirror fields (content, toolCalls) inline; tool results and thinking parts are additionally emitted as fan-out {role:'tool',...} and {role:'reasoning',...} entries.

forwardedProps security: Don't spread it directly into chat() — clients could override adapter, model, tools, etc. Always allowlist specific fields.

Core Patterns

1. SSE Format — toServerSentEventsStream / toServerSentEventsResponse

Wire format: Each event is data: <JSON>\n\n. Stream ends with data: [DONE]\n\n.

import {
  chat,
  toServerSentEventsStream,
  toServerSentEventsResponse,
} from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

// Option A: Get a ReadableStream (manual Response construction)
const abortController = new AbortController()
const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  abortController,
})
const sseStream = toServerSentEventsStream(stream, abortController)

const response = new Response(sseStream, {
  headers: {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    Connection: 'keep-alive',
  },
})

// Option B: Use the helper (sets headers automatically)
const response2 = toServerSentEventsResponse(stream, { abortController })
// Default headers: Content-Type: text/event-stream, Cache-Control: no-cache, Connection: keep-alive

Default response headers set by toServerSentEventsResponse():

Custom headers merge on top (user headers override defaults):

toServerSentEventsResponse(stream, {
  headers: {
    'X-Accel-Buffering': 'no', // Disable nginx buffering
    'Cache-Control': 'no-store', // Override default
  },
  abortController,
})

Error handling: If the stream throws, a RUN_ERROR event is emitted automatically before the stream closes. If the abortController is already aborted, the error event is suppressed and the stream closes silently.

2. HTTP Stream (NDJSON) — toHttpStream / toHttpResponse

Wire format: Each event is <JSON>\n (newline-delimited JSON, no SSE prefix, no [DONE] marker).

import { chat, toHttpStream, toHttpResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

// Option A: Get a ReadableStream
const abortController = new AbortController()
const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  abortController,
})
const ndjsonStream = toHttpStream(stream, abortController)

const response = new Response(ndjsonStream, {
  headers: {
    'Content-Type': 'application/x-ndjson',
  },
})

// Option B: Use the helper (does NOT set headers automatically)
const response2 = toHttpResponse(stream, { abortController })
// Note: toHttpResponse does NOT set Content-Type automatically.
// You should pass headers explicitly:
const response3 = toHttpResponse(stream, {
  headers: { 'Content-Type': 'application/x-ndjson' },
  abortController,
})

Client-side pairing: SSE endpoints are consumed by fetchServerSentEvents(). HTTP stream endpoints are consumed by fetchHttpStream(). Both are connection adapters from @tanstack/ai-react (or the framework-specific package).

3. AG-UI Event Types Reference

All events extend BaseAGUIEvent which carries type, timestamp, optional model, and optional rawEvent.

Typical event sequence for a text-only response:

RUN_STARTED -> TEXT_MESSAGE_START -> TEXT_MESSAGE_CONTENT (repeated) -> TEXT_MESSAGE_END -> RUN_FINISHED

Typical event sequence with tool calls:

RUN_STARTED -> TEXT_MESSAGE_START -> TEXT_MESSAGE_CONTENT* -> TEXT_MESSAGE_END
            -> TOOL_CALL_START -> TOOL_CALL_ARGS* -> TOOL_CALL_END
            -> RUN_FINISHED (finishReason: 'tool_calls')

Type aliases: StreamChunk is an alias for AGUIEvent (the discriminated union of all event interfaces). StreamChunkType is an alias for AGUIEventType (the string union of all event type literals).

4. Typed CUSTOM Events — ChatStream and KnownCustomEvent

The CUSTOM row above describes the raw StreamChunk union, where the single generic CustomEvent member types value as any -- once merged into a union, that any poisons every other member too, so narrowing on name still leaves value: any. chat() doesn't return raw StreamChunk; by default (no outputSchema, stream not explicitly false) it returns ChatStream, which swaps that generic member for KnownCustomEvent -- a discriminated union of every CUSTOM event TanStack AI itself emits, each with a literal name and a concrete value. Narrow with a plain if -- no helper, no cast:

import { chat } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
})

for await (const chunk of stream) {
  if (chunk.type === 'CUSTOM' && chunk.name === 'sandbox.file.diff') {
    console.log(chunk.value.path, chunk.value.diff) // typed, no helper, no cast
  } else if (
    chunk.type === 'CUSTOM' &&
    chunk.name === 'structured-output.complete'
  ) {
    console.log(chunk.value.object) // typed, no helper, no cast
  }
}

Caveat -- .endsWith() (or any non-literal check) does not narrow. SessionIdEvent['name'] is the template-literal type `${string}.session-id`. TypeScript's control-flow narrowing only understands exact comparisons (===) and in/type-predicate checks against a discriminant -- a runtime chunk.name.endsWith('.session-id') check doesn't inform the type system, so chunk.value stays the union of every KnownCustomEvent's value, not { sessionId: string }. Compare against the exact literal you expect, or write a user-defined type predicate ((c): c is SessionIdEvent => c.name.endsWith('.session-id')) and call that in the if instead.

User-emitted emitCustomEvent names are typed out of ChatStream. Tools that call context.emitCustomEvent('my-app:progress', ...) still stream a CUSTOM chunk at runtime, but 'my-app:progress' isn't one of KnownCustomEvent's literal names, so it's intentionally absent from ChatStream's type -- including a generic fallback member would reintroduce the value: any poison for every other event on the stream. To read your own event with a type, annotate the stream as the wider StreamChunk instead of ChatStream for that branch; its generic CUSTOM member already types value as any, so no cast is needed there either.

Source: docs/protocol/custom-events.md

Common Mistakes

MEDIUM: Proxy buffering breaks SSE streaming

Reverse proxies (nginx, Cloudflare, AWS ALB) buffer SSE responses by default, causing events to arrive in batches instead of streaming token-by-token.

Fix: Set proxy-bypass headers on the response.

toServerSentEventsResponse(stream, {
  headers: {
    'X-Accel-Buffering': 'no', // nginx
    'X-Content-Type-Options': 'nosniff', // Some CDNs
  },
  abortController,
})

For Cloudflare Workers, SSE streams automatically. For Cloudflare proxied origins, ensure "Response Buffering" is disabled in the dashboard.

Source: docs/protocol/sse-protocol.md

MEDIUM: Assuming all AG-UI events arrive in every response

Not all event types appear in every stream:

• STEP_STARTED / STEP_FINISHED only appear with thinking-enabled models (e.g., o3, claude-sonnet-4-5 with extended thinking). Standard models skip these entirely.
• TOOL_CALL_START / TOOL_CALL_ARGS / TOOL_CALL_END only appear when the model invokes tools. A text-only response has none.
• STATE_SNAPSHOT / STATE_DELTA only appear when server code explicitly emits them for stateful agent workflows.
• MESSAGES_SNAPSHOT only appears when the server explicitly sends a full transcript snapshot.
• CUSTOM events are application-defined and never emitted by default.

Code that expects a fixed sequence (e.g., always waiting for STEP_FINISHED before processing text) will hang or break on models that don't emit those events.

Source: docs/protocol/chunk-definitions.md

Tension

RESOLVED: TanStack AI is fully AG-UI compliant on both axes (server→client events AND client→server RunAgentInput). The wire format carries TanStack UIMessage anchors with their parts intact alongside AG-UI fan-out messages, so strict AG-UI servers see role-based messages while TanStack-aware servers read parts directly without transformation. See docs/migration/ag-ui-compliance.md for details.

Cross-References

• See also: ai-core/custom-backend-integration/SKILL.md -- Custom backends must implement SSE or HTTP stream format to work with TanStack AI client connection adapters.
• See also: ai-core/middleware/SKILL.md -- sandbox.file.diff's { path, diff } value (one of KnownCustomEvent's members) is populated from the same lazy before()/after()/diff() accessors documented there for onFile* middleware hooks.
• Full CUSTOM event taxonomy: docs/protocol/custom-events.md.

Chat Experience

This skill builds on ai-core. Read it first for critical rules.

Setup — Minimal Chat App

Server: API Route (TanStack Start)

// src/routes/api.chat.ts
import { createFileRoute } from '@tanstack/react-router'
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

export const Route = createFileRoute('/api/chat')({
  server: {
    handlers: {
      POST: async ({ request }) => {
        const abortController = new AbortController()
        const body = await request.json()
        const { messages } = body

        const stream = chat({
          adapter: openaiText('gpt-5.2'),
          messages,
          systemPrompts: ['You are a helpful assistant.'],
          abortController,
        })

        return toServerSentEventsResponse(stream, { abortController })
      },
    },
  },
})

Client: React Component

// src/routes/index.tsx
import { useState } from 'react'
import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'
import type { UIMessage } from '@tanstack/ai-react'

function ChatPage() {
  const [input, setInput] = useState('')

  const { messages, sendMessage, isLoading, error, stop } = useChat({
    connection: fetchServerSentEvents('/api/chat'),
  })

  const handleSubmit = () => {
    if (!input.trim()) return
    sendMessage(input.trim())
    setInput('')
  }

  return (
    <div>
      <div>
        {messages.map((message: UIMessage) => (
          <div key={message.id}>
            <strong>{message.role}:</strong>
            {message.parts.map((part, i) => {
              if (part.type === 'text') {
                return <p key={i}>{part.content}</p>
              }
              return null
            })}
          </div>
        ))}
      </div>

      {error && <div>Error: {error.message}</div>}

      <div>
        <input
          value={input}
          onChange={(e) => setInput(e.target.value)}
          onKeyDown={(e) => {
            if (e.key === 'Enter' && !e.shiftKey) {
              e.preventDefault()
              handleSubmit()
            }
          }}
          disabled={isLoading}
          placeholder="Type a message..."
        />
        {isLoading ? (
          <button onClick={stop}>Stop</button>
        ) : (
          <button onClick={handleSubmit} disabled={!input.trim()}>
            Send
          </button>
        )}
      </div>
    </div>
  )
}

Vue/Solid/Svelte/Preact have identical patterns with different hook imports (e.g., import { useChat } from '@tanstack/ai-solid').

Core Patterns

1. Streaming Chat with SSE

Server returns a streaming SSE Response; client parses it automatically.

Server:

import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { anthropicText } from '@tanstack/ai-anthropic'

const stream = chat({
  adapter: anthropicText('claude-sonnet-4-5'),
  messages,
  modelOptions: {
    temperature: 0.7,
    max_tokens: 2000, // Anthropic-native key
  },
  systemPrompts: ['You are a helpful assistant.'],
  abortController,
})

return toServerSentEventsResponse(stream, { abortController })

Client:

import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'

const { messages, sendMessage, isLoading, error, stop, status } = useChat({
  connection: fetchServerSentEvents('/api/chat'),
  body: { provider: 'anthropic', model: 'claude-sonnet-4-5' },
  onFinish: (message) => {
    console.log('Response complete:', message.id)
  },
  onError: (err) => {
    console.error('Stream error:', err)
  },
})

The body field is merged into the POST request body alongside messages, letting the server read data.provider, data.model, etc.

The status field tracks the chat lifecycle: 'ready' | 'submitted' | 'streaming' | 'error'.

2. Rendering Thinking/Reasoning Content

Models with extended thinking (Claude, Gemini) emit ThinkingPart in the message parts array.

import type { UIMessage } from '@tanstack/ai-react'

function MessageRenderer({ message }: { message: UIMessage }) {
  return (
    <div>
      {message.parts.map((part, i) => {
        if (part.type === 'thinking') {
          const isComplete = message.parts
            .slice(i + 1)
            .some((p) => p.type === 'text')
          return (
            <details key={i} open={!isComplete}>
              <summary>{isComplete ? 'Thought process' : 'Thinking...'}</summary>
              <pre>{part.content}</pre>
            </details>
          )
        }

        if (part.type === 'text' && part.content) {
          return <p key={i}>{part.content}</p>
        }

        if (part.type === 'tool-call') {
          return (
            <div key={part.id}>
              Tool call: {part.name} ({part.state})
            </div>
          )
        }

        return null
      })}
    </div>
  )
}

Server-side, enable thinking via modelOptions on the adapter:

import { geminiText } from '@tanstack/ai-gemini'

const stream = chat({
  adapter: geminiText('gemini-2.5-flash'),
  messages,
  modelOptions: {
    thinkingConfig: {
      includeThoughts: true,
      thinkingBudget: 100,
    },
  },
})

3. Sending Multimodal Content (Images)

Use sendMessage with a MultimodalContent object instead of a plain string.

import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'
import type { ContentPart } from '@tanstack/ai'

const { sendMessage } = useChat({
  connection: fetchServerSentEvents('/api/chat'),
})

function sendImageMessage(text: string, imageBase64: string, mimeType: string) {
  const contentParts: Array<ContentPart> = [
    { type: 'text', content: text },
    {
      type: 'image',
      source: { type: 'data', value: imageBase64, mimeType },
    },
  ]

  sendMessage({ content: contentParts })
}

function sendImageUrl(text: string, imageUrl: string) {
  const contentParts: Array<ContentPart> = [
    { type: 'text', content: text },
    {
      type: 'image',
      source: { type: 'url', value: imageUrl },
    },
  ]

  sendMessage({ content: contentParts })
}

Render image parts in received messages:

if (part.type === 'image') {
  const src =
    part.source.type === 'url'
      ? part.source.value
      : `data:${part.source.mimeType};base64,${part.source.value}`
  return <img key={i} src={src} alt="Attached image" />
}

4. Sending Audio Messages (Browser Recording)

Use useAudioRecorder from @tanstack/ai-react (or createAudioRecorder in Svelte) to capture audio in the browser. The resolved AudioRecording includes a ready-to-use part that slots directly into sendMessage.

import {
  useAudioRecorder,
  useChat,
  fetchServerSentEvents,
} from '@tanstack/ai-react'

const { isRecording, isSupported, start, stop } = useAudioRecorder()
const { sendMessage } = useChat({
  connection: fetchServerSentEvents('/api/chat'),
})

async function toggle() {
  if (!isRecording) {
    await start()
    return
  }
  const recording = await stop()
  await sendMessage({ content: [recording.part] })
}

recording.part is { type: 'audio', source: { type: 'data', value: base64, mimeType } }. Returns the recorder's native format (audio/webm or audio/mp4) with no transcoding.

5. HTTP Stream Format (Alternative to SSE)

Use toHttpResponse + fetchHttpStream for newline-delimited JSON instead of SSE.

Server:

import { chat, toHttpResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  abortController,
})

return toHttpResponse(stream, { abortController })

Client:

import { useChat, fetchHttpStream } from '@tanstack/ai-react'

const { messages, sendMessage } = useChat({
  connection: fetchHttpStream('/api/chat'),
})

The only difference is swapping toServerSentEventsResponse / fetchServerSentEvents for toHttpResponse / fetchHttpStream. Everything else stays identical.

6. MCP Tool Discovery via chat({ mcp })

Pass mcp to let chat() own discovery and lifecycle for one or more MCP clients. Useful when you want minimal boilerplate and don't need to reuse the clients across calls.

// Prop shape:
// chat({
//   ...,
//   mcp: {
//     clients: Array<MCPClient | MCPClients>,
//     connection?: 'close' | 'keep-alive',  // default: 'close'
//     lazyTools?: boolean,
//     onDiscoveryError?: (error: unknown, source) => void,
//   }
// })

• clients — one or more MCPClient / MCPClients instances.
• connection — 'close' (default) closes each client when the run ends (after the agent loop completes and the stream is drained); with 'keep-alive', chat() never closes the clients — the caller owns their lifecycle (keep connections warm across requests).
• lazyTools — forwarded to tools({ lazy: true }) so tool schemas are sent to the LLM on demand.
• onDiscoveryError — throw (or re-throw) to fail the entire call fast; return normally to skip that source and continue. Omit to rethrow (fail-fast).

When to use mcp vs. the tools spread:

Server-side example:

import { createFileRoute } from '@tanstack/react-router'
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { createMCPClient } from '@tanstack/ai-mcp'

export const Route = createFileRoute('/api/chat')({
  server: {
    handlers: {
      POST: async ({ request }) => {
        const { messages } = await request.json()

        const mcpClient = await createMCPClient({
          transport: { type: 'http', url: 'https://mcp.example.com/mcp' },
        })

        const stream = chat({
          adapter: openaiText('gpt-5.5'),
          messages,
          mcp: {
            clients: [mcpClient],
            connection: 'keep-alive', // chat() won't close it — reuse across requests
          },
        })

        return toServerSentEventsResponse(stream)
        // connection: 'keep-alive' — chat() never closes mcpClient; it stays open for reuse across runs.
      },
    },
  },
})

Common Mistakes

a. CRITICAL: Using Vercel AI SDK patterns (streamText, generateText)

// WRONG
import { streamText } from 'ai'
import { openai } from '@ai-sdk/openai'
const result = streamText({ model: openai('gpt-4o'), messages })

// CORRECT
import { chat } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
const stream = chat({ adapter: openaiText('gpt-5.2'), messages })

b. CRITICAL: Using Vercel createOpenAI() provider pattern

// WRONG
import { createOpenAI } from '@ai-sdk/openai'
const openai = createOpenAI({ apiKey })
streamText({ model: openai('gpt-4o'), messages })

// CORRECT
import { openaiText } from '@tanstack/ai-openai'
import { chat } from '@tanstack/ai'
chat({ adapter: openaiText('gpt-5.2'), messages })

c. CRITICAL: Using monolithic openai() instead of openaiText()

// WRONG
import { openai } from '@tanstack/ai-openai'
chat({ adapter: openai(), model: 'gpt-5.2', messages })

// CORRECT
import { openaiText } from '@tanstack/ai-openai'
chat({ adapter: openaiText('gpt-5.2'), messages })

The monolithic openai() adapter is deprecated. Use tree-shakeable adapters: openaiText(), openaiImage(), openaiSpeech(), etc.

d. HIGH: Using toResponseStream instead of toServerSentEventsResponse

// WRONG
import { toResponseStream } from '@tanstack/ai'
return toResponseStream(stream, { abortController })

// CORRECT
import { toServerSentEventsResponse } from '@tanstack/ai'
return toServerSentEventsResponse(stream, { abortController })

e. HIGH: Passing model as separate parameter to chat()

// WRONG
chat({ adapter: openaiText(), model: 'gpt-5.2', messages })

// CORRECT
chat({ adapter: openaiText('gpt-5.2'), messages })

The model is passed to the adapter factory, not to chat().

f. HIGH: Passing sampling options at the root of chat()

Sampling options (temperature, token limits, top_p/topP) are not top-level fields on chat(). They live inside modelOptions using the provider's native key.

// WRONG — temperature/maxTokens are not root options
chat({ adapter, messages, temperature: 0.7, maxTokens: 1000 })

// WRONG — there is no `options` field either
chat({ adapter, messages, options: { temperature: 0.7, maxTokens: 1000 } })

// CORRECT — inside modelOptions, provider-native keys (OpenAI shown)
chat({
  adapter,
  messages,
  modelOptions: { temperature: 0.7, max_output_tokens: 1000 },
})

temperature is universal across providers; token limits use provider-native keys (max_output_tokens for OpenAI, max_tokens for Anthropic/Grok, maxOutputTokens for Gemini, max_completion_tokens for Groq, maxCompletionTokens for OpenRouter, and num_predict nested under modelOptions.options for Ollama). See ai-core/adapter-configuration/SKILL.md.

g. HIGH: Using providerOptions instead of modelOptions

// WRONG
chat({
  adapter,
  messages,
  providerOptions: { responseFormat: { type: 'json_object' } },
})

// CORRECT
chat({
  adapter,
  messages,
  modelOptions: { responseFormat: { type: 'json_object' } },
})

h. HIGH: Implementing custom SSE stream instead of using toServerSentEventsResponse

// WRONG
const readable = new ReadableStream({
  async start(controller) {
    const encoder = new TextEncoder()
    for await (const chunk of stream) {
      controller.enqueue(encoder.encode(`data: ${JSON.stringify(chunk)}\n\n`))
    }
    controller.enqueue(encoder.encode('data: [DONE]\n\n'))
    controller.close()
  },
})
return new Response(readable, {
  headers: { 'Content-Type': 'text/event-stream' },
})

// CORRECT
import { toServerSentEventsResponse } from '@tanstack/ai'
return toServerSentEventsResponse(stream, { abortController })

toServerSentEventsResponse handles SSE formatting, abort signals, error events (RUN_ERROR), and correct headers automatically.

i. HIGH: Implementing custom onEnd/onFinish callbacks instead of middleware

// WRONG
chat({
  adapter,
  messages,
  onEnd: (result) => {
    trackAnalytics(result)
  },
})

// CORRECT
import type { ChatMiddleware } from '@tanstack/ai'

const analytics: ChatMiddleware = {
  name: 'analytics',
  onFinish(ctx, info) {
    trackAnalytics({ reason: info.finishReason, iterations: ctx.iteration })
  },
  onUsage(ctx, usage) {
    trackTokens(usage.totalTokens)
  },
}

chat({ adapter, messages, middleware: [analytics] })

chat() has no onEnd/onFinish option. Use middleware for lifecycle events. See also: ai-core/middleware/SKILL.md.

j. HIGH: Importing from @tanstack/ai-client instead of framework package

// WRONG
import { fetchServerSentEvents } from '@tanstack/ai-client'
import { useChat } from '@tanstack/ai-react'

// CORRECT
import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'

Framework packages re-export everything needed from @tanstack/ai-client. Import from @tanstack/ai-client only in vanilla JS (no framework).

k. MEDIUM: Not handling RUN_ERROR events in streaming context

Streaming errors arrive as RUN_ERROR events in the stream, not as thrown exceptions. The useChat hook surfaces these via the error state and onError callback. If you consume the stream manually (without useChat), check for RUN_ERROR chunks:

for await (const chunk of stream) {
  if (chunk.type === 'RUN_ERROR') {
    console.error('Stream error:', chunk.error.message)
    break
  }
  if (chunk.type === 'TEXT_MESSAGE_CONTENT') {
    process.stdout.write(chunk.delta)
  }
}

If not handled, the UI appears to hang with no feedback.

Cross-References

• See also: ai-core/tool-calling/SKILL.md -- Most chats include tools
• See also: ai-core/adapter-configuration/SKILL.md -- Adapter choice affects available features
• See also: ai-core/middleware/SKILL.md -- Use middleware for analytics and lifecycle events

Custom Backend Integration

This skill builds on ai-core and ai-core/chat-experience. Read them first.

Setup

Connect useChat to a custom SSE backend with auth headers:

import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'

function Chat() {
  const { messages, sendMessage, isLoading } = useChat({
    connection: fetchServerSentEvents('https://my-api.com/chat', {
      headers: {
        Authorization: `Bearer ${token}`,
      },
    }),
  })

  return (
    <div>
      {messages.map((msg) => (
        <div key={msg.id}>
          <strong>{msg.role}:</strong>
          {msg.parts.map((part, i) => {
            if (part.type === 'text') {
              return <p key={i}>{part.content}</p>
            }
            return null
          })}
        </div>
      ))}
      <button onClick={() => sendMessage('Hello')}>Send</button>
    </div>
  )
}

Both fetchServerSentEvents and fetchHttpStream accept a static URL string or a function returning a string (evaluated per request), and a static options object or a sync/async function returning options (also evaluated per request). This allows dynamic auth tokens and URLs without re-creating the adapter.

Core Patterns

1. Custom SSE Backend with fetchServerSentEvents

Use when your backend speaks SSE (text/event-stream) with data: {json}\n\n framing. This is the recommended default.

Static options:

import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'

const { messages, sendMessage } = useChat({
  connection: fetchServerSentEvents('https://my-api.com/chat', {
    headers: {
      Authorization: `Bearer ${token}`,
      'X-Tenant-Id': tenantId,
    },
    credentials: 'include',
  }),
})

Dynamic URL and options (evaluated per request):

import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'

const { messages, sendMessage } = useChat({
  connection: fetchServerSentEvents(
    () => `https://my-api.com/chat?session=${sessionId}`,
    async () => ({
      headers: {
        Authorization: `Bearer ${await getAccessToken()}`,
      },
      body: {
        provider: 'openai',
        model: 'gpt-4o',
      },
    }),
  ),
})

The body field in options is merged into the POST request body alongside messages and data, so the server receives { messages, data, provider, model }.

Custom fetch client (for proxies, interceptors, retries):

import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'

const { messages, sendMessage } = useChat({
  connection: fetchServerSentEvents('/api/chat', {
    fetchClient: myCustomFetch,
  }),
})

2. Custom NDJSON Backend with fetchHttpStream

Use when your backend sends newline-delimited JSON (application/x-ndjson) instead of SSE. Each line is one JSON-encoded StreamChunk followed by \n.

import { useChat, fetchHttpStream } from '@tanstack/ai-react'

const { messages, sendMessage } = useChat({
  connection: fetchHttpStream('https://my-api.com/chat', {
    headers: {
      Authorization: `Bearer ${token}`,
    },
  }),
})

fetchHttpStream accepts the same URL and options signatures as fetchServerSentEvents (static or dynamic, sync or async). The only difference is the parsing: no data: prefix stripping, no [DONE] sentinel -- just one JSON object per line.

Dynamic options work identically:

import { useChat, fetchHttpStream } from '@tanstack/ai-react'

const { messages, sendMessage } = useChat({
  connection: fetchHttpStream(
    () => `/api/chat?region=${region}`,
    async () => ({
      headers: { Authorization: `Bearer ${await refreshToken()}` },
    }),
  ),
})

3. Fully Custom Connection Adapter

For protocols that don't fit SSE or NDJSON (WebSockets, gRPC-web, custom binary, server functions), implement the ConnectionAdapter interface directly.

There are two mutually exclusive modes:

ConnectConnectionAdapter (pull-based / async iterable):

Use when the client initiates a request and consumes the response as a stream. This is the simpler model and covers most HTTP-based protocols.

import { useChat } from '@tanstack/ai-react'
import type { ConnectionAdapter } from '@tanstack/ai-react'
import type { StreamChunk, UIMessage } from '@tanstack/ai'

const websocketAdapter: ConnectionAdapter = {
  async *connect(
    messages: Array<UIMessage>,
    data?: Record<string, any>,
    abortSignal?: AbortSignal,
  ): AsyncGenerator<StreamChunk> {
    const ws = new WebSocket('wss://my-api.com/chat')

    // Wait for connection
    await new Promise<void>((resolve, reject) => {
      ws.onopen = () => resolve()
      ws.onerror = (e) => reject(e)
    })

    // Send messages
    ws.send(JSON.stringify({ messages, ...data }))

    // Create an async queue to bridge WebSocket events to an async iterable
    const queue: Array<StreamChunk> = []
    let resolve: (() => void) | null = null
    let done = false

    ws.onmessage = (event) => {
      const chunk: StreamChunk = JSON.parse(event.data)
      queue.push(chunk)
      resolve?.()
    }

    ws.onclose = () => {
      done = true
      resolve?.()
    }

    ws.onerror = () => {
      done = true
      resolve?.()
    }

    abortSignal?.addEventListener('abort', () => {
      ws.close()
    })

    // Yield chunks as they arrive
    while (!done || queue.length > 0) {
      if (queue.length > 0) {
        yield queue.shift()!
      } else {
        await new Promise<void>((r) => {
          resolve = r
        })
      }
    }
  },
}

function Chat() {
  const { messages, sendMessage } = useChat({
    connection: websocketAdapter,
  })

  // ... render messages
}

SubscribeConnectionAdapter (push-based / separate subscribe + send):

Use for push-based protocols where the server can send data at any time (persistent WebSocket connections, MQTT, server push). The subscribe method returns an AsyncIterable<StreamChunk> that stays open, and send dispatches messages through it.

import type { StreamChunk, UIMessage } from '@tanstack/ai'

// SubscribeConnectionAdapter is exported from @tanstack/ai-client
// (not re-exported by framework packages -- use ConnectionAdapter
//  union type from @tanstack/ai-react for typing)
const pushAdapter = {
  subscribe(abortSignal?: AbortSignal): AsyncIterable<StreamChunk> {
    // Return a long-lived async iterable that yields chunks
    // whenever the server pushes them
    return createPersistentStream(abortSignal)
  },

  async send(
    messages: Array<UIMessage>,
    data?: Record<string, any>,
    abortSignal?: AbortSignal,
  ): Promise<void> {
    // Dispatch messages; chunks arrive through subscribe()
    await persistentConnection.send(JSON.stringify({ messages, ...data }))
  },
}

function Chat() {
  const { messages, sendMessage } = useChat({
    connection: pushAdapter,
  })

  // ... render messages
}

The stream() helper function (re-exported from @tanstack/ai-react) provides a shorthand for creating a ConnectConnectionAdapter from an async generator:

import { useChat, stream } from '@tanstack/ai-react'
import type { StreamChunk, UIMessage } from '@tanstack/ai'

const directAdapter = stream(async function* (
  messages: Array<UIMessage>,
  data?: Record<string, any>,
): AsyncGenerator<StreamChunk> {
  const response = await fetch('https://my-api.com/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages, ...data }),
  })

  const reader = response.body!.getReader()
  const decoder = new TextDecoder()
  let buffer = ''

  while (true) {
    const { done, value } = await reader.read()
    if (done) break

    buffer += decoder.decode(value, { stream: true })
    const lines = buffer.split('\n')
    buffer = lines.pop() || ''

    for (const line of lines) {
      if (line.trim()) {
        yield JSON.parse(line) as StreamChunk
      }
    }
  }
})

const { messages, sendMessage } = useChat({
  connection: directAdapter,
})

Common Mistakes

a. HIGH: Providing both connect and subscribe+send in connection adapter

The ConnectionAdapter interface has two mutually exclusive modes. Providing both throws at runtime.

// WRONG -- throws "Connection adapter must provide either connect or both
// subscribe and send, not both modes"
const adapter = {
  async *connect(messages) {
    /* ... */
  },
  subscribe(signal) {
    /* ... */
  },
  async send(messages) {
    /* ... */
  },
}

// CORRECT -- pick one mode
// Option A: ConnectConnectionAdapter (pull-based)
const pullAdapter = {
  async *connect(messages, data, abortSignal) {
    // ... yield StreamChunks
  },
}

// Option B: SubscribeConnectionAdapter (push-based)
const pushAdapter = {
  subscribe(abortSignal) {
    return longLivedAsyncIterable
  },
  async send(messages, data, abortSignal) {
    await connection.dispatch({ messages, ...data })
  },
}

Source: ai-client/src/connection-adapters.ts line 116

b. MEDIUM: SSE browser connection limits

Browsers limit SSE connections to 6-8 per domain (the HTTP/1.1 connection limit). Multiple chat sessions on the same page, or multiple tabs to the same origin, can exhaust this limit. New connections queue indefinitely until an existing one closes.

Mitigations:

• Use HTTP/2 (multiplexes streams over a single TCP connection; no per-domain limit)
• Use fetchHttpStream instead of fetchServerSentEvents (each request is a standard POST, not a long-lived EventSource)
• Close idle connections when not actively streaming
• Use a single persistent WebSocket via SubscribeConnectionAdapter instead of per-request SSE connections

Source: docs/chat/connection-adapters.md

c. MEDIUM: HTTP stream without implementing reconnection

SSE has built-in browser auto-reconnection via the EventSource API. HTTP stream (NDJSON via fetchHttpStream) does not -- if the connection drops mid-stream, the partial response is silently lost with no automatic retry.

If your application needs resilience to transient network errors with HTTP streaming, implement retry logic in your connection adapter:

import { useChat } from '@tanstack/ai-react'
import type { ConnectionAdapter } from '@tanstack/ai-react'
import type { StreamChunk, UIMessage } from '@tanstack/ai'

const resilientAdapter: ConnectionAdapter = {
  async *connect(
    messages: Array<UIMessage>,
    data?: Record<string, any>,
    abortSignal?: AbortSignal,
  ): AsyncGenerator<StreamChunk> {
    const maxRetries = 3
    let attempt = 0

    while (attempt < maxRetries) {
      try {
        const response = await fetch('https://my-api.com/chat', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ messages, ...data }),
          signal: abortSignal,
        })

        if (!response.ok) {
          throw new Error(`HTTP ${response.status}`)
        }

        const reader = response.body!.getReader()
        const decoder = new TextDecoder()
        let buffer = ''

        while (true) {
          const { done, value } = await reader.read()
          if (done) break

          buffer += decoder.decode(value, { stream: true })
          const lines = buffer.split('\n')
          buffer = lines.pop() || ''

          for (const line of lines) {
            if (line.trim()) {
              yield JSON.parse(line) as StreamChunk
            }
          }
        }

        return // Stream completed successfully
      } catch (err) {
        if (abortSignal?.aborted) throw err
        attempt++
        if (attempt >= maxRetries) throw err
        // Exponential backoff
        await new Promise((r) => setTimeout(r, 1000 * 2 ** attempt))
      }
    }
  },
}

const { messages, sendMessage } = useChat({
  connection: resilientAdapter,
})

Note: fetchServerSentEvents in TanStack AI uses fetch() under the hood (not the browser EventSource API), so it also does not auto-reconnect. The SSE auto-reconnection advantage only applies when using the native EventSource API directly.

Source: docs/protocol/http-stream-protocol.md

Cross-References

• See also: ai-core/ag-ui-protocol/SKILL.md -- Understanding the AG-UI protocol helps build compatible custom servers
• See also: ai-core/chat-experience/SKILL.md -- Full chat setup patterns including server-side chat() and toServerSentEventsResponse()
• See also: ai-core/middleware/SKILL.md -- Use middleware for analytics and lifecycle events on the server side

Debug Logging

Dependency note: This skill builds on ai-core. Read it first for critical rules.

Use this skill when you need to turn debug logging on or off, narrow what's printed, or pipe logs into a custom logger (pino, winston, etc.). The same debug option works on every activity — chat(), summarize(), generateImage(), generateSpeech(), generateTranscription(), generateVideo().

Turn it on

import { chat } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  debug: true, // all categories on, prints to console
})

Each log line is prefixed with an emoji and [tanstack-ai:<category>]:

📤 [tanstack-ai:request] 📤 activity=chat provider=openai model=gpt-5.2 messages=1 tools=0 stream=true
🔁 [tanstack-ai:agentLoop] 🔁 run started
📥 [tanstack-ai:provider] 📥 provider=openai type=response.output_text.delta
📨 [tanstack-ai:output] 📨 type=TEXT_MESSAGE_CONTENT

Turn it off

chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  debug: false, // silence everything, including errors
})

Omitting debug is not the same as debug: false. When omitted, the errors category is still on (errors are cheap and important). Use debug: false or debug: { errors: false } for true silence.

DebugOption — the accepted shapes

type DebugOption = boolean | DebugConfig

interface DebugConfig {
  // Per-category flags. Any flag omitted from a DebugConfig defaults to true.
  request?: boolean
  provider?: boolean
  output?: boolean
  middleware?: boolean
  tools?: boolean
  agentLoop?: boolean
  config?: boolean
  errors?: boolean
  // Optional custom logger. Defaults to ConsoleLogger.
  logger?: Logger
}

Resolution rules for the debug?: DebugOption field on every activity:

Narrow what's printed

Pass a DebugConfig object. Unspecified categories default to true, so it's easiest to toggle by setting specific flags to false:

chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  debug: { middleware: false }, // everything except middleware
})

To print only a specific set, set the rest to false explicitly:

chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  debug: {
    provider: true,
    output: true,
    middleware: false,
    tools: false,
    agentLoop: false,
    config: false,
    errors: true, // keep errors on — they're cheap and important
    request: false,
  },
})

Pipe into your own logger

import type { Logger } from '@tanstack/ai'
import pino from 'pino'

const pinoLogger = pino()
const logger: Logger = {
  debug: (msg, meta) => pinoLogger.debug(meta, msg),
  info: (msg, meta) => pinoLogger.info(meta, msg),
  warn: (msg, meta) => pinoLogger.warn(meta, msg),
  error: (msg, meta) => pinoLogger.error(meta, msg),
}

chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  debug: { logger }, // all categories on, piped to pino
})

The default console logger is exported as ConsoleLogger if you want to wrap it:

import { ConsoleLogger } from '@tanstack/ai'

Categories

Chat-only categories simply never fire for non-chat activities — those concepts don't exist in their pipelines.

Non-chat activities

Same debug option everywhere:

summarize({ adapter, text, debug: true })
generateImage({ adapter, prompt: 'a cat', debug: { logger } })
generateSpeech({ adapter, text, debug: { request: true } })
generateTranscription({ adapter, audio, debug: false })
generateVideo({ adapter, prompt: 'a wave', debug: { output: true } })

Realtime session adapters in provider packages (e.g. openaiRealtime, elevenlabsRealtime) accept the same debug?: DebugOption on their session options. They emit request, provider, and errors lines; the chat-only categories don't apply.

Common Mistakes

a. HIGH: Treating omitted debug as silent

// WRONG — expecting this to be completely silent
chat({ adapter, messages })
// Errors still print via [tanstack-ai:errors] ... on failure.

// CORRECT — explicit silence
chat({ adapter, messages, debug: false })
chat({ adapter, messages, debug: { errors: false } })

debug undefined means "only errors"; debug: false means "nothing at all".

Source: docs/advanced/debug-logging.md

b. MEDIUM: Reaching for middleware when debug would do

// WRONG — writing logging middleware to see chunks flow
const chunkLogger: ChatMiddleware = {
  name: 'chunk-logger',
  onChunk: (ctx, chunk) => {
    console.log(chunk.type, chunk)
  },
}
chat({ adapter, messages, middleware: [chunkLogger] })

// CORRECT — just turn on the relevant categories
chat({
  adapter,
  messages,
  debug: { provider: true, output: true },
})

For observing the built-in pipeline, the debug option is strictly faster than writing logging middleware. Reach for middleware when you need to transform chunks, not just see them.

Source: docs/advanced/debug-logging.md

c. LOW: Logger implementation that can throw

A user-supplied Logger that throws will have its exception swallowed by the SDK so it never masks the real error that triggered the log call. Still, prefer implementations that don't throw — silenced exceptions are harder to debug than loud ones.

// WRONG — a logger that can throw on serialization
const fragile: Logger = {
  debug: (msg, meta) => console.debug(msg, JSON.stringify(meta)), // cyclic meta → throws
  /* ... */
}

// CORRECT — guard serialization in the logger itself
const safe: Logger = {
  debug: (msg, meta) => {
    try {
      console.debug(msg, meta)
    } catch {
      console.debug(msg)
    }
  },
  /* ... */
}

Source: packages/ai/src/logger/internal-logger.ts

Cross-References

• See also: ai-core/middleware/SKILL.md — if you need to transform chunks/config, not just observe them.
• See also: Observability (docs/advanced/observability.md) — the programmatic event client for a richer, structured feed beyond log lines.

Middleware

Dependency note: This skill builds on ai-core. Read it first for critical rules.

Setup — Analytics Tracking Middleware

import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  middleware: [
    {
      onStart: (ctx) => {
        console.log('Chat started:', ctx.model)
      },
      onFinish: (ctx, info) => {
        trackAnalytics({ model: ctx.model, tokens: info.usage?.totalTokens })
      },
      onError: (ctx, info) => {
        reportError(info.error)
      },
    },
  ],
})

return toServerSentEventsResponse(stream)

Hooks Reference

Every hook receives a ChatMiddlewareContext as its first argument, which provides requestId, streamId, phase, iteration, chunkIndex, model, provider, signal, abort(), defer(), and more.

Terminal hooks (onFinish, onAbort, onError) are mutually exclusive -- exactly one fires per chat() invocation.

Sampling in onConfig: temperature, topP, and maxTokens are not first-class fields on ChatMiddlewareConfig. To adjust sampling from middleware, return a partial that mutates config.modelOptions using the provider's native key (e.g. OpenAI temperature / max_output_tokens, Anthropic max_tokens, Ollama nested options.num_predict). Returning a top-level temperature/maxTokens has no effect.

Phase values

ctx.phase is one of:

Structured-output lifecycle rules (when chat({ outputSchema }) is used):

• onStructuredOutputConfig fires before onConfig at the structured-output boundary.
• onConfig re-fires at the same boundary with ctx.phase === 'structuredOutput', receiving the post-onStructuredOutputConfig view of the config (minus outputSchema).
• onChunk and onUsage fire for every chunk and usage event emitted by the structured-output call, with ctx.phase === 'structuredOutput'.
• onIteration does not fire for finalization — it is agent-loop-only.
• onFinish fires once at the end of the whole chat() invocation, after the structured-output finalization completes (not after the agent loop). Terminal-hook exclusivity still holds (one of onFinish / onAbort / onError).
• Terminal info and structured-output: info.usage / info.finishReason / info.content reflect the agent loop's terminal state, NOT the finalization step. Finalization state is intentionally segregated to keep agent-loop semantics clean. For a tools-less chat({ outputSchema }) run, info.usage is undefined and info.finishReason is null (no agent-loop iteration produced RUN_FINISHED). To capture finalization tokens, use onUsage — it fires for both agent-loop iterations and the final call. For the structured-output result itself, observe the structured-output.complete CUSTOM event in onChunk.

onStructuredOutputConfig

A dedicated config hook that fires only at the structured-output boundary (when chat({ outputSchema }) is invoked). Use it to transform the JSON Schema sent to the provider (inject $defs, strip vendor-incompatible keywords) or to apply structured-output-specific config changes that should not affect the agent-loop adapter calls.

Signature:

onStructuredOutputConfig?: (
  ctx: ChatMiddlewareContext,
  config: StructuredOutputMiddlewareConfig,
) =>
  | void
  | null
  | Partial<StructuredOutputMiddlewareConfig>
  | Promise<void | Partial<StructuredOutputMiddlewareConfig>>

StructuredOutputMiddlewareConfig shape:

interface StructuredOutputMiddlewareConfig extends ChatMiddlewareConfig {
  outputSchema: JSONSchema // The JSON Schema being sent to the provider
}

Ordering rule:

• onStructuredOutputConfig fires before onConfig at the structured-output boundary.
• onConfig re-fires at the same boundary with ctx.phase === 'structuredOutput', receiving the post-onStructuredOutputConfig view of the config (minus outputSchema).
• Use onConfig for general-purpose transforms that apply to every adapter call (agent-loop iterations and the final structured-output call).
• Use onStructuredOutputConfig when you need to transform the JSON Schema or apply structured-output-specific behavior.

Core Patterns

Pattern 1: Analytics and Logging Middleware

Use onStart, onFinish, onUsage, and onError for comprehensive observability. Use ctx.defer() for non-blocking async side effects that should not block the stream.

import {
  chat,
  toServerSentEventsResponse,
  type ChatMiddleware,
} from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

const analytics: ChatMiddleware = {
  name: 'analytics',
  onStart: (ctx) => {
    console.log(`[${ctx.requestId}] Chat started — model: ${ctx.model}`)
  },
  onUsage: (ctx, usage) => {
    console.log(`[${ctx.requestId}] Tokens: ${usage.totalTokens}`)
  },
  onFinish: (ctx, info) => {
    ctx.defer(
      fetch('/api/analytics', {
        method: 'POST',
        body: JSON.stringify({
          requestId: ctx.requestId,
          model: ctx.model,
          duration: info.duration,
          tokens: info.usage?.totalTokens,
          finishReason: info.finishReason,
        }),
      }),
    )
  },
  onError: (ctx, info) => {
    ctx.defer(
      fetch('/api/errors', {
        method: 'POST',
        body: JSON.stringify({
          requestId: ctx.requestId,
          error: String(info.error),
          duration: info.duration,
        }),
      }),
    )
  },
}

const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  middleware: [analytics],
})

return toServerSentEventsResponse(stream)

Pattern 2: Tool Interception Middleware

Use onBeforeToolCall to validate, gate, or transform tool arguments before execution. Use onAfterToolCall to log results and timing. The first middleware that returns a non-void decision from onBeforeToolCall short-circuits remaining middleware for that call.

import type { ChatMiddleware } from '@tanstack/ai'

const toolGuard: ChatMiddleware = {
  name: 'tool-guard',
  onBeforeToolCall: (ctx, hookCtx) => {
    // Block dangerous tools
    if (hookCtx.toolName === 'deleteDatabase') {
      return { type: 'abort', reason: 'Dangerous operation blocked' }
    }

    // Enforce default arguments
    if (hookCtx.toolName === 'search' && !hookCtx.args.limit) {
      return {
        type: 'transformArgs',
        args: { ...hookCtx.args, limit: 10 },
      }
    }

    // Return void to continue normally
  },
  onAfterToolCall: (ctx, info) => {
    if (info.ok) {
      console.log(`${info.toolName} completed in ${info.duration}ms`)
    } else {
      console.error(`${info.toolName} failed:`, info.error)
    }
  },
}

onBeforeToolCall decision types:

Pattern 3: Structured-Output Middleware

When chat({ outputSchema }) is used, the final structured-output adapter call now flows through the same middleware chain as the agent loop (with ctx.phase === 'structuredOutput'). Before this change, the final call bypassed middleware entirely — onChunk, onUsage, onConfig, and terminal hooks did not see it.

Example A — Observability (tracing every chunk, including finalization):

import type { ChatMiddleware } from '@tanstack/ai'

const tracing: ChatMiddleware = {
  name: 'tracing',
  onChunk(ctx, chunk) {
    span.addEvent('chunk', { phase: ctx.phase, type: chunk.type })
  },
}

This middleware now observes every chunk from the final structured-output call, attributed to ctx.phase === 'structuredOutput'. Before the fix, the final adapter call bypassed middleware entirely — tracing would only see agent-loop chunks.

Example B — Schema rewriting (inject shared $defs):

import type { ChatMiddleware } from '@tanstack/ai'

const injectDefs: ChatMiddleware = {
  name: 'inject-defs',
  onStructuredOutputConfig(_ctx, config) {
    return {
      outputSchema: { ...config.outputSchema, $defs: { ...sharedDefs } },
    }
  },
}

onStructuredOutputConfig is the right hook here because it has direct access to config.outputSchema and runs only on the structured-output boundary — schema rewrites do not leak into the agent-loop adapter calls.

Pattern 4: Multiple Middleware Composition

Middleware executes in array order (left-to-right). Ordering matters for hooks that pipe or short-circuit:

import { chat, type ChatMiddleware } from '@tanstack/ai'
import { toolCacheMiddleware } from '@tanstack/ai/middlewares'
import { openaiText } from '@tanstack/ai-openai'

const logging: ChatMiddleware = {
  name: 'logging',
  onStart: (ctx) => console.log(`[${ctx.requestId}] started`),
  onChunk: (ctx, chunk) => {
    console.log(`[${ctx.requestId}] chunk: ${chunk.type}`)
  },
  onFinish: (ctx, info) => {
    console.log(`[${ctx.requestId}] done in ${info.duration}ms`)
  },
}

const configTransform: ChatMiddleware = {
  name: 'config-transform',
  onConfig: (ctx, config) => {
    if (ctx.phase === 'init') {
      return {
        systemPrompts: [...config.systemPrompts, 'Always respond in JSON.'],
        // Sampling options are NOT first-class config fields — mutate them
        // through `config.modelOptions` using the provider's native key.
        // (e.g. OpenAI `temperature` / `max_output_tokens`.)
        modelOptions: { ...config.modelOptions, temperature: 0.2 },
      }
    }
  },
}

const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  tools: [weatherTool, stockTool],
  middleware: [
    logging, // Runs first
    configTransform, // Transforms config second
    toolCacheMiddleware({ ttl: 60_000 }), // Caches tool results third
  ],
})

Composition rules by hook:

Built-in: toolCacheMiddleware

Caches tool call results by name + arguments. Import from @tanstack/ai/middlewares:

import { chat } from '@tanstack/ai'
import { toolCacheMiddleware } from '@tanstack/ai/middlewares'

const stream = chat({
  adapter,
  messages,
  tools: [weatherTool],
  middleware: [
    toolCacheMiddleware({
      ttl: 60_000, // Cache entries expire after 60 seconds
      maxSize: 50, // Max 50 entries (LRU eviction)
      toolNames: ['getWeather'], // Only cache specific tools
    }),
  ],
})

Options: maxSize (default 100), ttl (default Infinity), toolNames (default all), keyFn (custom cache key), storage (custom backend like Redis). See docs/advanced/middleware.md for custom storage examples.

Sandbox File-Event Hooks (sandbox group)

Declare a sandbox: ChatSandboxHooks group on defineChatMiddleware to react to every file created/changed/deleted inside a sandbox provided by withSandbox (from @tanstack/ai-sandbox). These fire per-run, server-side, and each handler receives the run's ChatMiddlewareContext as the first argument:

import { defineChatMiddleware } from '@tanstack/ai'
import { db } from './db'

const auditMiddleware = defineChatMiddleware({
  name: 'audit',
  sandbox: {
    onFile: (ctx, e) => console.log(ctx.runId, e.type, e.path),
    onFileCreate: (ctx, e) => db.log({ run: ctx.runId, event: e }),
  },
})

These are independent of the stream: the engine also emits a sandbox.file CUSTOM chunk per change regardless of whether any sandbox hooks are registered, so a client can react to the same edits without middleware. See ai-core/ag-ui-protocol/SKILL.md for reading that chunk (and the opt-in sandbox.file.diff chunk) off ChatStream.

before() / after() / diff() — lazy, git-backed content accessors

Each hook receives a SandboxFileHookEvent: the serializable { type, path, timestamp } plus three lazy accessors for the file's content:

interface SandboxFileHookEvent {
  type: 'create' | 'change' | 'delete'
  path: string
  timestamp: number
  before(): Promise<string> // content at the session baseline ('' if new / non-git)
  after(): Promise<string> // current content ('' if deleted)
  diff(): Promise<string> // unified patch vs the baseline
}

import { defineChatMiddleware } from '@tanstack/ai'
import { db } from './db'

const auditMiddleware = defineChatMiddleware({
  name: 'audit',
  sandbox: {
    onFileChange: async (ctx, e) => {
      const [before, after] = await Promise.all([e.before(), e.after()])
      db.log({ run: ctx.runId, path: e.path, before, after })
    },
  },
})

Lazy — path-only hooks pay nothing. before(), after(), and diff() are methods, not fields: each only reads the file or shells out to git when called. A hook that only reads e.path/e.type (like the onFile logger above) never touches the filesystem or spawns a process.

Git session baseline. The sandbox snapshots git rev-parse HEAD once at setup as the session baseline (empty string if the workspace isn't a git repo or has no commits). before() and diff() always diff against that same fixed baseline for the rest of the run, so onFileChange reports the file's cumulative change since the run started, not just the delta since the last poll. after() always reads current on-disk content. None of the three accessors throw: a deleted file resolves after() to '' (it still has before()); a new file resolves before() to '' (it still has after()); a non-git workspace resolves both before() and after() to '' and makes diff() fall back to a synthesized add-patch built from after() — except for a delete event in a non-git workspace, where there's nothing to synthesize and diff() resolves to ''. In a git workspace a file git isn't tracking yet (a file the agent created, and every later edit to it) diffs empty because git diff ignores untracked files, so diff() falls back to the same synthesized add-patch whenever the file is absent at the baseline — a create-or-edit of an untracked file never streams an empty diff. An empty diff for a tracked file (identical to the baseline) stays empty, as it should. A git-ignored file is withheld: the file event still fires (you're notified it changed) but diff() returns '', so a secret like a .env never has its contents surfaced in the diff feed.

Failures are logged, not silent. Every git/exec/fs failure behind these accessors (and behind the find-poll watcher) still falls back to ''/an empty snapshot, but logs first: real anomalies (a failed git diff, an unreadable file, a lost find poll) under the errors category (on by default); expected-empty conditions (a new file's before(), a non-git baseline) under the sandbox debug category.

Hook errors are swallowed per hook. A throwing sandbox hook is caught and logged under the errors category (on by default) — it cannot break the run or stop other hooks (or the sandbox.file chunk) from continuing.

Source: docs/sandbox/observability.md

Common Mistakes

a. MEDIUM: Trying to modify StreamChunks in middleware

// WRONG -- mutating the chunk object directly
const broken: ChatMiddleware = {
  name: 'broken',
  onChunk: (ctx, chunk) => {
    chunk.delta = 'modified' // Mutation does nothing; chunk is not modified in-place
  },
}

// CORRECT -- return a new chunk to replace the original
const correct: ChatMiddleware = {
  name: 'correct',
  onChunk: (ctx, chunk) => {
    if (chunk.type === 'TEXT_MESSAGE_CONTENT') {
      return { ...chunk, delta: chunk.delta.replace(/secret/g, '[REDACTED]') }
    }
    // Return void to pass through unchanged
  },
}

Middleware onChunk hooks are functional transforms. Return a new chunk, an array of chunks, null (to drop), or void (to pass through). Mutating the input object has no effect on the stream output.

Source: docs/advanced/middleware.md

b. MEDIUM: Middleware exceptions breaking the stream

// WRONG -- unhandled error kills the entire streaming response
const fragile: ChatMiddleware = {
  name: 'fragile-analytics',
  onFinish: async (ctx, info) => {
    // If this fetch fails, the stream breaks
    await fetch('/api/analytics', {
      method: 'POST',
      body: JSON.stringify({ duration: info.duration }),
    })
  },
}

// CORRECT -- wrap in try-catch and/or use ctx.defer()
const resilient: ChatMiddleware = {
  name: 'resilient-analytics',
  onFinish: (ctx, info) => {
    // Option 1: defer (non-blocking, errors are isolated)
    ctx.defer(
      fetch('/api/analytics', {
        method: 'POST',
        body: JSON.stringify({ duration: info.duration }),
      }),
    )
  },
  onChunk: (ctx, chunk) => {
    // Option 2: try-catch for synchronous/critical hooks
    try {
      logChunk(chunk)
    } catch (err) {
      console.error('Logging failed:', err)
    }
    // Return void to pass through
  },
}

Wrap all middleware hooks in try-catch to prevent analytics or logging failures from killing the chat stream. For async side effects, prefer ctx.defer() which runs after the terminal hook and isolates failures.

Source: docs/advanced/middleware.md

Cross-References

• See also: ai-core/chat-experience/SKILL.md -- Middleware hooks into the chat lifecycle
• See also: ai-core/structured-outputs/SKILL.md -- Middleware now wraps the final structured-output call; use onStructuredOutputConfig for JSON-Schema transforms
• See also: ai-core/ag-ui-protocol/SKILL.md -- Reading the sandbox.file / sandbox.file.diff CUSTOM chunks the sandbox runtime emits alongside these sandbox hooks, via ChatStream's typed KnownCustomEvent narrowing

TanStack AI — Core Concepts

TanStack AI is a type-safe, provider-agnostic AI SDK. Server-side functions live in @tanstack/ai and provider adapter packages. Client-side hooks live in framework packages (@tanstack/ai-react, @tanstack/ai-solid, etc.). Always import from the framework package on the client — never from @tanstack/ai-client directly (unless vanilla JS).

Sub-Skills

Quick Decision Tree

• Setting up a chatbot? → ai-core/chat-experience
• Adding function calling? → ai-core/tool-calling
• Generating media (images, audio, video)? → ai-core/media-generation
• Need structured JSON output? → ai-core/structured-outputs
• Choosing/configuring a provider? → ai-core/adapter-configuration
• Building a server-only AG-UI backend? → ai-core/ag-ui-protocol
• Adding analytics or post-stream events? → ai-core/middleware
• Connecting to a custom backend? → ai-core/custom-backend-integration
• Turning on debug logging to trace chunks/tools/middleware? → ai-core/debug-logging
• Debugging mistakes? → Check Common Mistakes in the relevant sub-skill

Critical Rules

1. This is NOT the Vercel AI SDK. Use chat() not streamText(). Use openaiText() not createOpenAI(). Import from @tanstack/ai, not ai.
2. Import from framework package on client. Use @tanstack/ai-react (or solid/vue/svelte/preact), not @tanstack/ai-client.
3. Use toServerSentEventsResponse() to convert streams to HTTP responses. Never implement SSE manually.
4. Use middleware for lifecycle events. No onEnd/onFinish callbacks on chat() — use middleware: [{ onFinish: ... }].
5. Ask the user which adapter and model they want. Suggest the latest model. Also ask if they want Code Mode.
6. Tools must be passed to both server and client. Server gets the tool in chat({ tools }), client gets the definition in useChat({ clientTools }).

Version

Targets TanStack AI v0.10.0.

Tool Calling

This skill builds on ai-core. Read it first for critical rules.

Setup

Complete end-to-end example: shared definition, server tool, client tool, server route, React client.

// tools/definitions.ts
import { toolDefinition } from '@tanstack/ai'
import { z } from 'zod'

export const getProductsDef = toolDefinition({
  name: 'get_products',
  description: 'Search for products in the catalog',
  inputSchema: z.object({
    query: z.string().meta({ description: 'Search keyword' }),
    limit: z.number().optional().meta({ description: 'Max results' }),
  }),
  outputSchema: z.object({
    products: z.array(
      z.object({ id: z.string(), name: z.string(), price: z.number() }),
    ),
  }),
})

export const updateCartUIDef = toolDefinition({
  name: 'update_cart_ui',
  description: 'Update the shopping cart UI with item count',
  inputSchema: z.object({ itemCount: z.number(), message: z.string() }),
  outputSchema: z.object({ displayed: z.boolean() }),
})

// tools/server.ts
import { getProductsDef } from './definitions'

export const getProducts = getProductsDef.server(async ({ query, limit }) => {
  const results = await db.products.search(query, { limit: limit ?? 10 })
  return {
    products: results.map((p) => ({ id: p.id, name: p.name, price: p.price })),
  }
})

// api/chat/route.ts
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { getProducts } from '@/tools/server'
import { updateCartUIDef } from '@/tools/definitions'

export async function POST(request: Request) {
  const { messages } = await request.json()
  const stream = chat({
    adapter: openaiText('gpt-5.5'),
    messages,
    tools: [getProducts, updateCartUIDef], // server tool + client definition
  })
  return toServerSentEventsResponse(stream)
}

// app/chat.tsx
import {
  useChat,
  fetchServerSentEvents,
  clientTools,
  createChatClientOptions,
  type InferChatMessages,
} from "@tanstack/ai-react";
import { updateCartUIDef } from "@/tools/definitions";
import { useState } from "react";

function ChatPage() {
  const [cartCount, setCartCount] = useState(0);

  const updateCartUI = updateCartUIDef.client((input) => {
    setCartCount(input.itemCount);
    return { displayed: true };
  });

  const tools = clientTools(updateCartUI);
  const chatOptions = createChatClientOptions({
    connection: fetchServerSentEvents("/api/chat"),
    tools,
  });
  type Messages = InferChatMessages<typeof chatOptions>;

  const { messages, sendMessage } = useChat(chatOptions);

  return (
    <div>
      <span>Cart: {cartCount}</span>
      {(messages as Messages).map((msg) => (
        <div key={msg.id}>
          {msg.parts.map((part) => {
            if (part.type === "text") return <p>{part.content}</p>;
            if (part.type === "tool-call") {
              return <div key={part.id}>Tool: {part.name} ({part.state})</div>;
            }
            return null;
          })}
        </div>
      ))}
    </div>
  );
}

Core Patterns

Pattern 1: Server-Only Tool

Define with toolDefinition(), implement with .server(), pass to chat({ tools }). The server executes it automatically. The client never runs code for this tool.

import { toolDefinition } from '@tanstack/ai'
import { z } from 'zod'

const getUserDataDef = toolDefinition({
  name: 'get_user_data',
  description: 'Look up user by ID',
  inputSchema: z.object({
    userId: z.string().meta({ description: "The user's ID" }),
  }),
  outputSchema: z.object({ name: z.string(), email: z.string() }),
})

const getUserData = getUserDataDef.server(async ({ userId }) => {
  const user = await db.users.findUnique({ where: { id: userId } })
  return { name: user.name, email: user.email }
})

// In your route handler:
const stream = chat({
  adapter: openaiText('gpt-5.5'),
  messages,
  tools: [getUserData],
})

Pattern 2: Client-Only Tool

Pass the bare definition (no .server()) to chat({ tools }) so the LLM knows about it. Pass the .client() implementation to useChat via clientTools().

import { toolDefinition } from '@tanstack/ai'
import { z } from 'zod'

export const showNotificationDef = toolDefinition({
  name: 'show_notification',
  description: 'Display a toast notification to the user',
  inputSchema: z.object({
    message: z.string(),
    type: z.enum(['success', 'error', 'info']),
  }),
  outputSchema: z.object({ shown: z.boolean() }),
})

Server -- pass definition only (no execute function):

const stream = chat({
  adapter: openaiText('gpt-5.5'),
  messages,
  tools: [showNotificationDef],
})

Client -- pass .client() implementation:

import {
  useChat,
  fetchServerSentEvents,
  clientTools,
  createChatClientOptions,
} from "@tanstack/ai-react";
import { showNotificationDef } from "@/tools/definitions";
import { useState } from "react";

function ChatPage() {
  const [toast, setToast] = useState<string | null>(null);

  const showNotification = showNotificationDef.client((input) => {
    setToast(input.message);
    setTimeout(() => setToast(null), 3000);
    return { shown: true };
  });

  const { messages, sendMessage } = useChat(
    createChatClientOptions({
      connection: fetchServerSentEvents("/api/chat"),
      tools: clientTools(showNotification),
    })
  );

  return (
    <div>
      {toast && <div className="toast">{toast}</div>}
      {messages.map((msg) => (
        <div key={msg.id}>
          {msg.parts.map((part) =>
            part.type === "text" ? <p>{part.content}</p> : null
          )}
        </div>
      ))}
    </div>
  );
}

Pattern 3: Tool with Approval Flow

Set needsApproval: true in the definition. Execution pauses until the client calls addToolApprovalResponse(). The part has state: "approval-requested" and an approval object with an id.

import { toolDefinition } from '@tanstack/ai'
import { z } from 'zod'

export const sendEmailDef = toolDefinition({
  name: 'send_email',
  description: 'Send an email to a recipient',
  inputSchema: z.object({
    to: z.string().email(),
    subject: z.string(),
    body: z.string(),
  }),
  outputSchema: z.object({ success: z.boolean(), messageId: z.string() }),
  needsApproval: true,
})

export const sendEmail = sendEmailDef.server(async ({ to, subject, body }) => {
  const result = await emailService.send({ to, subject, body })
  return { success: true, messageId: result.id }
})

Client -- render approval UI and respond:

import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";

function ChatPage() {
  const { messages, addToolApprovalResponse } = useChat({
    connection: fetchServerSentEvents("/api/chat"),
  });

  return (
    <div>
      {messages.map((msg) => (
        <div key={msg.id}>
          {msg.parts.map((part) => {
            if (part.type === "text") return <p>{part.content}</p>;
            if (
              part.type === "tool-call" &&
              part.state === "approval-requested" &&
              part.approval
            ) {
              return (
                <div key={part.id}>
                  <p>Approve "{part.name}"?</p>
                  {/* `part.input` is the parsed, typed object (populated once
                      the arguments are complete, as they are at approval
                      time); `part.arguments` remains the raw JSON string. */}
                  <pre>{JSON.stringify(part.input, null, 2)}</pre>
                  <button
                    onClick={() =>
                      addToolApprovalResponse({
                        id: part.approval!.id,
                        approved: true,
                      })
                    }
                  >
                    Approve
                  </button>
                  <button
                    onClick={() =>
                      addToolApprovalResponse({
                        id: part.approval!.id,
                        approved: false,
                      })
                    }
                  >
                    Deny
                  </button>
                </div>
              );
            }
            return null;
          })}
        </div>
      ))}
    </div>
  );
}

Type-safe approval: With typed tools, part.approval exists only on parts for tools defined with needsApproval: true. Tools without approval have no approval field (reading it is a compile error). For a tool-agnostic handler over a typed union, narrow with 'approval' in part (if (part.type === 'tool-call' && 'approval' in part && part.approval)), or type a shared component against the base ToolCallPart. An untyped useChat() keeps approval on every tool-call part, which is why the snippet above (no tools generic) reads it directly.

Pattern 4: Lazy Tool Discovery

Set lazy: true on rarely-needed tools. The LLM sees their names via a synthetic __lazy__tool__discovery__ tool and discovers schemas on demand. Saves tokens.

import {
  toolDefinition,
  chat,
  toServerSentEventsResponse,
  maxIterations,
} from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { z } from 'zod'

const getProductsDef = toolDefinition({
  name: 'getProducts',
  description: 'List all products',
  inputSchema: z.object({}),
  outputSchema: z.array(
    z.object({ id: z.number(), name: z.string(), price: z.number() }),
  ),
})
const getProducts = getProductsDef.server(async () => db.products.findMany())

const compareProductsDef = toolDefinition({
  name: 'compareProducts',
  description: 'Compare two or more products side by side',
  inputSchema: z.object({ productIds: z.array(z.number()).min(2) }),
  lazy: true, // not sent to LLM upfront
})
const compareProducts = compareProductsDef.server(async ({ productIds }) => {
  return db.products.findMany({ where: { id: { in: productIds } } })
})

export async function POST(request: Request) {
  const { messages } = await request.json()
  const stream = chat({
    adapter: openaiText('gpt-5.5'),
    messages,
    tools: [getProducts, compareProducts],
    agentLoopStrategy: maxIterations(20),
  })
  return toServerSentEventsResponse(stream)
}

The LLM sees getProducts and __lazy__tool__discovery__ upfront. To compare, it first calls __lazy__tool__discovery__({ toolNames: ["compareProducts"] }), gets the full schema, then calls compareProducts directly. Once discovered, a tool stays available for the conversation. When all lazy tools are discovered, the discovery tool is removed automatically.

Tuning the lazy catalog with lazyToolsConfig

By default the discovery-tool catalog lists only bare names ('none'). Pass lazyToolsConfig to chat() to include more context:

const stream = chat({
  adapter: openaiText('gpt-5.5'),
  messages,
  tools: [getProducts, compareProducts],
  agentLoopStrategy: maxIterations(20),
  lazyToolsConfig: { includeDescription: 'first-sentence' },
})

includeDescription values:

The post-discovery payload always returns the full description and schema regardless of this setting.

MCP Tools

@tanstack/ai-mcp lets a server-side chat() call discover and invoke tools hosted on any MCP server (Streamable HTTP, SSE, or stdio).

MCP tools and UI resources: When an MCP tool result carries a ui:// resource URI (via _meta.ui.resourceUri), TanStack AI surfaces it as a UIResourcePart on the assistant UIMessage in the client message list. UIResourcePart is a presentational-only part — it never enters model input. See the @tanstack/ai-mcp skill for the full MCP Apps API (createMcpAppCallHandler, createMcpAppBridge, MCPAppResource).

Basic usage — auto-discovery

// src/routes/api.chat.ts
import { createFileRoute } from '@tanstack/react-router'
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { createMCPClient } from '@tanstack/ai-mcp'

export const Route = createFileRoute('/api/chat')({
  server: {
    handlers: {
      POST: async ({ request }) => {
        const { messages } = await request.json()

        // 1. Connect to the MCP server.
        const mcp = await createMCPClient({
          transport: { type: 'http', url: 'https://mcp.example.com/mcp' },
        })

        // 2. Discover all tools from the server (returns ServerTool[]).
        const mcpTools = await mcp.tools()

        // 3. Spread them into chat() — they work exactly like hand-written tools.
        // Caller owns the lifecycle — chat() never closes the client. Tools run
        // while the response streams, so close in a middleware terminal hook
        // (a try/finally around the return would close before tools execute).
        const stream = chat({
          adapter: openaiText('gpt-5.5'),
          messages,
          tools: [...mcpTools],
          middleware: [
            {
              name: 'mcp-close',
              onFinish: () => mcp.close(),
              onAbort: () => mcp.close(),
              onError: () => mcp.close(),
            },
          ],
        })
        return toServerSentEventsResponse(stream)
      },
    },
  },
})

Typed path — pass toolDefinition instances

Pass bare toolDefinition() instances (no .server()) to client.tools([...]). The MCP client supplies a callTool proxy as the execute function, while input/output validation and types come from the definitions' Zod schemas.

import { toolDefinition } from '@tanstack/ai'
import { createMCPClient } from '@tanstack/ai-mcp'
import { z } from 'zod'

const getWeather = toolDefinition({
  name: 'get_weather',
  description: 'Current weather for a city',
  inputSchema: z.object({ city: z.string() }),
  outputSchema: z.object({ temperature: z.number(), conditions: z.string() }),
})

const mcp = await createMCPClient({
  transport: { type: 'http', url: 'https://mcp.example.com/mcp' },
})

// Returns ServerTool[] typed to the definitions' input/output schemas.
// Throws MCPToolNotFoundError if the server does not expose a tool with that name.
const tools = await mcp.tools([getWeather])

const stream = chat({ adapter: openaiText('gpt-5.5'), messages, tools })

Multiple servers with createMCPClients

import { createMCPClients } from '@tanstack/ai-mcp'

// Each key becomes the default prefix for that server's tools.
await using pool = await createMCPClients({
  github: { transport: { type: 'http', url: 'https://mcp.github.com/mcp' } },
  linear: { transport: { type: 'http', url: 'https://mcp.linear.app/mcp' } },
})

// Tools auto-prefixed: 'github_search_repos', 'linear_create_issue', etc.
const tools = await pool.tools()

const stream = chat({ adapter: openaiText('gpt-5.5'), messages, tools })

Use pool.clients.<name> for typed per-server access (resources, prompts, typed tools([defs]) overload).

ToolExecutionContext.abortSignal — cancelling long-running tools

Every server tool's execute function now receives abortSignal in its context. When the chat run aborts (e.g. the client disconnects or calls the run's abortController), the signal fires and any in-flight callTool call is cancelled automatically.

You can also forward it from your own server tools:

const longRunningTool = myToolDef.server(async (args, ctx) => {
  // Forward to fetch, a DB query, or an MCP callTool call.
  const response = await fetch('https://slow.api/data', {
    signal: ctx?.abortSignal,
  })
  return response.json()
})

MCP tools wire this automatically — makeMcpExecute passes ctx?.abortSignal as the signal option to client.callTool(...), so MCP server calls cancel with the chat run without any extra code.

stdio transport (Node-only)

import { createMCPClient } from '@tanstack/ai-mcp'
import { stdioTransport } from '@tanstack/ai-mcp/stdio'

const mcp = await createMCPClient({
  transport: stdioTransport({ command: 'npx', args: ['-y', 'my-mcp-server'] }),
})

Import stdioTransport from the /stdio subpath only — it contains Node.js child_process imports and must not be bundled for edge runtimes.

chat({ mcp }) — discovery + lifecycle in one prop

Instead of manually calling client.tools() and managing close(), pass an mcp object and let chat() handle discovery and lifecycle.

// Prop shape (ChatMCPOptions):
// mcp: {
//   clients: Array<MCPClient | MCPClients>,
//   connection?: 'close' | 'keep-alive',  // default: 'close'
//   lazyTools?: boolean,
//   onDiscoveryError?: (error: unknown, source) => void,
// }

• At run start, chat() calls .tools() on every entry in clients and merges the results — identical to spreading await client.tools() into tools: [...].
• lazyTools: true is forwarded to tools({ lazy: true }).
• onDiscoveryError: throw to fail-fast; return to skip that source.
• connection: 'close' (default) closes each client when the run ends (after the agent loop completes and the stream is drained). With 'keep-alive', chat() never closes the clients — the caller owns their lifecycle (keep connections warm across requests).

When to use mcp vs. the tools spread:

Example:

import { createFileRoute } from '@tanstack/react-router'
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { createMCPClient } from '@tanstack/ai-mcp'

export const Route = createFileRoute('/api/chat')({
  server: {
    handlers: {
      POST: async ({ request }) => {
        const { messages } = await request.json()

        const mcpClient = await createMCPClient({
          transport: { type: 'http', url: 'https://mcp.example.com/mcp' },
        })

        const stream = chat({
          adapter: openaiText('gpt-5.5'),
          messages,
          mcp: {
            clients: [mcpClient],
            connection: 'keep-alive',
            onDiscoveryError: (err, source) => {
              console.warn('MCP discovery failed, skipping source:', err)
              // returning (not throwing) skips this source and continues
            },
          },
        })

        return toServerSentEventsResponse(stream)
      },
    },
  },
})

Provider Skills

Not to be confused with @tanstack/ai-code-mode-skills, which are locally-generated TypeScript functions executed client-side. Provider Skills are hosted, provider-managed bundles that the model loads on demand and runs inside the provider's server-side sandbox.

Provider Skills are inert without an execution tool. The execution tool is what activates the sandbox; skills are additional capability bundles that run inside it:

• Anthropic: skills require the code_execution tool (@tanstack/ai-anthropic/tools).
• OpenAI: skills live inside the shell tool (@tanstack/ai-openai/tools) and are Responses API only.

Anthropic: codeExecutionTool with skills

Import from @tanstack/ai-anthropic/tools:

import { codeExecutionTool } from '@tanstack/ai-anthropic/tools'
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { anthropicText } from '@tanstack/ai-anthropic'

export async function POST(request: Request) {
  const { messages } = await request.json()
  const stream = chat({
    adapter: anthropicText('claude-sonnet-4-5'),
    messages,
    tools: [
      codeExecutionTool(
        { type: 'code_execution_20250825', name: 'code_execution' },
        {
          skills: [{ type: 'anthropic', skill_id: 'pptx', version: 'latest' }],
        },
      ),
    ],
  })
  return toServerSentEventsResponse(stream)
}

AnthropicContainerSkill shape: { type: 'anthropic' | 'custom'; skill_id: string; version?: string }. Constraints: max 8 skills per request; skill_id must be 1–64 characters.

The adapter automatically:

• Lifts the skills into the request's top-level container.skills param (the shape Anthropic's API requires).
• Attaches the required beta headers (code-execution-2025-08-25 plus skills-2025-10-02 when skills are present). You do not set these manually.

Deprecation: Setting skills via modelOptions.container.skills is deprecated. Use codeExecutionTool(config, { skills }) instead — the legacy path bypasses the beta-header wiring.

OpenAI: shellTool with skills (Responses API only)

Import from @tanstack/ai-openai/tools:

import { shellTool } from '@tanstack/ai-openai/tools'
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

export async function POST(request: Request) {
  const { messages } = await request.json()
  const stream = chat({
    adapter: openaiText('gpt-5.2'),
    messages,
    tools: [
      shellTool({
        environment: {
          type: 'container_auto',
          skills: [
            { type: 'skill_reference', skill_id: 'skill_abc', version: '2' },
          ],
        },
      }),
    ],
  })
  return toServerSentEventsResponse(stream)
}

SkillReference shape: { type: 'skill_reference'; skill_id: string; version?: string }. version is a string — use a positive integer as a string (e.g. '2') or 'latest'. This is Responses API only; Chat Completions does not support the shell tool.

Scope

Only hosted/managed-by-id skills (type: 'anthropic' / type: 'custom' for Anthropic; type: 'skill_reference' for OpenAI) are wired. Inline bundles, local-path, and upload-API skill creation are not handled by these factories.

Common Mistakes

a. HIGH: Not passing tool definitions to both server and client

Server tools need chat({ tools }). Client tools need their definition in chat({ tools }) AND their .client() in useChat({ tools: clientTools(...) }).

Wrong -- tool only on server, client cannot execute:

chat({ adapter, messages, tools: [myToolDef] })
useChat({ connection: fetchServerSentEvents('/api/chat') }) // no tools

Wrong -- tool only on client, LLM does not know about it:

chat({ adapter, messages }); // no tools
useChat({ ..., tools: clientTools(myToolDef.client(() => result)) });

Correct:

chat({ adapter, messages, tools: [myToolDef] });
useChat({ ..., tools: clientTools(myToolDef.client((input) => ({ success: true }))) });

Source: docs/tools/tools.md

Cross-References

• See also: ai-core/chat-experience/SKILL.md -- Tools are used within chat
• See also: @tanstack/ai-code-mode package skills -- Code Mode is an alternative to tools for complex multi-step operations

Triage GitHub Issues, PRs & Discussions in Parallel

When to use

The user wants a prioritized view of everything open on the current repo's GitHub: which PRs to merge/review first, which issues to fix first, and which discussions need maintainer engagement. Trigger phrases include "triage the backlog", "what should I look at first", "prioritize open PRs and issues", "sweep open work", "triage discussions".

Do not invoke for single-item review (just look at it directly) or when the user wants ongoing automation (use /schedule instead).

Prerequisites

• gh CLI is authenticated (gh auth status). If not, stop and ask the user to authenticate — do not attempt to fix auth automatically.
• Run from inside a git repo whose origin points at the GitHub repo to triage. Confirm with gh repo view --json nameWithOwner,hasDiscussionsEnabled.
• If hasDiscussionsEnabled is false, skip the discussions section entirely (don't fetch, don't include in budget, note in report that discussions are disabled).

Procedure

1. Fetch open work

Run these gh calls in parallel. Use JSON so the downstream agent prompts are self-contained. The discussions call is a GraphQL query because gh has no built-in discussion list command.

gh pr list --state open --limit 200 --json number,title,url,author,createdAt,updatedAt,isDraft,mergeable,reviewDecision,labels,additions,deletions,changedFiles,statusCheckRollup

gh issue list --state open --limit 200 --json number,title,url,author,createdAt,updatedAt,labels,comments,reactionGroups

gh api graphql -f query='
query($owner: String!, $name: String!) {
  repository(owner: $owner, name: $name) {
    discussions(first: 100, orderBy: {field: UPDATED_AT, direction: DESC}, states: OPEN) {
      totalCount
      nodes {
        number
        title
        url
        createdAt
        updatedAt
        upvoteCount
        isAnswered
        locked
        category { name }
        author { login }
        labels(first: 5) { nodes { name } }
        comments(first: 0) { totalCount }
        reactions { totalCount }
      }
    }
  }
}' -F owner=<OWNER> -F name=<REPO>

Substitute <OWNER> and <REPO> from gh repo view --json nameWithOwner.

If the combined total exceeds 100 items, tell the user the counts (PRs / issues / discussions) and ask whether to cap at 100 most-recently-updated or split into batches. The agent cap is 100 total across all three categories.

2. Decide the parallel split

• Count nPRs, nIssues, nDiscussions.
• If nPRs + nIssues + nDiscussions <= 100: spawn one agent per item.
• Otherwise: prioritize PRs first (they block contributors), then issues by most-recently-updated, then discussions by most-recently-updated, up to the 100 budget. Note in the final report which items were skipped.

3. Fan out subagents

Dispatch all agents in a single message using multiple Agent tool calls (Claude Code parallelizes when they're in one block). Use subagent_type: "general-purpose" and run_in_background: false — you need the results synchronously to write the report.

Per-PR prompt template (substitute the bracketed values):

Triage GitHub PR [URL]. You have read-only access via `gh` and web tools.

Gather:
- `gh pr view [NUMBER] --json title,body,author,createdAt,updatedAt,isDraft,mergeable,mergeStateStatus,reviewDecision,labels,additions,deletions,changedFiles,statusCheckRollup,comments`
- `gh pr diff [NUMBER]` (skim — don't dump it)
- Recent review comments if any

Return ONLY a JSON object on a single line (no prose, no fences), matching:
{"kind":"pr","number":N,"title":"...","url":"...","author":"...","ageDays":N,"sizeLOC":N,"ciStatus":"passing|failing|pending|none","mergeable":true|false,"reviewState":"approved|changes_requested|review_required|none","draft":true|false,"priority":"P0|P1|P2|P3","reason":"<=140 chars","blockedBy":"<=80 chars or empty","recommendedAction":"merge|review|request-changes|close|wait"}

Priority rubric:
- P0: ready-to-merge (approved + green CI + mergeable + non-draft), or fixes broken main
- P1: small/focused, passing CI, needs review; or bug fix with clear reproduction
- P2: feature work, larger diff, no blockers
- P3: draft, stale (>30 days no activity), or has unresolved conflicts/failures

Be terse. One JSON object. No commentary.

Per-issue prompt template:

Triage GitHub issue [URL]. Read-only access via `gh`.

Gather:
- `gh issue view [NUMBER] --json title,body,author,createdAt,updatedAt,labels,comments,reactionGroups,assignees`
- Skim comments for repro steps, workarounds, related PRs

Return ONLY a JSON object on one line:
{"kind":"issue","number":N,"title":"...","url":"...","author":"...","ageDays":N,"reactions":N,"comments":N,"hasRepro":true|false,"linkedPR":"<url or empty>","category":"bug|feature|docs|question|chore","priority":"P0|P1|P2|P3","reason":"<=140 chars","recommendedAction":"fix|investigate|answer|close|wait-for-info"}

Priority rubric:
- P0: regression / data loss / security / blocks many users (high reactions + recent activity)
- P1: confirmed bug with reproduction, or high-engagement feature request
- P2: feature requests, minor bugs, docs gaps
- P3: questions, unreproducible, no activity in 60+ days

One JSON object. No commentary.

Per-discussion prompt template:

Triage GitHub discussion [URL]. Read-only access via `gh api graphql`.

Gather:
- `gh api graphql -f query='{ repository(owner:"<OWNER>", name:"<REPO>") { discussion(number: [NUMBER]) { title body url createdAt updatedAt upvoteCount isAnswered locked category { name } author { login } labels(first: 10) { nodes { name } } comments(first: 30) { totalCount nodes { author { login } body createdAt isAnswer upvoteCount } } reactions { totalCount } } } }'`
- Skim comments for: maintainer engagement, repro steps that suggest a real bug, links to issues/PRs

Return ONLY a JSON object on one line:
{"kind":"discussion","number":N,"title":"...","url":"...","author":"...","category":"Q&A|Ideas|General|Show and tell|Announcements|Polls|other","ageDays":N,"updatedDaysAgo":N,"upvotes":N,"comments":N,"reactions":N,"isAnswered":true|false|null,"maintainerEngaged":true|false,"looksLikeBug":true|false,"priority":"P0|P1|P2|P3","reason":"<=140 chars","recommendedAction":"answer|convert-to-issue|engage|mark-answered|close|wait"}

Priority rubric (category-aware):
- Q&A:
  - P0: unanswered AND (looksLikeBug OR upvotes>=5 OR ageDays>=7 with no maintainer reply)
  - P1: unanswered with clear question, some engagement, recent
  - P2: recently asked, no engagement yet, awaiting community signal
  - P3: effectively answered but not marked, stale (60+ days), or low-effort
- Ideas:
  - P0: upvotes>=10 AND updatedDaysAgo<=14 — strong roadmap demand
  - P1: upvotes>=3 with clear scope, or moderate engagement
  - P2: legitimate idea, low engagement so far
  - P3: stale, duplicate, off-roadmap, or off-topic
- General / Show and tell / Announcements / Polls / other:
  - P1: high engagement (upvotes+comments>=10) and recent — surfaces trends
  - P2: normal engagement
  - P3: low engagement, off-topic, or stale (60+ days)

Notes for recommendedAction:
- "convert-to-issue" if looksLikeBug is true and no linked issue exists
- "answer" for unanswered Q&A
- "mark-answered" for Q&A where a comment clearly answers but isAnswered is false
- "engage" for high-signal Ideas needing maintainer feedback
- "close" for off-topic, duplicate, or out-of-scope
- "wait" if community signal is still forming

One JSON object. No commentary.

4. Aggregate

Collect every agent's JSON line. If an agent returned prose instead of JSON (rare), extract what you can or mark priority: "P3", reason: "agent parse failed".

Sort:

1. PRs by priority (P0→P3), then by ageDays ascending within each tier (newer first for P0/P1 to capture momentum; for P3 by oldest first — those are stalest).
2. Issues by priority, then by reactions + comments desc within each tier.
3. Discussions by priority, then by upvotes * 2 + comments + reactions desc within each tier. Inside the same tier, surface Q&A above Ideas above other categories (response latency matters most for Q&A).

5. Write the report

Save to TRIAGE_REPORT.md at the repo root (or .agent/triage/TRIAGE_REPORT-YYYY-MM-DD.md if the repo has a .agent/ directory). Ask before overwriting an existing report from today.

Report skeleton:

# Triage Report — <repo nameWithOwner> — <YYYY-MM-DD>

Scanned **N PRs**, **M issues**, and **D discussions**. Skipped K items over the 100-agent budget (listed at bottom).

## PRs to review first

### P0 — merge/fix today

- [#NUM Title](url) — <reason>. _Action: <recommendedAction>_

### P1 — review this week

- [#NUM Title](url) — <reason>. _Action: <recommendedAction>_

### P2 — when time permits

<one-line per item>

### P3 — needs author input or close

<one-line per item>

## Issues to address first

### P0 — fix now

- [#NUM Title](url) — <reason>. _Action: <recommendedAction>_

### P1 — schedule this sprint

- [#NUM Title](url) — <reason>. _Action: <recommendedAction>_

### P2 — backlog

<one-line per item>

### P3 — close or ask for info

<one-line per item>

## Discussions to engage with

### P0 — respond today

- [#NUM Title](url) _(<category>)_ — <reason>. _Action: <recommendedAction>_

### P1 — respond this week

- [#NUM Title](url) _(<category>)_ — <reason>. _Action: <recommendedAction>_

### P2 — when time permits

<one-line per item, prefix with category>

### P3 — close, mark answered, or let community drive

<one-line per item, prefix with category>

## Skipped (over budget)

<list any items not triaged>

## How this was generated

N parallel triage agents ran via the `triage-github` skill on <date>. Each agent independently scored its item; this report aggregates and ranks them. Priorities are heuristic — sanity-check P0s before acting, especially `convert-to-issue` and `close` recommendations on discussions.

If discussions are disabled on the repo, omit the "Discussions to engage with" section and add a one-liner near the top noting they're disabled.

6. Summarize for the user

After writing the file, give the user a 3–5 line summary: total counts, top 3 PRs to review, top 3 issues to fix, top 3 discussions to engage with, and the report path. Do not paste the full report into chat.

Notes

• Cost: 100 agents is expensive. If the combined open-item total is small (say <20), just triage them yourself in the main thread instead of fanning out — mention this and proceed.
• Rate limits: gh shares one auth token; 100 concurrent gh calls usually fits inside GitHub's per-hour quota for authenticated users, but if the user has run heavy gh traffic recently, batch the agents in two waves of 50. Discussion GraphQL queries cost more rate-limit points per call than REST — factor that in.
• Failed agents: if an agent times out or returns garbage, include it in the report under a "Triage failures" subsection rather than silently dropping it.
• Don't take actions: this skill is read-only. Do not close issues, request changes, merge PRs, comment on discussions, convert discussions to issues, or post any reply. The report is for the human to act on.
• Discussion category names vary per repo. The common GitHub defaults are Q&A, Ideas, General, Show and tell, Announcements, Polls. Unknown categories should be tagged as "other" and ranked under the General rubric.

Sandboxes

Harness adapters declare requires: [SandboxCapability]. chat() errors unless some middleware provides it — withSandbox(...) does. The adapter then runs the agent CLI inside the sandbox and streams its events back.

Setup — Claude Code in a Docker sandbox

import { chat } from '@tanstack/ai'
import { claudeCodeText } from '@tanstack/ai-claude-code'
import {
  defineSandbox,
  defineWorkspace,
  withSandbox,
} from '@tanstack/ai-sandbox'
import { dockerSandbox } from '@tanstack/ai-sandbox-docker'

const sandbox = defineSandbox({
  id: 'repo-agent',
  provider: dockerSandbox({ image: 'node:22' }),
  workspace: defineWorkspace({
    source: { type: 'git', url: 'https://github.com/owner/repo', ref: 'main' },
    packageManager: 'pnpm',
    setup: ['corepack enable', 'pnpm install'],
    scripts: { test: 'pnpm test' },
    secrets: { ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY ?? '' },
  }),
  lifecycle: { reuse: 'thread', snapshot: 'after-setup', keepAlive: '30m' },
})

const stream = chat({
  threadId,
  adapter: claudeCodeText('sonnet'),
  messages,
  middleware: [withSandbox(sandbox)],
})

Type-safe secrets

import { createSecrets, bearer } from '@tanstack/ai-sandbox'

const secrets = createSecrets({
  GH: process.env.GH_TOKEN ?? '',
  SENTRY: process.env.SENTRY_TOKEN ?? '',
})
// secrets.GH is a SecretRef — the underlying string is stored in a
// non-enumerable symbol-keyed registry and never logged, snapshotted,
// or written to the sandbox store.

Pass secrets to defineWorkspace({ secrets }) so skill and MCP projectors can resolve them. Use secret: secrets.GH in gitSkill for private-repo auth and secrets.GH / bearer(secrets.GH) in MCP header values:

• secrets.GH — resolves to the raw token value.
• bearer(secrets.GH) — resolves to "Bearer <value>".

Declarative provisioning (skills, plugins, MCP, instructions)

import {
  agentSkill,
  gitSkill,
  mcpSkill,
  fileSkill,
  bearer,
  createSecrets,
  defineWorkspace,
} from '@tanstack/ai-sandbox'

const secrets = createSecrets({ GH: process.env.GH_TOKEN ?? '' })

defineWorkspace({
  source: { type: 'git', url: 'https://github.com/owner/repo' },
  secrets,
  skills: [
    agentSkill('tanstack'), // named skill (no-op with warning on CLIs that lack the concept)
    gitSkill({
      repo: 'owner/private-skills',
      secret: secrets.GH, // resolved at bootstrap time, never stored
      // into: '/abs/path/inside/sandbox'  // optional; defaults to .tanstack-skills/<repo>
    }),
    mcpSkill('my-mcp', {
      url: 'https://mcp.example.com',
      headers: { Authorization: bearer(secrets.GH) },
    }),
    fileSkill({ path: '.hints.md', content: 'Prefer pnpm.' }),
  ],
  plugins: ['@anthropic/plugin-foo'], // no-op with warning on CLIs without a plugin concept
  instructions: 'Always run `pnpm test` before proposing a change.',
})

Each skill type is projected per harness (Claude Code → .mcp.json; Codex → .codex/config.toml; OpenCode → opencode.json). instructions is written as AGENTS.md at the workspace root; CLAUDE.md and GEMINI.md are created as symlinks (falling back to copies on symlink failure). Skills/plugins that a CLI lacks emit a console.warn and are skipped.

gitSkill into field: an absolute path inside the sandbox where the repo is cloned. Defaults to <root>/.tanstack-skills/<repo-basename>.

Fast init

Shallow clone (depth)

githubRepo / gitSource default to --depth 1 --single-branch. Override:

import { githubRepo, defineWorkspace } from '@tanstack/ai-sandbox'

defineWorkspace({ source: githubRepo({ repo: 'owner/app' }) }) // depth 1 (default)
defineWorkspace({ source: githubRepo({ repo: 'owner/app', depth: 10 }) }) // 10 commits
defineWorkspace({ source: githubRepo({ repo: 'owner/app', depth: 'full' }) }) // full history

Serial / parallel setup callback

setup accepts a plain Array<string> (all serial) or a callback that records serial and parallel groups over a persistent shell whose cwd/env carry over between serial steps:

defineWorkspace({
  source: githubRepo({ repo: 'owner/app' }),
  setup: ({ serial, parallel }) => {
    serial('corepack enable')
    serial('pnpm install')
    parallel(['pnpm build', 'pnpm typecheck']) // concurrent; inherit cwd+env from shell
    serial('echo done')
  },
})

Snapshot-after-setup and snapshotMaxAge

When the provider supports snapshots, bootstrap takes one automatically after setup completes. Subsequent runs resume from the snapshot (skipping setup). Override or add a TTL:

lifecycle: {
  snapshot: 'after-setup', // default when provider.capabilities().snapshots
  snapshotMaxAge: '24h',   // re-create when the snapshot is older than this
}

Providers without snapshot support skip the step silently.

Providers

• localProcessSandbox() — runs on the host (no isolation; dev loop only).
• dockerSandbox({ image }) — isolated container; snapshots, fork, resume-by-id.

Both implement the same SandboxHandle: fs (read/write/list/mkdir/remove/ rename/exists), git (clone/status/add/commit/push/pull/branch), process (exec + duplex spawn), ports.connect(port), env.set, optional snapshot()/fork(), destroy(). Providers advertise support via capabilities(); calling an unsupported optional method throws UnsupportedCapabilityError.

Policy

import { defineSandboxPolicy } from '@tanstack/ai-sandbox'

const policy = defineSandboxPolicy({
  commands: {
    allow: ['pnpm test'],
    ask: ['curl *'],
    deny: ['sudo *', 'rm -rf *'],
  },
  capabilities: { fileWrite: 'allow', network: 'ask' },
  default: 'ask', // deny > ask > allow
})
// pass to defineSandbox({ policy }); harness adapters map it to native permissions

Lifecycle & resume

reuse: 'thread' resumes one sandbox per threadId; the compound key folds in provider + workspace hash + tenant so changing the repo/setup/image starts fresh. Ensure order: resume running → restore snapshot → create + bootstrap.

File-event hooks

Watch the workspace for create/change/delete events. Provider-agnostic: native fs.watch on local-process, a portable find poll on Docker/exec-only providers (no extra deps or image changes).

Declare hooks on defineSandbox({ hooks }) (sandbox-scoped) or on any chat middleware via the sandbox group (run-scoped):

import {
  defineSandbox,
  defineChatMiddleware,
  withSandbox,
} from '@tanstack/ai-sandbox'
import { dockerSandbox } from '@tanstack/ai-sandbox-docker'

// Sandbox-scoped hooks (all optional):
const sandbox = defineSandbox({
  id: 'repo-agent',
  provider: dockerSandbox({ image: 'node:22' }),
  hooks: {
    onFile: (e) => console.log(e.type, e.path), // catch-all
    onFileCreate: (e) => console.log('created', e.path),
    onFileChange: (e) => console.log('changed', e.path),
    onFileDelete: (e) => console.log('deleted', e.path),
    onReady: (handle) => console.log('ready', handle.id),
    onError: (err) => console.error(err),
    onDestroy: () => console.log('destroyed'),
  },
  fileEvents: true, // default; set false to disable watching entirely
})

// Run-scoped hooks via chat middleware (ctx is ChatMiddlewareContext):
const auditMiddleware = defineChatMiddleware({
  name: 'audit',
  sandbox: {
    onFile: (ctx, e) => console.log(ctx.runId, e.type, e.path),
    onFileCreate: (ctx, e) => db.log({ run: ctx.runId, event: e }),
    onFileChange: (ctx, e) => metrics.increment('file.change'),
    onFileDelete: (ctx, e) => console.warn('deleted', e.path),
  },
})

// No extra middleware needed — sandbox.file CUSTOM events are emitted
// automatically. Read them from the stream:
for await (const chunk of stream) {
  if (chunk.type === 'CUSTOM' && chunk.name === 'sandbox.file') {
    const value = chunk.value
    if (
      value !== null &&
      typeof value === 'object' &&
      'type' in value &&
      'path' in value
    ) {
      console.log('file event', value) // { type, path, timestamp }
    }
  }
}

watchWorkspace() is available as a low-level building block for watching outside a chat() run:

import { watchWorkspace } from '@tanstack/ai-sandbox'

const watcher = await watchWorkspace(handle, {
  onEvent: (e) => console.log(e.type, e.path),
  ignore: ['.git', 'node_modules'], // default
})
await watcher.stop()

Enable the sandbox debug category to log watcher start/stop, event dispatch, and lifecycle transitions:

chat({ threadId, adapter, messages, debug: { sandbox: true } })
// or debug: true to enable all categories

Edge / serverless execution

A request-scoped Worker can't hold a multi-minute agent run open. The serverless/edge model splits this: a trigger starts the run and returns immediately, a durable orchestrator drives it, and clients tail from a resumable cursor.

Core primitives (@tanstack/ai-sandbox, transport- and runtime-agnostic):

• RunEventLog / InMemoryRunEventLog — append-only, seq-indexed log of a run's StreamChunks with replay-then-tail reads. A dropped connection / new tab / hibernated orchestrator reconnect by passing their last-seen seq (read({ fromSeq })). TerminalRunStatus = done | error | aborted.
• pipeToRunLog / RunController — the run driver. pipeToRunLog pumps a chat() stream into a log and never rejects: a thrown stream error becomes a terminal RUN_ERROR event, so detached clients always observe failures. RunController.start is fire-and-track; attach(runId, { fromSeq }) tails; drain() awaits in-flight runs (e.g. in a waitUntil).
• Transport-agnostic tool-bridge — createToolBridgeCore + handleBridgeJsonRpc are the portable core; startHostToolBridge is the node:http host transport. The ToolBridgeProvisioner capability injects the transport, so an edge orchestrator serves the same core from its own fetch handler (no raw TCP listener). Default = host transport.
• Co-located host-tool seam — toolDescriptors / remoteToolStubs / httpRemoteToolExecutor (container side) + executeHostTool (orchestrator side): only chat()-tool EXECUTION crosses the container→orchestrator boundary, not the whole MCP protocol.
• SandboxCapabilities.writableStdin — false for providers (e.g. Cloudflare) with no writable host→process stdin; stdin-fed harnesses then deliver the prompt via a file + in-shell redirection (claude -p … < file).

Cloudflare runtime (@tanstack/ai-sandbox-cloudflare):

• createCloudflareSandboxAgent(config) → { Coordinator, Sandbox, worker } — an app's worker.ts is one configured call plus the wrangler-required DO re-exports. Two models via mode: do-drives (the DO runs chat()) and colocated (harness + bridge run in-container; the DO is a thin coordinator, pair with runInContainerHarness from /runner).
• DurableObjectRunEventLog mirrors InMemoryRunEventLog over DO storage; timingSafeBearerEqualWeb is the Web-Crypto constant-time bearer check.

Events

• claude-code.session-id (CUSTOM) — resumable session id → pass back via modelOptions.sessionId.
• file.changed (CUSTOM) — { path, diff } working-tree diff after the run.
• sandbox.file (CUSTOM) — { type, path, timestamp } per file create/change/ delete, emitted automatically when a sandbox is active.

Critical rules

• Harness adapters require a sandbox. Always include withSandbox(...) in middleware — without it chat() throws a missing-capability error.
• Secrets (workspace.secrets) are injected into the sandbox env and never persisted (no snapshots, no sandbox store, no event log). Always create them with createSecrets(...) so the values stay hidden behind SecretRef tokens. The agent binary (claude) must exist in the sandbox image (install it in setup or bake it into the image).
• Secret-bearing projected files (e.g. MCP config with resolved header values) are re-written on every projection call so rotated secrets re-apply; they are never included in a snapshot.
• chat()-provided tools are bridged into the in-sandbox agent over a host-side MCP tool-proxy: the agent calls them as mcp__tanstack__<tool> and each call is proxied back to the host where the tool's execute() runs (with its closures / DB / secrets). The agent also has its own native tools (Bash/Edit/Read/…). The host bridge binds on the host; the sandbox reaches it (localhost, or host.docker.internal for Docker), gated by a per-run bearer token.
• Use localProcessSandbox() only in trusted/dev contexts (no isolation).
• Skills/plugins that a CLI lacks (e.g. agentSkill on Codex, plugins on Codex) warn and skip — they do not throw.

Adapter Configuration

Dependency: This skill builds on ai-core. Read it first for critical rules.

Before implementing: Ask the user which provider and model they want. Then fetch the latest available models from the provider's source code (check the adapter's model metadata file, e.g. packages/ai-openai/src/model-meta.ts) or from the provider's API/docs to recommend the most current model. The model lists in this skill and its reference files may be outdated. Always verify against the source before recommending a specific model.

Setup

Create an adapter and use it with chat():

import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  modelOptions: {
    temperature: 0.7,
    max_output_tokens: 1000,
  },
})

return toServerSentEventsResponse(stream)

The adapter factory function takes the model name as a string literal and an optional config object (API key, base URL, etc.). The model name is passed into the factory, not into chat().

Sampling options (temperature, token limits, top_p/topP, etc.) live inside modelOptions using each provider's native key — they are not top-level options on chat(). See the per-provider table in Configuring Sampling below.

Core Patterns

1. Adapter Selection

Each provider has a dedicated package with tree-shakeable adapter factories. The text adapter is the primary one for chat/completions:

// Each factory takes model as first arg, optional config as second
import { openaiText } from '@tanstack/ai-openai'
import { anthropicText } from '@tanstack/ai-anthropic'
import { geminiText } from '@tanstack/ai-gemini'
import { grokText } from '@tanstack/ai-grok'
import { groqText } from '@tanstack/ai-groq'
import { openRouterText } from '@tanstack/ai-openrouter'
import { ollamaText } from '@tanstack/ai-ollama'
import { bedrockText } from '@tanstack/ai-bedrock'

// Model string is passed to the factory, NOT to chat()
const adapter = openaiText('gpt-5.2')
const adapter2 = anthropicText('claude-sonnet-4-6')
const adapter3 = geminiText('gemini-2.5-pro')
const adapter4 = grokText('grok-4')
const adapter5 = groqText('llama-3.3-70b-versatile')
const adapter6 = openRouterText('anthropic/claude-sonnet-4')
const adapter7 = ollamaText('llama3.3')
const adapter8 = bedrockText('us.anthropic.claude-3-7-sonnet-20250219-v1:0')

// Optional: pass explicit API key
const adapterWithKey = openaiText('gpt-5.2', {
  apiKey: 'sk-...',
})

@tanstack/ai-bedrock (Amazon Bedrock) branches on config.api:

• bedrockText(model) or bedrockText(model, { api: 'converse' }) (the default) — Bedrock's native Converse API via @aws-sdk/client-bedrock-runtime (adapter name bedrock-converse). Reaches the broad catalog: Claude, Nova, Llama, Mistral, DeepSeek, and more.
• bedrockText(model, { api: 'chat' }) — OpenAI-compatible Chat Completions endpoint (adapter name bedrock). Open-weight models only (gpt-oss, DeepSeek V3.x, Gemma, Qwen, etc.). Does NOT reach Claude, Nova, or Llama.
• bedrockText(model, { api: 'responses' }) — OpenAI-compatible Responses API, mantle-only (adapter name bedrock-responses). Currently gpt-oss family.

Use createBedrockText(model, apiKey, config?) to pass the key explicitly. Auth resolves from BEDROCK_API_KEY / AWS_BEARER_TOKEN_BEDROCK, or SigV4 via the standard AWS credential chain (no extra packages needed — handled by @aws-sdk/client-bedrock-runtime).

2. Runtime Adapter Switching

Use an adapter factory map to switch providers dynamically based on user input or configuration:

import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import type { TextAdapter } from '@tanstack/ai/adapters'
import { openaiText } from '@tanstack/ai-openai'
import { anthropicText } from '@tanstack/ai-anthropic'
import { geminiText } from '@tanstack/ai-gemini'

// Define a map of provider+model to adapter factory calls
const adapters: Record<string, () => TextAdapter> = {
  'openai/gpt-5.2': () => openaiText('gpt-5.2'),
  'anthropic/claude-sonnet-4-6': () => anthropicText('claude-sonnet-4-6'),
  'gemini/gemini-2.5-pro': () => geminiText('gemini-2.5-pro'),
}

export function handleChat(providerModel: string, messages: Array<any>) {
  const createAdapter = adapters[providerModel]
  if (!createAdapter) {
    throw new Error(`Unknown provider/model: ${providerModel}`)
  }

  const stream = chat({
    adapter: createAdapter(),
    messages,
  })

  return toServerSentEventsResponse(stream)
}

3. Configuring Reasoning / Thinking

Different providers expose reasoning/thinking through their modelOptions:

import { chat } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { anthropicText } from '@tanstack/ai-anthropic'
import { geminiText } from '@tanstack/ai-gemini'

// OpenAI: reasoning with effort and summary
const openaiStream = chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  modelOptions: {
    reasoning: {
      effort: 'high',
      summary: 'auto',
    },
  },
})

// Anthropic: extended thinking with budget_tokens
const anthropicStream = chat({
  adapter: anthropicText('claude-sonnet-4-6'),
  messages,
  modelOptions: {
    max_tokens: 16000,
    thinking: {
      type: 'enabled',
      budget_tokens: 8000, // must be >= 1024 and < max_tokens
    },
  },
})

// Anthropic: adaptive thinking (claude-sonnet-4-6 and newer)
const adaptiveStream = chat({
  adapter: anthropicText('claude-sonnet-4-6'),
  messages,
  modelOptions: {
    max_tokens: 16000,
    thinking: {
      type: 'adaptive',
    },
    effort: 'high', // 'max' | 'high' | 'medium' | 'low'
  },
})

// Gemini: thinking config with budget or level
const geminiStream = chat({
  adapter: geminiText('gemini-2.5-pro'),
  messages,
  modelOptions: {
    thinkingConfig: {
      includeThoughts: true,
      thinkingBudget: 4096,
    },
  },
})

4. Extending Adapters with Custom Models

Use extendAdapter() and createModel() to add custom or fine-tuned models while preserving type safety for the original models:

import { extendAdapter, createModel } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

// Define custom models
const customModels = [
  createModel('ft:gpt-5.2:my-org:custom-model:abc123', ['text', 'image']),
  createModel('my-local-proxy-model', ['text']),
] as const

// Create extended factory - original models still fully typed
const myOpenai = extendAdapter(openaiText, customModels)

// Use original models - full type inference preserved
const gpt5 = myOpenai('gpt-5.2')

// Use custom models - accepted by the type system
const custom = myOpenai('ft:gpt-5.2:my-org:custom-model:abc123')

// Type error: 'nonexistent-model' is not a valid model
// myOpenai('nonexistent-model')

At runtime, extendAdapter simply passes through to the original factory. The _customModels parameter is only used for type inference.

5. Configuring Sampling

Sampling controls (temperature, token limits, nucleus sampling) are passed inside modelOptions using each provider's native key. They are not top-level fields on chat()/ai()/generate().

// OpenAI — native keys
chat({
  adapter: openaiText('gpt-5.2'),
  messages,
  modelOptions: { temperature: 0.7, top_p: 0.9, max_output_tokens: 1000 },
})

// Anthropic
chat({
  adapter: anthropicText('claude-sonnet-4-6'),
  messages,
  modelOptions: { temperature: 0.7, top_p: 0.9, max_tokens: 1000 },
})

// Gemini — camelCase
chat({
  adapter: geminiText('gemini-2.5-pro'),
  messages,
  modelOptions: { temperature: 0.7, topP: 0.9, maxOutputTokens: 1000 },
})

// Ollama — NESTED under modelOptions.options
chat({
  adapter: ollamaText('llama3.3'),
  messages,
  modelOptions: {
    options: { temperature: 0.7, top_p: 0.9, num_predict: 1000 },
  },
})

Per-provider sampling keys (all live inside modelOptions):

temperature is the one key every provider names identically; token limits and some sampling options use provider-native names. Ollama nests all sampling under modelOptions.options.

Anthropic max_tokens default: Anthropic's API requires max_tokens, so the adapter always sends one. When you omit modelOptions.max_tokens, it defaults to the selected model's full output ceiling (its max_output_tokens from model metadata — e.g. 64K for Sonnet, 128K for Opus), not a low constant. max_tokens is a ceiling, not a reservation (billing is per token generated), so leaving it unset is the right default for codegen / agentic / long-form output and avoids silent stop_reason: "max_tokens" truncation. Set it only to cap output below the model ceiling. Other providers treat token limits as optional and don't apply this flooring.

6. Capability Flag: supportsCombinedToolsAndSchema

Adapters can declare an optional capability method:

supportsCombinedToolsAndSchema?(modelOptions?: TProviderOptions): boolean

When true, the engine wires outputSchema into the regular chatStream call alongside tools and harvests the schema-constrained JSON from the agent loop's final-turn text — skipping the separate structuredOutput / structuredOutputStream finalization round-trip. When false (or the method is omitted), the legacy finalization path runs.

Current per-adapter status (#605):

Subclasses can override to narrow the capability. When extending an adapter for a custom model that doesn't support the combination, return false explicitly.

6. OpenAI-Compatible Providers

Any provider that implements the OpenAI Chat Completions API (DeepSeek, Moonshot/Kimi, Together, Fireworks, Cerebras, Qwen/DashScope, Perplexity, NVIDIA NIM, LM Studio, etc.) can be used through the generic openaiCompatible factory from @tanstack/ai-openai/compatible — no dedicated package required.

import { openaiCompatible } from '@tanstack/ai-openai/compatible'
import { createModel } from '@tanstack/ai'

// Provider-factory: configure baseURL + apiKey + models ONCE,
// then select a model per call (the model arg is a type-safe union).
const deepseek = openaiCompatible({
  name: 'deepseek', // optional label for devtools/errors (default 'openai-compatible')
  baseURL: 'https://api.deepseek.com/v1',
  apiKey: process.env.DEEPSEEK_API_KEY!,
  models: [
    'deepseek-chat', // bare string → optimistic defaults: text/image in, streaming, tools, structured output
    createModel('deepseek-reasoner', {
      // rich def → precise per-model capabilities
      input: ['text'],
      features: ['reasoning', 'structured_outputs'],
    }),
  ],
})

chat({ adapter: deepseek('deepseek-chat'), messages })
chat({ adapter: deepseek('deepseek-reasoner'), messages })

config also accepts any OpenAI SDK ClientOptions (notably defaultHeaders and defaultQuery) for providers that need extra auth headers or query params.

For a single model, use the one-shot helper:

import { openaiCompatibleText } from '@tanstack/ai-openai/compatible'

chat({
  adapter: openaiCompatibleText('deepseek-chat', {
    baseURL: 'https://api.deepseek.com/v1',
    apiKey: process.env.DEEPSEEK_API_KEY!,
  }),
  messages,
})

Pass api: 'responses' to target the OpenAI Responses API instead of Chat Completions (only for the rare compatible provider that implements it, e.g. Azure OpenAI); the default is 'chat-completions', which is what nearly all compatible providers speak.

Verify the provider's current baseURL and model ids against its live docs — they drift. See docs/adapters/openai-compatible.md for the full provider table.

Common Mistakes

a. HIGH: Confusing legacy monolithic with tree-shakeable adapter

The legacy openai() (and anthropic(), etc.) monolithic adapters are deprecated. They take the model in chat(), not in the factory.

// WRONG: Legacy monolithic adapter pattern
import { openai } from '@tanstack/ai-openai'
chat({ adapter: openai(), model: 'gpt-5.2', messages })

// CORRECT: Tree-shakeable adapter, model in factory
import { openaiText } from '@tanstack/ai-openai'
chat({ adapter: openaiText('gpt-5.2'), messages })

Source: docs/migration/migration.md

b. MEDIUM: Wrong API key environment variable name

Each provider uses a specific env var name. Using the wrong one causes a runtime error:

Source: adapter source code (utils/client.ts in each adapter package).

References

Detailed per-adapter reference files:

• OpenAI Adapter
• Anthropic Adapter
• Gemini Adapter
• Ollama Adapter
• Grok Adapter
• Groq Adapter
• OpenRouter Adapter

Tension

HIGH Tension: Type safety vs. quick prototyping -- Per-model type safety requires specific model string literals. Quick prototyping wants dynamic selection with string variables. Agents optimizing for quick setup silently lose type safety. If model names come from user input or config files, use extendAdapter() to add custom names.

Cross-References

• See also: ai-core/chat-experience/SKILL.md -- Adapter choice affects chat setup
• See also: ai-core/structured-outputs/SKILL.md -- outputSchema handles provider differences transparently

Media Generation

Dependency note: This skill builds on ai-core. Read it first for critical rules.

All media activities (image, speech, transcription, video) follow the same server/client architecture: a generate*() function on the server, an SSE transport via toServerSentEventsResponse(), and a framework hook on the client.

Setup -- Image Generation End-to-End

Server (API route or TanStack Start server function)

// routes/api/generate/image.ts
import { generateImage, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiImage } from '@tanstack/ai-openai'

export async function POST(req: Request) {
  const { prompt, size, numberOfImages } = await req.json()

  const stream = generateImage({
    adapter: openaiImage('gpt-image-1'),
    prompt,
    size,
    numberOfImages,
    stream: true,
  })

  return toServerSentEventsResponse(stream)
}

Client (React)

import { useGenerateImage, fetchServerSentEvents } from '@tanstack/ai-react'
import { useState } from 'react'

function ImageGenerator() {
  const [prompt, setPrompt] = useState('')
  const { generate, result, isLoading, error, reset } = useGenerateImage({
    connection: fetchServerSentEvents('/api/generate/image'),
  })

  return (
    <div>
      <input
        value={prompt}
        onChange={(e) => setPrompt(e.target.value)}
        placeholder="Describe an image..."
      />
      <button
        onClick={() => generate({ prompt })}
        disabled={isLoading || !prompt.trim()}
      >
        {isLoading ? 'Generating...' : 'Generate'}
      </button>

      {error && <p>Error: {error.message}</p>}

      {result?.images.map((img, i) => (
        <img
          key={i}
          src={img.url || `data:image/png;base64,${img.b64Json}`}
          alt={img.revisedPrompt || 'Generated image'}
        />
      ))}

      {result && <button onClick={reset}>Clear</button>}
    </div>
  )
}

TanStack Start: Server Function Streaming (recommended)

When using TanStack Start, return toServerSentEventsResponse() from a server function. The client fetcher receives a Response and the hook parses it as SSE automatically:

// lib/server-functions.ts
import { createServerFn } from '@tanstack/react-start'
import { generateImage, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiImage } from '@tanstack/ai-openai'

export const generateImageStreamFn = createServerFn({ method: 'POST' })
  .inputValidator((data: { prompt: string; model?: string }) => data)
  .handler(({ data }) => {
    return toServerSentEventsResponse(
      generateImage({
        adapter: openaiImage(data.model ?? 'gpt-image-1'),
        prompt: data.prompt,
        stream: true,
      }),
    )
  })

import { useGenerateImage } from '@tanstack/ai-react'
import { generateImageStreamFn } from '../lib/server-functions'

function ImageGenerator() {
  const { generate, result, isLoading } = useGenerateImage({
    fetcher: (input) => generateImageStreamFn({ data: input }),
  })

  return (
    <button
      onClick={() => generate({ prompt: 'A sunset over mountains' })}
      disabled={isLoading}
    >
      {isLoading ? 'Generating...' : 'Generate'}
    </button>
  )
}

Core Patterns

1. Image Generation

Supported adapters: openaiImage (dall-e-2, dall-e-3, gpt-image-1, gpt-image-1-mini, gpt-image-2) and geminiImage (gemini-3.1-flash-image-preview, gemini-3.1-flash-lite-image, imagen-4.0-generate-001, etc.).

import { generateImage } from '@tanstack/ai'
import { openaiImage } from '@tanstack/ai-openai'
import { geminiImage } from '@tanstack/ai-gemini'

// OpenAI with quality/background options
const openaiResult = await generateImage({
  adapter: openaiImage('gpt-image-1'),
  prompt: 'A cat wearing a hat',
  size: '1024x1024',
  numberOfImages: 2,
  modelOptions: {
    quality: 'high',
    background: 'transparent',
    outputFormat: 'png',
  },
})

// Gemini native model with aspect-ratio sizes
const geminiResult = await generateImage({
  adapter: geminiImage('gemini-3.1-flash-image-preview'),
  prompt: 'A futuristic cityscape at night',
  size: '16:9_4K',
})

// Gemini Imagen model
const imagenResult = await generateImage({
  adapter: geminiImage('imagen-4.0-generate-001'),
  prompt: 'A landscape photo',
  modelOptions: { aspectRatio: '16:9' },
})

Result shape: ImageGenerationResult with images array where each entry has b64Json?, url?, and revisedPrompt?. OpenAI image URLs expire after 1 hour -- download or display immediately.

Image-conditioned generation: multimodal prompt parts

Both generateImage() and generateVideo() accept the prompt either as a plain string or as an ordered array of content parts (TextPart / ImagePart / VideoPart / AudioPart — the same shapes used elsewhere in TanStack AI). Part order is meaningful: natively multimodal providers (Gemini, OpenRouter) receive parts in order; named-field providers (OpenAI, fal, xAI) extract media parts and flatten the text. Prompt text is always sent verbatim — to reference inputs from the prompt, write the provider's own syntax (fal @Image1, OpenAI "image 1" prose); the SDK never injects or rewrites markers. Each media part may carry an optional metadata.role hint that adapters use to route the part to the provider-specific field. The accepted part types are narrowed per model at compile time via the adapter's input-modality map.

import { generateImage } from '@tanstack/ai'
import { openaiImage } from '@tanstack/ai-openai'

// Image-to-image (OpenAI gpt-image-2 / gpt-image-1, dall-e-2)
await generateImage({
  adapter: openaiImage('gpt-image-2'),
  prompt: [
    { type: 'text', content: 'Turn this into a cinematic product photo' },
    { type: 'image', source: { type: 'url', value: 'https://…/product.png' } },
  ],
})

// Multi-reference (up to 16 for gpt-image models; up to ~14 for Gemini native
// — a provider limit, not enforced by the SDK)
await generateImage({
  adapter: openaiImage('gpt-image-2'),
  prompt: [
    { type: 'text', content: 'Apply the second image as style to the first' },
    { type: 'image', source: { type: 'url', value: 'https://…/product.png' } },
    { type: 'image', source: { type: 'url', value: 'https://…/style.png' } },
  ],
})

// Inpaint via metadata.role === 'mask' (OpenAI gpt-image models, dall-e-2; fal mask_url)
await generateImage({
  adapter: openaiImage('gpt-image-2'),
  prompt: [
    { type: 'text', content: 'Replace the masked region with a tree' },
    { type: 'image', source: { type: 'url', value: photoUrl } },
    {
      type: 'image',
      source: { type: 'url', value: maskUrl },
      metadata: { role: 'mask' },
    },
  ],
})

// Image-to-video (OpenAI Sora: single input_reference; fal: image_url + optional end_image_url)
import { generateVideo } from '@tanstack/ai'
import { falVideo } from '@tanstack/ai-fal'

await generateVideo({
  adapter: falVideo('fal-ai/kling-video/v3/pro/image-to-video'),
  prompt: [
    { type: 'image', source: { type: 'url', value: firstFrameUrl } },
    { type: 'text', content: 'Slow cinematic push-in' },
    {
      type: 'image',
      source: { type: 'url', value: lastFrameUrl },
      metadata: { role: 'end_frame' },
    },
  ],
})

URL inputs that require an upload throw by default. Most adapters pass a type: 'url' source straight through to the provider. Three paths can't — OpenAI images.edit(), OpenAI Sora input_reference, and Gemini Veo — because the provider only accepts uploaded bytes (Veo also takes a gs:// reference). For those, an HTTP(S) URL would have to be downloaded and buffered in memory, which can OOM constrained runtimes, so they throw on an HTTP(S) URL image input by default. Pass a data: URI (or gs:// for Veo), or opt in with allowUrlFetch: true on the adapter config (createOpenaiImage(model, apiKey, { allowUrlFetch: true }), and likewise on createOpenaiVideo / createGeminiVideo). data: URIs never need the flag.

Role hints (metadata.role):

Provider support matrix:

Video and audio prompt parts follow the same metadata.role convention for video-to-video and lipsync flows on fal; other providers throw when they're passed.

2. Audio Generation (Music, Sound Effects)

Distinct from TTS — generateAudio() produces non-speech audio content. Supported adapters: geminiAudio (Lyria 3 Pro / Lyria 3 Clip) and falAudio (MiniMax Music, DiffRhythm, Stable Audio, ElevenLabs SFX, etc.).

import { generateAudio } from '@tanstack/ai'
import { falAudio } from '@tanstack/ai-fal'

const result = await generateAudio({
  adapter: falAudio('fal-ai/diffrhythm'),
  prompt: 'An upbeat electronic track with synths',
  duration: 10,
})

// result.audio.url or result.audio.b64Json (provider-dependent)
// result.audio.contentType e.g. "audio/mpeg"

Client hook:

import { useGenerateAudio, fetchServerSentEvents } from '@tanstack/ai-react'

const { generate, result, isLoading } = useGenerateAudio({
  connection: fetchServerSentEvents('/api/generate/audio'),
})

// Trigger: generate({ prompt: 'Upbeat synths', duration: 10 })
// Play:    <audio src={result.audio.url} controls />

3. Text-to-Speech

Adapter: openaiSpeech (tts-1, tts-1-hd, gpt-4o-audio-preview).

import { generateSpeech } from '@tanstack/ai'
import { openaiSpeech } from '@tanstack/ai-openai'

const result = await generateSpeech({
  adapter: openaiSpeech('tts-1-hd'),
  text: 'Hello, welcome to TanStack AI!',
  voice: 'alloy', // alloy | echo | fable | onyx | nova | shimmer | ash | ballad | coral | sage | verse
  format: 'mp3', // mp3 | opus | aac | flac | wav | pcm
  speed: 1.0, // 0.25 to 4.0
})

// result.audio is base64-encoded audio
// result.format is the output format string
// result.contentType is the MIME type (e.g. "audio/mpeg")

Client hook:

import { useGenerateSpeech, fetchServerSentEvents } from '@tanstack/ai-react'

const { generate, result, isLoading } = useGenerateSpeech({
  connection: fetchServerSentEvents('/api/generate/speech'),
})

// Trigger: generate({ text: 'Hello!', voice: 'alloy' })
// Play:   <audio src={`data:audio/${result.format};base64,${result.audio}`} controls />

4. Audio Transcription

Adapter: openaiTranscription (whisper-1, gpt-4o-transcribe, gpt-4o-mini-transcribe, gpt-4o-transcribe-diarize).

Capturing audio in the browser: Use useAudioRecorder from @tanstack/ai-react to record directly in the browser, then pass the recording as the audio input to generate(), or use recording.part as a prompt part in chat/generation calls. No transcoding or extra dependencies required — the recorder returns the native browser format (audio/webm or audio/mp4). For transcription, wrap it as a data: URL so the provider gets the real content type; passing raw recording.base64 makes the adapter assume audio/mpeg and mislabel the webm/mp4 bytes.

const { isRecording, start, stop } = useAudioRecorder()
const { generate } = useTranscription({
  connection: fetchServerSentEvents('/api/transcribe'),
})
// ...
const recording = await stop()
const mimeType = recording.mimeType.split(';')[0] // strip ;codecs=...
await generate({ audio: `data:${mimeType};base64,${recording.base64}` })

import { generateTranscription } from '@tanstack/ai'
import { openaiTranscription } from '@tanstack/ai-openai'

const result = await generateTranscription({
  adapter: openaiTranscription('whisper-1'),
  audio: audioFile, // File, Blob, base64 string, or data URL
  language: 'en',
  responseFormat: 'verbose_json',
  modelOptions: {
    timestamp_granularities: ['word', 'segment'],
  },
})

// result.text       -- full transcribed text
// result.language   -- detected/specified language
// result.duration   -- audio duration in seconds
// result.segments   -- timestamped segments (word-level timestamps are in result.words)

For speaker diarization, use openaiTranscription('gpt-4o-transcribe-diarize'). When no response format is given it defaults the request to response_format: 'diarized_json' and chunking_strategy: 'auto' (a top-level responseFormat of 'json'/'text' opts out of speaker segments); do not pass prompt, include, or timestamp_granularities with this model.

Client hook:

import { useTranscription, fetchServerSentEvents } from '@tanstack/ai-react'

const { generate, result, isLoading } = useTranscription({
  connection: fetchServerSentEvents('/api/transcribe'),
})

// Trigger: generate({ audio: dataUrl, language: 'en' })

5. Video Generation (Experimental -- async polling)

Video generation uses a jobs/polling architecture. The server creates a job, polls for status, and streams updates to the client.

import {
  generateVideo,
  getVideoJobStatus,
  toServerSentEventsResponse,
} from '@tanstack/ai'
import { openaiVideo } from '@tanstack/ai-openai'

// Non-streaming: manual polling loop
const { jobId } = await generateVideo({
  adapter: openaiVideo('sora-2'),
  prompt: 'A golden retriever playing in sunflowers',
  size: '1280x720',
  duration: 8,
})

let status = await getVideoJobStatus({ adapter: openaiVideo('sora-2'), jobId })
while (status.status !== 'completed' && status.status !== 'failed') {
  await new Promise((r) => setTimeout(r, 5000))
  status = await getVideoJobStatus({ adapter: openaiVideo('sora-2'), jobId })
}

// Streaming: server handles polling, client gets real-time updates
const stream = generateVideo({
  adapter: openaiVideo('sora-2'),
  prompt: 'A flying car over a city',
  stream: true,
  pollingInterval: 3000,
  maxDuration: 600_000,
})
return toServerSentEventsResponse(stream)

Google Veo (@tanstack/ai-gemini) uses the same jobs/polling flow. Its duration option is typed per model (4 | 6 | 8 for the Veo 3.1 models); use adapter.snapDuration(seconds) to coerce raw seconds and adapter.availableDurations() to enumerate the valid set. Image prompt parts route by metadata.role: first un-roled / 'start_frame' image → input image, 'end_frame' → lastFrame, 'reference' / 'character' → referenceImages:

import { geminiVideo } from '@tanstack/ai-gemini'

const adapter = geminiVideo('veo-3.1-generate-preview')
adapter.availableDurations() // { kind: 'discrete', values: [4, 6, 8] }

const { jobId } = await generateVideo({
  adapter,
  prompt: 'A golden retriever playing in sunflowers',
  size: '16:9', // Veo sizes are aspect ratios: '16:9' | '9:16'
  duration: adapter.snapDuration(7), // 6
  modelOptions: { resolution: '1080p', generateAudio: true },
})
// Note: Veo result URLs require the Google API key to download
// (x-goog-api-key header or ?key= query parameter).

Gemini Omni Flash (geminiVideo('gemini-omni-flash-preview')) is served by the Interactions API instead of Veo's operations flow — same adapter, routed by model. Clips are 720p; duration is any number of seconds in the 3–10 range (fractional ok, default 10 — availableDurations() reports the range), size is the aspect ratio ('16:9' | '9:16'), and the finished video arrives inline as a data:video/mp4;base64,… URL (no key needed to use it). Image/video prompt parts are sent as interaction content blocks, grouped as images, then videos, then text (no metadata.role routing); data sources go inline, url sources pass through as-is (never downloaded — use Gemini Files API URIs for remote media). For conversational editing, pass a prior generation's jobId as modelOptions.previous_interaction_id with a prompt describing the change:

import { geminiVideo } from '@tanstack/ai-gemini'

const omni = geminiVideo('gemini-omni-flash-preview')
const first = await generateVideo({
  adapter: omni,
  prompt: 'A violinist outdoors',
})
// …poll first.jobId to completion, then edit it:
const edited = await generateVideo({
  adapter: omni,
  prompt: 'Make the violin invisible',
  modelOptions: { previous_interaction_id: first.jobId },
})

Other video adapters: openaiVideo('sora-2') (pixel sizes like '1280x720', durations 4/8/12s, single input_reference image prompt part), grokVideo(...) (grok-imagine-video does text-to-video + image-to-video; grok-imagine-video-1.5 is image-to-video only — needs an image prompt part as the starting frame, text-only throws; aspect-ratio size template like '16:9_720p', integer durations 1-15s, reports usage.unitsBilled seconds and exact usage.cost), and falVideo(...) (hosted models, see cost tracking below).

Client hook with job tracking:

import { useGenerateVideo, fetchServerSentEvents } from '@tanstack/ai-react'

const { generate, result, jobId, videoStatus, isLoading } = useGenerateVideo({
  connection: fetchServerSentEvents('/api/generate/video'),
  onJobCreated: (id) => console.log('Job created:', id),
  onStatusUpdate: (status) =>
    console.log(`${status.status} (${status.progress}%)`),
})

// videoStatus: { jobId, status, progress?, url?, error?, usage? }
// result (on completion): { url }

6. Cost tracking (fal billable units)

fal bills media generation by usage-based units, not tokens. Every fal media adapter (falImage, falAudio, falSpeech, falTranscription, falVideo) surfaces the real billed quantity on the result as usage.unitsBilled, read from fal's x-fal-billable-units response header — no fetch interceptor needed. It rides on the canonical TokenUsage shape (token fields are 0 for media), mirroring how duration-billed transcription surfaces durationSeconds.

import { generateImage } from '@tanstack/ai'
import { falImage } from '@tanstack/ai-fal'

const result = await generateImage({
  adapter: falImage('fal-ai/flux/dev'),
  prompt: 'a serene mountain lake',
})

// usage.unitsBilled is the priced quantity. Multiply by the endpoint unit
// price (GET https://api.fal.ai/v1/models/pricing?endpoint_id=…) for exact cost.
if (result.usage?.unitsBilled != null) {
  const cost = result.usage.unitsBilled * unitPrice
}

For video, the units arrive with the completed result: getVideoJobStatus() returns usage and emits a video:usage devtools event when fal reports it.

Common Hook API

All generation hooks return the same shape:

Provide either connection (streaming SSE transport) or fetcher (direct async call / server function returning Response). Use onResult to transform what is stored:

const { result } = useGenerateSpeech({
  connection: fetchServerSentEvents('/api/generate/speech'),
  onResult: (raw) => ({
    audioUrl: `data:${raw.contentType};base64,${raw.audio}`,
    duration: raw.duration,
  }),
})
// result is typed as { audioUrl: string; duration?: number } | null

Common Mistakes

a. HIGH: Using the removed embedding() function

The embedding() function and openaiEmbed adapter were removed in v0.5.0. Agents trained on older code may still generate this pattern.

Wrong:

import { embedding } from '@tanstack/ai'
import { openaiEmbed } from '@tanstack/ai-openai'

const result = await embedding({
  adapter: openaiEmbed(),
  model: 'text-embedding-3-small',
  input: 'Hello, world!',
})

Correct -- use the provider SDK directly:

import OpenAI from 'openai'

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })

const result = await openai.embeddings.create({
  model: 'text-embedding-3-small',
  input: 'Hello, world!',
})

Source: docs/migration/migration.md. Note: Fixed in v0.5.0 but agents trained on older code may still generate this pattern.

b. HIGH: Forgetting toServerSentEventsResponse with TanStack Start server functions

When using TanStack Start server functions with stream: true, you MUST wrap the stream with toServerSentEventsResponse(). Returning the raw stream from a server function will not work.

Wrong:

export const generateImageStreamFn = createServerFn({ method: 'POST' }).handler(
  ({ data }) => {
    // BUG: returning raw stream -- client cannot parse this
    return generateImage({
      adapter: openaiImage('gpt-image-1'),
      prompt: data.prompt,
      stream: true,
    })
  },
)

Correct:

import { generateImage, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiImage } from '@tanstack/ai-openai'

export const generateImageStreamFn = createServerFn({ method: 'POST' }).handler(
  ({ data }) => {
    return toServerSentEventsResponse(
      generateImage({
        adapter: openaiImage('gpt-image-1'),
        prompt: data.prompt,
        stream: true,
      }),
    )
  },
)

Source: maintainer interview.

c. MEDIUM: Not downloading OpenAI image URLs before they expire

OpenAI image URLs expire after 1 hour. If you store the URL and display it later, the image will silently break. Always download or display the image immediately, or convert to base64 for persistence.

const result = await generateImage({
  adapter: openaiImage('dall-e-3'),
  prompt: 'A mountain landscape',
})

// GOOD: download immediately
for (const img of result.images) {
  if (img.url) {
    const response = await fetch(img.url)
    const blob = await response.blob()
    // Save blob to storage...
  }
}

// GOOD: use b64Json when available (no expiration)
// gpt-image-1 returns b64Json by default

Source: docs/media/image-generation.md.

d. MEDIUM: Using stream: true for activities that do not support streaming

Not all generation activities support streaming. Passing stream: true to an activity that does not support it may hang or produce unexpected results. Check the activity documentation before enabling streaming. All built-in activities (generateImage, generateAudio, generateSpeech, generateTranscription, generateVideo, summarize) support stream: true, but custom useGeneration setups may not.

Source: docs/media/generations.md.

e. HIGH: Passing responseMimeType or negativePrompt to Gemini Lyria

Gemini's GenerateContentConfig (used by Lyria 3 Pro / Lyria 3 Clip) does not support responseMimeType or negativePrompt. Lyria 3 Clip always returns 30-second audio/mp3; Lyria 3 Pro returns audio/mp3. These fields are not in GeminiAudioProviderOptions — don't reach for them via as any.

// WRONG — both fields are silently ignored or rejected by the SDK
generateAudio({
  adapter: geminiAudio('lyria-3-pro-preview'),
  prompt: 'ambient piano',
  modelOptions: {
    responseMimeType: 'audio/wav', // unsupported
    negativePrompt: 'vocals', // unsupported
  } as any,
})

// CORRECT — shape the prompt itself for what you want
generateAudio({
  adapter: geminiAudio('lyria-3-pro-preview'),
  prompt: 'ambient piano, no vocals',
})

Source: Gemini API GenerateContentConfig type; docs/media/audio-generation.md.

f. MEDIUM: Passing duration to Lyria expecting it to control length

Lyria 3 Clip is fixed at 30 seconds — the duration option is ignored on that model. Lyria 3 Pro accepts duration via natural-language in the prompt ("2-minute ambient track with a 30-second build"), not via the duration field. duration works for fal audio models (mapped to each model's native field like music_length_ms or seconds_total), but not for Lyria.

// For Lyria: put length guidance in the prompt
generateAudio({
  adapter: geminiAudio('lyria-3-pro-preview'),
  prompt: 'A 2-minute ambient piano piece with gentle strings',
  // duration: 120  // ← does nothing; rely on the prompt
})

// For fal: duration works and is translated per-model
generateAudio({
  adapter: falAudio('fal-ai/minimax-music/v2'),
  prompt: 'upbeat synth melody',
  duration: 60, // → music_length_ms: 60_000
})

Source: Google Lyria 3 docs; docs/media/audio-generation.md.

g. MEDIUM: Gemini TTS multi-speaker with 0 or 3+ speakers

multiSpeakerVoiceConfig.speakerVoiceConfigs is validated to be length 1 or 2. Passing an empty array or three+ entries throws at the adapter boundary (not at Gemini's API) with a clear error. Don't try to work around it with as any.

generateSpeech({
  adapter: geminiSpeech('gemini-2.5-pro-preview-tts'),
  text: '[Alice] Hi. [Bob] Hello!',
  modelOptions: {
    multiSpeakerVoiceConfig: {
      speakerVoiceConfigs: [
        {
          speaker: 'Alice',
          voiceConfig: { prebuiltVoiceConfig: { voiceName: 'Kore' } },
        },
        {
          speaker: 'Bob',
          voiceConfig: { prebuiltVoiceConfig: { voiceName: 'Puck' } },
        },
      ],
    },
  },
})

Source: Gemini TTS adapter validation; CodeRabbit review of PR #463.

h. HIGH: Passing image prompt parts to a model that doesn't support image-conditioned generation

Not every model accepts image-conditioned prompts. The prompt type is narrowed per model, so passing an image part to a text-only model (dall-e-3, Imagen, grok-2-image) is a compile-time error; adapters also throw a clear runtime error as a backstop, so users learn at call time rather than getting silently wrong output.

// WRONG — dall-e-3 has no edit/inputs API; image parts are a type error
generateImage({
  adapter: openaiImage('dall-e-3'),
  prompt: [
    { type: 'text', content: 'Edit this' },
    { type: 'image', source: { type: 'url', value: url } }, // ❌ type error
  ],
})

// WRONG — Imagen is text-to-image only; same compile-time rejection
generateImage({
  adapter: geminiImage('imagen-4.0-generate-001'),
  prompt: [
    { type: 'text', content: 'Edit this' },
    { type: 'image', source: { type: 'url', value: url } }, // ❌ type error
  ],
})

// CORRECT — use a model that supports image-conditioned generation
generateImage({
  adapter: openaiImage('gpt-image-2'), // edits up to 16 images
  prompt: [
    { type: 'text', content: 'Edit this' },
    { type: 'image', source: { type: 'url', value: url } },
  ],
})

generateImage({
  adapter: geminiImage('gemini-3.1-flash-image-preview'), // native multimodal
  prompt: [
    { type: 'text', content: 'Edit this' },
    { type: 'image', source: { type: 'url', value: url } },
  ],
})

Source: docs/media/image-generation.md, docs/media/video-generation.md.

i. LOW: Writing a logging middleware to see media chunks flow through

Every media activity — generateAudio, generateSpeech, generateTranscription, generateImage, generateVideo — accepts the same debug?: DebugOption option that chat() does. Reach for debug instead of wiring up logging middleware.

// When a speech generation sounds wrong or a transcription returns garbage
generateSpeech({
  adapter: openaiSpeech('tts-1'),
  text: 'Hello',
  debug: { provider: true, output: true }, // raw SDK chunks + yielded chunks
})

See the ai-core/debug-logging sub-skill for full details on categories and piping into a custom logger.

Source: docs/advanced/debug-logging.md.

Cross-References

• See also: ai-core/adapter-configuration/SKILL.md -- Each media activity requires a specific activity adapter (e.g., openaiImage for images, openaiSpeech for speech, openaiTranscription for transcription, openaiVideo for video). The adapter-configuration skill covers provider setup, API keys, and model selection.
• See also: ai-core/debug-logging/SKILL.md -- When a media request returns unexpected output or fails mid-stream, toggle debug: true on any generate*() call to see request metadata, raw provider chunks, and errors. Covers per-category toggling and piping into pino/winston.

Structured Outputs

Dependency note: This skill builds on ai-core. Read it first for critical rules. The useChat patterns below build on ai-core/chat-experience — read that for the base hook surface, then come back here for the structured-output specifics.

Setup

import { chat } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { z } from 'zod'

const person = await chat({
  adapter: openaiText('gpt-5.2'),
  messages: [{ role: 'user', content: 'John Doe, 30' }],
  outputSchema: z.object({
    name: z.string(),
    age: z.number(),
  }),
})

person.name // string — fully typed, no cast
person.age // number

When outputSchema is provided, chat() returns Promise<InferSchemaType<TSchema>> instead of AsyncIterable<StreamChunk>. The result is fully typed.

Adding stream: true switches the return to StructuredOutputStream<InferSchemaType<TSchema>> — incremental JSON deltas plus a terminal validated object. See Pattern 3 below for direct iteration, Pattern 4 for the useChat shape on the client, and Pattern 5 for multi-turn structured chats.

Decision: which pattern fits

Core Patterns

Pattern 1: Basic structured output with Zod

import { chat } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { z } from 'zod'

const PersonSchema = z.object({
  name: z.string().meta({ description: "The person's full name" }),
  age: z.number().meta({ description: "The person's age in years" }),
  email: z.string().email().meta({ description: 'Email address' }),
})

// chat() returns Promise<{ name: string; age: number; email: string }>
const person = await chat({
  adapter: openaiText('gpt-5.2'),
  messages: [
    {
      role: 'user',
      content:
        'Extract the person info: John Doe is 30 years old, email john@example.com',
    },
  ],
  outputSchema: PersonSchema,
})

console.log(person.name) // "John Doe"
console.log(person.age) // 30
console.log(person.email) // "john@example.com"

Pattern 2: Complex nested schemas

import { chat } from '@tanstack/ai'
import { anthropicText } from '@tanstack/ai-anthropic'
import { z } from 'zod'

const CompanySchema = z.object({
  name: z.string(),
  founded: z.number().meta({ description: 'Year the company was founded' }),
  headquarters: z.object({
    city: z.string(),
    country: z.string(),
    address: z.string().optional(),
  }),
  employees: z.array(
    z.object({
      name: z.string(),
      role: z.string(),
      department: z.string(),
    }),
  ),
  financials: z
    .object({
      revenue: z
        .number()
        .meta({ description: 'Annual revenue in millions USD' }),
      profitable: z.boolean(),
    })
    .optional(),
})

const company = await chat({
  adapter: anthropicText('claude-sonnet-4-5'),
  messages: [
    {
      role: 'user',
      content: 'Extract company info from this article: ...',
    },
  ],
  outputSchema: CompanySchema,
})

// Full type safety on nested properties
console.log(company.headquarters.city)
console.log(company.employees[0].role)
console.log(company.financials?.revenue)

Pattern 3: Direct stream iteration

Pass stream: true alongside outputSchema to get an async iterable of standard streaming chunks plus a terminal validated object. Use this when you're a single process end-to-end — Node script, CLI, test, or a server endpoint that responds with one JSON blob. For the in-browser progressive-UI case, jump to Pattern 4 instead.

import { chat } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { z } from 'zod'

const PersonSchema = z.object({
  name: z.string(),
  age: z.number(),
  email: z.string().email(),
})

const stream = chat({
  adapter: openaiText('gpt-5.2'),
  messages: [
    { role: 'user', content: 'Extract: John Doe is 30, john@example.com' },
  ],
  outputSchema: PersonSchema,
  stream: true,
})

for await (const chunk of stream) {
  if (chunk.type === 'CUSTOM' && chunk.name === 'structured-output.complete') {
    // Terminal event. `chunk.value.object` is fully validated and typed
    // against the schema you passed in — no helper or cast required.
    chunk.value.object.name // string
    chunk.value.object.age // number
    chunk.value.reasoning // string | undefined (thinking models only)
  }
}

The terminal event is a CUSTOM chunk: { type: 'CUSTOM', name: 'structured-output.complete', value: { object: T, raw: string, reasoning?: string } }. The return type of chat({ outputSchema, stream: true }) carries T through, so a plain discriminated narrow (chunk.type === 'CUSTOM' && chunk.name === 'structured-output.complete') is enough — no type guard helper.

Adapter coverage for streaming:

Native combined mode vs fallback is signaled by the adapter's optional supportsCombinedToolsAndSchema(modelOptions) method. When it returns true, the engine wires the JSON Schema into the regular chatStream call and harvests the final-turn text — middleware sees the run through beforeModel / modelStream as usual, and the 'structuredOutput' middleware phase does not fire. When it returns false (or is omitted), the engine takes the legacy finalization path: agent loop, then a separate structuredOutput / structuredOutputStream call with 'structuredOutput' phase tagging.

Consumer code is identical across providers — always read the final object off structured-output.complete.

Pattern 4: useChat with outputSchema (progressive UI)

Pass outputSchema to useChat and you get a partial field that fills in as JSON streams in, plus a final field that snaps to the validated object on the terminal event. No onChunk ceremony, no manual JSON accumulation, no parsePartialJSON calls.

Server (same as Pattern 3, just behind an SSE endpoint):

// app/api/extract-person/route.ts (or your framework's equivalent)
import { chat, toServerSentEventsResponse } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'
import { z } from 'zod'

const PersonSchema = z.object({
  name: z.string(),
  age: z.number(),
  email: z.string().email(),
})

export async function POST(request: Request) {
  const { messages } = await request.json()
  const stream = chat({
    adapter: openaiText('gpt-5.2'),
    messages,
    outputSchema: PersonSchema,
    stream: true,
  })
  return toServerSentEventsResponse(stream)
}

Client:

import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'
import { z } from 'zod'

const PersonSchema = z.object({
  name: z.string(),
  age: z.number(),
  email: z.string().email(),
})

function PersonExtractor() {
  const { sendMessage, isLoading, partial, final } = useChat({
    connection: fetchServerSentEvents('/api/extract-person'),
    outputSchema: PersonSchema,
  })

  return (
    <div>
      <button
        disabled={isLoading}
        onClick={() => sendMessage('Extract: John Doe, 30, john@example.com')}
      >
        Extract
      </button>
      {/* `partial` fills in field by field while streaming. */}
      <p>Name: {partial.name ?? '…'}</p>
      <p>Age: {partial.age ?? '…'}</p>
      <p>Email: {partial.email ?? '…'}</p>
      {final && <pre>Validated: {JSON.stringify(final, null, 2)}</pre>}
    </div>
  )
}

• partial is DeepPartial<z.infer<typeof PersonSchema>> — every property optional, every nested array element optional. Updated from TEXT_MESSAGE_CONTENT deltas.
• final is z.infer<typeof PersonSchema> | null — populated when structured-output.complete arrives.
• outputSchema is for client-side type inference only. Validation runs on the server against the schema you pass to chat({ outputSchema }) there.
• Same shape works for non-streaming adapters: the fallback path emits one whole-JSON TEXT_MESSAGE_CONTENT then the terminal event, so partial populates and final snaps in the same render tick — same consumer code as the native-streaming providers, just without an intermediate field-by-field reveal.

Pattern 5: Multi-turn structured chat

Every assistant turn produced by useChat({ outputSchema }) carries its own typed StructuredOutputPart on messages[i].parts. Old turns stay renderable; new turns produce new parts; history is preserved without manual state plumbing. This is what makes the recipe-builder shape ("now make it vegan") work.

import { useChat, fetchServerSentEvents } from '@tanstack/ai-react'
import type { StructuredOutputPart } from '@tanstack/ai-client'
import { z } from 'zod'

const RecipeSchema = z.object({
  title: z.string(),
  cuisine: z.string(),
  servings: z.number(),
  ingredients: z.array(z.object({ item: z.string(), amount: z.string() })),
  steps: z.array(z.string()),
})
type Recipe = z.infer<typeof RecipeSchema>
type RecipePart = StructuredOutputPart<Recipe>

function RecipeBuilder() {
  const { messages, sendMessage } = useChat({
    outputSchema: RecipeSchema,
    connection: fetchServerSentEvents('/api/recipes'),
  })

  return (
    <div>
      {messages.map((m) => {
        if (m.role === 'user') {
          const text = m.parts
            .filter((p) => p.type === 'text')
            .map((p) => p.content)
            .join('')
          return <UserBubble key={m.id} text={text} />
        }
        if (m.role === 'assistant') {
          // `data` is `Recipe` because the schema generic flows from
          // `useChat({ outputSchema })` through `messages` to the part.
          const part = m.parts.find(
            (p): p is RecipePart => p.type === 'structured-output',
          )
          if (!part) return null
          return <RecipeCard key={m.id} part={part} />
        }
        return null
      })}
      <button onClick={() => sendMessage('pasta for two')}>Cook</button>
      <button onClick={() => sendMessage('now make it vegan')}>Modify</button>
    </div>
  )
}

function RecipeCard({ part }: { part: RecipePart }) {
  // `data` lands on complete, `partial` fills in while streaming.
  // Both are typed against the schema. No casts.
  const recipe = part.data ?? part.partial ?? ({} as Partial<Recipe>)
  return <h3>{recipe.title ?? 'Plating up…'}</h3>
}

Key behaviors:

• Per-turn parts. Each sendMessage() produces a new assistant message with its own StructuredOutputPart. The previous turn's part is untouched — messages.map(...) renders the whole history.
• Typed by schema. messages[i].parts.find(p => p.type === 'structured-output').data is typed as Recipe (no cast, no unknown). Works because useChat<TSchema> threads InferSchemaType<TSchema> down through UIMessage<TTools, TData> → MessagePart<TTools, TData> → StructuredOutputPart<TData>. In @tanstack/ai core the message types are single-generic (UIMessage<TData>); the tools generic lives in @tanstack/ai-client and the framework hook packages — import from your framework package or ai-client, not from @tanstack/ai.
• partial / final are derived. The hook-level partial and final are NOT singleton state — they're derived from the latest assistant message's part (the one after the most recent user message). Between sendMessage() and the first chunk, partial reads {} and final reads null because no new assistant turn exists yet.
• Round-trip preserves history. When the client sends turn N+1, each prior assistant turn's structured-output part is serialized back as { role: 'assistant', content: <part.raw> } so the model sees its own prior structured response. Streaming / errored parts are dropped from the round-trip.

Common Mistakes

HIGH: Filtering TextParts out of useChat renderers when using outputSchema

Earlier versions of the library routed structured-output JSON deltas through TextPart, so renderers had to filter them out:

// OBSOLETE — this guard was needed only because JSON used to land in a TextPart
const last = messages.at(-1)
last?.parts.map((part) => {
  if (part.type === 'text') return null // ❌ hides the structured JSON
  // ...
})

That hack is gone. With outputSchema set, TEXT_MESSAGE_CONTENT deltas now route into a dedicated StructuredOutputPart (with raw, partial, data, status, optional errorMessage). Render the structured part directly; let real TextParts through.

// CORRECT — find the structured-output part directly; let actual TextParts render
last?.parts.map((part, i) => {
  if (part.type === 'thinking')
    return <ReasoningView key={i} text={part.content} />
  if (part.type === 'tool-call') return <ToolCallView key={i} part={part} />
  if (part.type === 'structured-output')
    return <RecipeCard key={i} part={part} />
  if (part.type === 'text') return <p key={i}>{part.content}</p> // ← real text, not JSON
  return null
})

If you still have an if (part.type === 'text') return null line in a structured-output renderer specifically for "hiding the JSON," delete it.

Source: PR #577 — structured-output became a typed UIMessage part.

HIGH: Treating partial / final as sticky state across turns

partial and final are derived from the latest assistant message's structured-output part, not a sticky hook-level slot. In a multi-turn chat:

• Between sendMessage() and the first chunk, partial reads {} and final reads null (no assistant message after the latest user yet).
• Once the latest turn completes, partial === final. Earlier turns' data is NOT in partial / final — it lives on the prior assistant messages' parts.

To render history, walk messages directly (see Pattern 5). Use partial / final for a sticky summary of the most recent turn only.

// WRONG — `final` only reflects the latest turn; earlier recipes vanish from this view
{final && <RecipeCard recipe={final} />}

// CORRECT for history — walk messages, render every assistant's structured-output part
{messages.map((m) =>
  m.role === 'assistant'
    ? m.parts.find((p) => p.type === 'structured-output')
      ? <RecipeCard key={m.id} part={...} />
      : null
    : null
)}

Source: PR #577 — partial/final derive from the latest assistant turn's part.

HIGH: Parsing streaming JSON deltas yourself

When iterating chat({ outputSchema, stream: true }) directly (Pattern 3), the TEXT_MESSAGE_CONTENT chunks contain partial JSON fragments — they are not valid JSON until the stream completes. Always read the validated object from the terminal structured-output.complete event. Validation runs once, on the complete payload.

// WRONG -- partial JSON, throws SyntaxError mid-stream, no schema validation
for await (const chunk of stream) {
  if (chunk.type === 'TEXT_MESSAGE_CONTENT') {
    const obj = JSON.parse(chunk.delta) // ❌ partial, invalid
  }
}

// CORRECT -- trust the terminal event
for await (const chunk of stream) {
  if (chunk.type === 'CUSTOM' && chunk.name === 'structured-output.complete') {
    const result = chunk.value.object // ✅ typed and validated
  }
}

If you need progressive parsed state in a non-React environment, use a partial-JSON parser on the accumulated raw string at render time — but do NOT treat the result as schema-validated; only the terminal event is. In useChat, this is already done for you (partial field on Pattern 4).

Source: maintainer interview

HIGH: Trying to implement provider-specific structured output strategies

The adapter already handles provider differences (OpenAI uses response_format, Anthropic uses tool-based extraction, Gemini uses responseSchema). Never configure this yourself.

// WRONG -- do not set provider-specific response format
chat({
  adapter,
  messages,
  modelOptions: {
    responseFormat: { type: 'json_schema', json_schema: mySchema },
  },
})

// CORRECT -- just pass outputSchema, the adapter handles the rest
chat({
  adapter,
  messages,
  outputSchema: z.object({ name: z.string(), age: z.number() }),
})

There is no scenario where you need to know the provider's strategy. Just pass outputSchema to chat().

Source: maintainer interview

HIGH: Passing raw objects instead of using the project's schema library

Agents often generate raw JSON Schema objects or plain TypeScript types instead of using the schema validation library already in the project (Zod, ArkType, Valibot). Always check what the project uses and match it.

// WRONG -- raw object, no runtime validation, no type inference
chat({
  adapter,
  messages,
  outputSchema: {
    type: 'object',
    properties: {
      name: { type: 'string' },
      age: { type: 'number' },
    },
    required: ['name', 'age'],
    additionalProperties: false,
  },
})

// CORRECT -- use the project's schema library (e.g. Zod)
import { z } from 'zod'

chat({
  adapter,
  messages,
  outputSchema: z.object({
    name: z.string(),
    age: z.number(),
  }),
})

Using the project's schema library gives you runtime validation, TypeScript type inference on the result, and correct JSON Schema conversion automatically. Check package.json for zod, arktype, or valibot and use whichever is already installed.

Source: maintainer interview

Middleware coverage

The final structured-output adapter call runs through the same middleware pipeline as the agent loop. onChunk observes chunks attributed to ctx.phase === 'structuredOutput'; onUsage fires for the final call's tokens; onFinish fires once at the end of the whole chat() invocation, after the structured-output result is available.

For schema-aware middleware (e.g., transforming the JSON Schema before the provider call, stripping system prompts), use the dedicated onStructuredOutputConfig hook. See middleware skill.

Cross-References

• See also: ai-core/chat-experience/SKILL.md — Base useChat surface; the structured-output additions documented here layer on top.
• See also: ai-core/adapter-configuration/SKILL.md — Adapter handles structured-output strategy transparently.
• See also: ai-core/tool-calling/SKILL.md — Combine tools with outputSchema for an agent loop that runs tools first and returns a typed object. Tool-approval and client-tool flows compose with structured runs without extra wiring; see docs/structured-outputs/with-tools.md.
• See also: ai-core/middleware/SKILL.md — onStructuredOutputConfig hook and the structuredOutput phase for observing/transforming the final structured-output call.