branch:
README.md
11703 bytesRaw
# @cloudflare/ai-chat
AI chat agents with automatic message persistence, resumable streaming, and tool support. Built on Cloudflare Durable Objects and the [AI SDK](https://ai-sdk.dev).
## Install
```sh
npm install @cloudflare/ai-chat agents ai workers-ai-provider
```
## Quick Start
### Server
```typescript
import { AIChatAgent } from "@cloudflare/ai-chat";
import { createWorkersAI } from "workers-ai-provider";
import { streamText, convertToModelMessages } from "ai";
export class ChatAgent extends AIChatAgent {
async onChatMessage() {
const workersai = createWorkersAI({ binding: this.env.AI });
const result = streamText({
model: workersai("@cf/moonshotai/kimi-k2.5"),
messages: await convertToModelMessages(this.messages)
});
return result.toUIMessageStreamResponse();
}
}
```
That gives you: automatic message persistence in SQLite, resumable streaming on disconnect/reconnect, and real-time WebSocket delivery to all connected clients.
### Client
```tsx
import { useAgent } from "agents/react";
import { useAgentChat } from "@cloudflare/ai-chat/react";
function Chat() {
const agent = useAgent({ agent: "ChatAgent" });
const { messages, sendMessage, clearHistory, status } = useAgentChat({
agent
});
return (
<div>
{messages.map((msg) => (
<div key={msg.id}>
<strong>{msg.role}:</strong>
{msg.parts.map((part, i) =>
part.type === "text" ? <span key={i}>{part.text}</span> : null
)}
</div>
))}
<form
onSubmit={(e) => {
e.preventDefault();
const input = e.currentTarget.elements.namedItem(
"input"
) as HTMLInputElement;
sendMessage({
role: "user",
parts: [{ type: "text", text: input.value }]
});
input.value = "";
}}
>
<input name="input" placeholder="Type a message..." />
</form>
</div>
);
}
```
### Wrangler Config
```jsonc
// wrangler.jsonc
{
"ai": { "binding": "AI" },
"durable_objects": {
"bindings": [{ "name": "ChatAgent", "class_name": "ChatAgent" }]
},
"migrations": [
{
"tag": "v1",
"new_sqlite_classes": ["ChatAgent"]
}
]
}
```
## Tools
### Server-side tools
Tools with an `execute` function run on the server automatically:
```typescript
import { createWorkersAI } from "workers-ai-provider";
import { streamText, convertToModelMessages, tool } from "ai";
import { z } from "zod";
export class ChatAgent extends AIChatAgent {
async onChatMessage() {
const workersai = createWorkersAI({ binding: this.env.AI });
const result = streamText({
model: workersai("@cf/moonshotai/kimi-k2.5"),
messages: await convertToModelMessages(this.messages),
tools: {
getWeather: tool({
description: "Get weather for a city",
inputSchema: z.object({ city: z.string() }),
execute: async ({ city }) => {
const data = await fetchWeather(city);
return { temperature: data.temp, condition: data.condition };
}
})
},
maxSteps: 5
});
return result.toUIMessageStreamResponse();
}
}
```
### Client-side tools
Tools without `execute` are handled on the client via `onToolCall`. Use this for tools that need browser APIs (geolocation, clipboard, camera):
```typescript
// Server: define tool without execute
getLocation: tool({
description: "Get the user's location from their browser",
inputSchema: z.object({})
// No execute -- client handles it
});
```
```tsx
// Client: handle via onToolCall
const { messages, sendMessage } = useAgentChat({
agent,
onToolCall: async ({ toolCall, addToolOutput }) => {
if (toolCall.toolName === "getLocation") {
const pos = await new Promise((resolve, reject) =>
navigator.geolocation.getCurrentPosition(resolve, reject)
);
addToolOutput({
toolCallId: toolCall.toolCallId,
output: { lat: pos.coords.latitude, lng: pos.coords.longitude }
});
}
}
});
```
### Tool approval (human-in-the-loop)
Use `needsApproval` for tools that require user confirmation before executing:
```typescript
// Server
processPayment: tool({
description: "Process a payment",
inputSchema: z.object({ amount: z.number(), recipient: z.string() }),
needsApproval: async ({ amount }) => amount > 100, // Only require approval for large amounts
execute: async ({ amount, recipient }) => charge(amount, recipient)
});
```
```tsx
// Client
const { messages, addToolApprovalResponse } = useAgentChat({ agent });
// When rendering tool parts with state === "approval-requested":
<button onClick={() => addToolApprovalResponse({ id: approvalId, approved: true })}>
Approve
</button>
<button onClick={() => addToolApprovalResponse({ id: approvalId, approved: false })}>
Reject
</button>
```
## Resumable Streaming
Streams automatically resume on disconnect/reconnect. No configuration needed.
When a client disconnects mid-stream, chunks are buffered in SQLite. On reconnect, the client receives all buffered chunks and continues receiving the live stream.
Disable with `resume: false`:
```tsx
const { messages } = useAgentChat({ agent, resume: false });
```
## Storage Management
### Limiting stored messages
Cap the number of messages kept in SQLite:
```typescript
export class ChatAgent extends AIChatAgent {
maxPersistedMessages = 200; // Keep last 200 messages
async onChatMessage() {
// ...
}
}
```
Oldest messages are deleted when the count exceeds the limit. This controls storage only -- it does not affect what is sent to the LLM.
### Controlling LLM context
Use the AI SDK's `pruneMessages()` to control what is sent to the model, independently of what is stored:
```typescript
import { createWorkersAI } from "workers-ai-provider";
import { streamText, convertToModelMessages, pruneMessages } from "ai";
export class ChatAgent extends AIChatAgent {
maxPersistedMessages = 200;
async onChatMessage() {
const workersai = createWorkersAI({ binding: this.env.AI });
const result = streamText({
model: workersai("@cf/moonshotai/kimi-k2.5"),
messages: pruneMessages({
messages: await convertToModelMessages(this.messages),
reasoning: "before-last-message",
toolCalls: "before-last-2-messages"
})
});
return result.toUIMessageStreamResponse();
}
}
```
### Row size protection
Messages approaching SQLite's 2MB row limit are automatically compacted. Large tool outputs are replaced with an LLM-friendly summary that instructs the model to suggest re-running the tool. Compacted messages include `metadata.compactedToolOutputs` so clients can detect and display this gracefully.
## Custom Request Data
Include custom data with every chat request using the `body` option:
```tsx
const { messages, sendMessage } = useAgentChat({
agent,
body: {
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
userId: "abc"
}
});
// Or use a function for dynamic values:
body: () => ({ token: getAuthToken(), timestamp: Date.now() });
```
Access these fields on the server via `options.body`:
```typescript
async onChatMessage(onFinish, options) {
const { timezone, userId } = options?.body ?? {};
}
```
## API Reference
### `AIChatAgent<Env, State>`
Extends `Agent` from the `agents` package.
| Property / Method | Type | Description |
| ------------------------------------ | --------------------- | --------------------------------------------------------------------------- |
| `messages` | `UIMessage[]` | Current conversation messages (loaded from SQLite) |
| `maxPersistedMessages` | `number \| undefined` | Max messages to keep in SQLite. Default: unlimited |
| `onChatMessage(onFinish?, options?)` | Override | Handle incoming chat messages. Return a `Response`. `onFinish` is optional. |
| `persistMessages(messages)` | `Promise<void>` | Manually persist messages (usually automatic) |
| `saveMessages(messages)` | `Promise<void>` | Persist messages and trigger `onChatMessage` |
### `useAgentChat(options)`
React hook for chat interactions. Wraps the AI SDK's `useChat` with WebSocket transport.
**Options:**
| Option | Type | Description |
| ----------------------------- | --------------------------------------- | -------------------------------------------------------- |
| `agent` | `ReturnType<typeof useAgent>` | Agent connection (required) |
| `onToolCall` | `({ toolCall, addToolOutput }) => void` | Handle client-side tool execution |
| `autoContinueAfterToolResult` | `boolean` | Auto-continue after client tool results. Default: `true` |
| `resume` | `boolean` | Enable stream resumption. Default: `true` |
| `body` | `object \| () => object` | Custom data sent with every request (see below) |
| `prepareSendMessagesRequest` | `(options) => { body?, headers? }` | Advanced per-request customization |
| `getInitialMessages` | `(options) => Promise<UIMessage[]>` | Custom initial message loader |
**Returns:**
| Property | Type | Description |
| ------------------------- | ---------------------------------- | ------------------------------------------------------- |
| `messages` | `UIMessage[]` | Chat messages |
| `sendMessage` | `(message) => void` | Send a message |
| `clearHistory` | `() => void` | Clear conversation |
| `addToolOutput` | `({ toolCallId, output }) => void` | Provide tool output |
| `addToolApprovalResponse` | `({ id, approved }) => void` | Approve/reject a tool |
| `setMessages` | `(messages \| updater) => void` | Set messages (syncs to server) |
| `status` | `string` | `"idle"` \| `"submitted"` \| `"streaming"` \| `"error"` |
### Exports
| Import path | What it provides |
| --------------------------- | --------------------------------------------------- |
| `@cloudflare/ai-chat` | `AIChatAgent`, `createToolsFromClientSchemas` |
| `@cloudflare/ai-chat/react` | `useAgentChat` |
| `@cloudflare/ai-chat/types` | `MessageType`, `OutgoingMessage`, `IncomingMessage` |
## Examples
- [Resumable streaming chat](../../examples/resumable-stream-chat/) -- automatic stream resumption
- [Human-in-the-loop guide](../../guides/human-in-the-loop/) -- tool approval with `needsApproval` + `onToolCall`
- [Playground](../../examples/playground/) -- kitchen-sink demo of all SDK features
## License
MIT