Building a Streaming Chatbot

The Chat class makes it effortless to create a conversational user interface for your chatbot application. It enables the streaming of chat messages from your AI provider, manages the chat state, and updates the UI automatically as new messages arrive.

To summarize, the Chat class provides the following features:

Message Streaming: All the messages from the AI provider are streamed to the chat UI in real-time.
Managed States: The class manages the states for input, messages, status, error and more for you.
Seamless Integration: Easily integrate your chat AI into any design or layout with minimal effort.

In this guide, you will learn how to use the Chat class to create a chatbot application with real-time message streaming. Check out our chatbot with tools guide to learn how to use tools in your chatbot.

Example

The request flow works like this:

The user submits a message and sendMessage posts it to your API route.
Your route calls the provider and returns a UI message stream.
The class appends chunks to the last message as they arrive, updating the UI as it goes.

<script lang="ts">
  import { Chat } from '@ai-sdk/svelte';

  const chat = new Chat();
</script>

{#each chat.messages as message (message.id)}
  <Message {message} />
{/each}
<ChatInput
  onsubmit={(text) => chat.sendMessage({ text })}
  disabled={chat.status !== 'ready'}
/>

import { openai } from '@ai-sdk/openai';
import { convertToModelMessages, streamText } from 'ai';

export async function POST({ request }) {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4o'),
    messages: await convertToModelMessages(messages),
  });

  return result.toUIMessageStreamResponse();
}

The UI messages have a new parts property that contains the message parts. We recommend rendering the messages using the parts property instead of the content property. The parts property supports different message types, including text, tool invocation, and tool result, and allows for more flexible and complex chat UIs.

In the Chat component, the Chat class will request to your AI provider endpoint whenever the user sends a message using sendMessage. The messages are then streamed back in real-time and displayed in the chat UI.

Customized UI

Chat also provides ways to manage the chat message states via code, show status, and update messages without being triggered by user interactions.

Status

The Chat class exposes a status. It has the following possible values:

submitted: The message has been sent to the API and we're awaiting the start of the response stream.
streaming: The response is actively streaming in from the API, receiving chunks of data.
ready: The full response has been received and processed; a new user message can be submitted.
error: An error occurred during the API request, preventing successful completion.

const chat = new Chat({
  transport: new DefaultChatTransport({ api: '/api/chat' }),
});

// ...

{#if chat.status === 'submitted' || chat.status === 'streaming'}
  <div>
    {#if chat.status === 'submitted'}<Spinner />{/if}
    <button type="button" onclick={() => chat.stop()}>
      Stop
    </button>
  </div>
{/if}

Error State

Similarly, the error state reflects the error object thrown during the fetch request. It can be used to display an error message, disable the submit button, or show a retry button:

We recommend showing a generic error message to the user, such as "Something went wrong." This is a good practice to avoid leaking information from the server.

const chat = new Chat({
  transport: new DefaultChatTransport({ api: '/api/chat' }),
});

// ...

{#if chat.error}
  <div>
    <p>An error occurred.</p>
    <button type="button" onclick={() => chat.regenerate()}>
      Retry
    </button>
  </div>
{/if}

Cancellation and regeneration

It's also a common use case to abort the response message while it's still streaming back from the AI provider. You can do this by calling the stop method on the Chat instance.

const chat = new Chat();

<button
  onclick={() => chat.stop()}
  disabled={!(chat.status === 'streaming' || chat.status === 'submitted')}
>
  Stop
</button>

API reference

Chat(options)

Creates a chat helper. All options are optional; the defaults talk to /api/chat and render at native stream speed.

Prop	Type	Description
`transport`	`ChatTransport<UIMessage>`	How messages reach your API route
`messages`	`UIMessage[]`	Initial messages to seed the conversation
`onFinish`	`(event: FinishEvent) => void`	Runs when the assistant response completes
`onError`	`(error: Error) => void`	Runs when the fetch request fails
`throttle`	`number`	Milliseconds between UI updates while streaming

Event Callbacks

Chat provides optional event callbacks that you can use to handle different stages of the chatbot lifecycle:

onFinish: Called when the assistant response is completed. The event includes the response message, all messages, and flags for abort, disconnect, and errors.
onError: Called when an error occurs during the fetch request.
onData: Called whenever a data part is received.

These callbacks can be used to trigger additional actions, such as logging, analytics, or custom UI updates.

const chat = new Chat({
  onFinish: ({ message }) => saveToHistory(message),
  onError: error => console.error(error),
});

Math

Display math sits in the flow rhythm and scrolls when it runs long. Inline math like $e^{i π} + 1 = 0$ rides the line without stretching it.

Display

The quadratic formula, as a block:

x = \frac{- b \pm \sqrt{b^{2} - 4 a c}}{2 a}

Prose continues after the block at the normal distance, so equations read as part of the argument, not decoration.

Overflow

A long expansion scrolls inside its own box instead of breaking the column:

{(a + b)}^{4} = a^{4} + 4 a^{3} b + 6 a^{2} b^{2} + 4 a b^{3} + b^{4}