Prompt file imported from GreenShadeZhang/ag-ui-sample (.github/prompts/ag-ui.prompt.md). Fill in {{errorMessage}} before use. Copyright stays with the author.

Agents

Source: https://docs.ag-ui.com/concepts/agents

Learn about agents in the Agent User Interaction Protocol

Agents

Agents are the core components in the AG-UI protocol that process requests and generate responses. They establish a standardized way for front-end applications to communicate with AI services through a consistent interface, regardless of the underlying implementation.

What is an Agent?

In AG-UI, an agent is a class that:

1. Manages conversation state and message history
2. Processes incoming messages and context
3. Generates responses through an event-driven streaming interface
4. Follows a standardized protocol for communication

Agents can be implemented to connect with any AI service, including:

• Large language models (LLMs) like GPT-4 or Claude
• Custom AI systems
• Retrieval augmented generation (RAG) systems
• Multi-agent systems

Agent Architecture

All agents in AG-UI extend the AbstractAgent class, which provides the foundation for:

• State management
• Message history tracking
• Event stream processing
• Tool usage

import { AbstractAgent } from "@ag-ui/client"

class MyAgent extends AbstractAgent {
  protected run(input: RunAgentInput): RunAgent {
    // Implementation details
  }
}

Core Components

AG-UI agents have several key components:

1. Configuration: Agent ID, thread ID, and initial state
2. Messages: Conversation history with user and assistant messages
3. State: Structured data that persists across interactions
4. Events: Standardized messages for communication with clients
5. Tools: Functions that agents can use to interact with external systems

Agent Types

AG-UI provides different agent implementations to suit various needs:

AbstractAgent

The base class that all agents extend. It handles core event processing, state management, and message history.

HttpAgent

A concrete implementation that connects to remote AI services via HTTP:

import { HttpAgent } from "@ag-ui/client"

const agent = new HttpAgent({
  url: "https://your-agent-endpoint.com/agent",
  headers: {
    Authorization: "Bearer your-api-key",
  },
})

Custom Agents

You can create custom agents to integrate with any AI service by extending AbstractAgent:

class CustomAgent extends AbstractAgent {
  // Custom properties and methods

  protected run(input: RunAgentInput): RunAgent {
    // Implement the agent's logic
  }
}

Implementing Agents

Basic Implementation

To create a custom agent, extend the AbstractAgent class and implement the required run method:

import {
  AbstractAgent,
  RunAgent,
  RunAgentInput,
  EventType,
  BaseEvent,
} from "@ag-ui/client"
import { Observable } from "rxjs"

class SimpleAgent extends AbstractAgent {
  protected run(input: RunAgentInput): RunAgent {
    const { threadId, runId } = input

    return () =>
      new Observable<BaseEvent>((observer) => {
        // Emit RUN_STARTED event
        observer.next({
          type: EventType.RUN_STARTED,
          threadId,
          runId,
        })

        // Send a message
        const messageId = Date.now().toString()

        // Message start
        observer.next({
          type: EventType.TEXT_MESSAGE_START,
          messageId,
          role: "assistant",
        })

        // Message content
        observer.next({
          type: EventType.TEXT_MESSAGE_CONTENT,
          messageId,
          delta: "Hello, world!",
        })

        // Message end
        observer.next({
          type: EventType.TEXT_MESSAGE_END,
          messageId,
        })

        // Emit RUN_FINISHED event
        observer.next({
          type: EventType.RUN_FINISHED,
          threadId,
          runId,
        })

        // Complete the observable
        observer.complete()
      })
  }
}

Agent Capabilities

Agents in the AG-UI protocol provide a rich set of capabilities that enable sophisticated AI interactions:

Interactive Communication

Agents establish bi-directional communication channels with front-end applications through event streams. This enables:

• Real-time streaming responses character-by-character
• Immediate feedback loops between user and AI
• Progress indicators for long-running operations
• Structured data exchange in both directions

Tool Usage

Agents can use tools to perform actions and access external resources. Importantly, tools are defined and passed in from the front-end application to the agent, allowing for a flexible and extensible system:

// Tool definition
const confirmAction = {
  name: "confirmAction",
  description: "Ask the user to confirm a specific action before proceeding",
  parameters: {
    type: "object",
    properties: {
      action: {
        type: "string",
        description: "The action that needs user confirmation",
      },
      importance: {
        type: "string",
        enum: ["low", "medium", "high", "critical"],
        description: "The importance level of the action",
      },
      details: {
        type: "string",
        description: "Additional details about the action",
      },
    },
    required: ["action"],
  },
}

// Running an agent with tools from the frontend
agent.runAgent({
  tools: [confirmAction], // Frontend-defined tools passed to the agent
  // other parameters
})

Tools are invoked through a sequence of events:

1. TOOL_CALL_START: Indicates the beginning of a tool call
2. TOOL_CALL_ARGS: Streams the arguments for the tool call
3. TOOL_CALL_END: Marks the completion of the tool call

Front-end applications can then execute the tool and provide results back to the agent. This bidirectional flow enables sophisticated human-in-the-loop workflows where:

• The agent can request specific actions be performed
• Humans can execute those actions with appropriate judgment
• Results are fed back to the agent for continued reasoning
• The agent maintains awareness of all decisions made in the process

This mechanism is particularly powerful for implementing interfaces where AI and humans collaborate. For example, CopilotKit leverages this exact pattern with their useCopilotAction hook, which provides a simplified way to define and handle tools in React applications.

By keeping the AI informed about human decisions through the tool mechanism, applications can maintain context and create more natural collaborative experiences between users and AI assistants.

State Management

Agents maintain a structured state that persists across interactions. This state can be:

• Updated incrementally through STATE_DELTA events
• Completely refreshed with STATE_SNAPSHOT events
• Accessed by both the agent and front-end
• Used to store user preferences, conversation context, or application state

// Accessing agent state
console.log(agent.state.preferences)

// State is automatically updated during agent runs
agent.runAgent().subscribe((event) => {
  if (event.type === EventType.STATE_DELTA) {
    // State has been updated
    console.log("New state:", agent.state)
  }
})

Multi-Agent Collaboration

AG-UI supports agent-to-agent handoff and collaboration:

• Agents can delegate tasks to other specialized agents
• Multiple agents can work together in a coordinated workflow
• State and context can be transferred between agents
• The front-end maintains a consistent experience across agent transitions

For example, a general assistant agent might hand off to a specialized coding agent when programming help is needed, passing along the conversation context and specific requirements.

Human-in-the-Loop Workflows

Agents support human intervention and assistance:

• Agents can request human input on specific decisions
• Front-ends can pause agent execution and resume it after human feedback
• Human experts can review and modify agent outputs before they're finalized
• Hybrid workflows combine AI efficiency with human judgment

This enables applications where the agent acts as a collaborative partner rather than an autonomous system.

Conversational Memory

Agents maintain a complete history of conversation messages:

• Past interactions inform future responses
• Message history is synchronized between client and server
• Messages can include rich content (text, structured data, references)
• The context window can be managed to focus on relevant information

// Accessing message history
console.log(agent.messages)

// Adding a new user message
agent.messages.push({
  id: "msg_123",
  role: "user",
  content: "Can you explain that in more detail?",
})

Metadata and Instrumentation

Agents can emit metadata about their internal processes:

• Reasoning steps through custom events
• Performance metrics and timing information
• Source citations and reference tracking
• Confidence scores for different response options

This allows front-ends to provide transparency into the agent's decision-making process and help users understand how conclusions were reached.

Using Agents

Once you've implemented or instantiated an agent, you can use it like this:

// Create an agent instance
const agent = new HttpAgent({
  url: "https://your-agent-endpoint.com/agent",
})

// Add initial messages if needed
agent.messages = [
  {
    id: "1",
    role: "user",
    content: "Hello, how can you help me today?",
  },
]

// Run the agent
agent
  .runAgent({
    runId: "run_123",
    tools: [], // Optional tools
    context: [], // Optional context
  })
  .subscribe({
    next: (event) => {
      // Handle different event types
      switch (event.type) {
        case EventType.TEXT_MESSAGE_CONTENT:
          console.log("Content:", event.delta)
          break
        // Handle other events
      }
    },
    error: (error) => console.error("Error:", error),
    complete: () => console.log("Run complete"),
  })

Agent Configuration

Agents accept configuration through the constructor:

interface AgentConfig {
  agentId?: string // Unique identifier for the agent
  description?: string // Human-readable description
  threadId?: string // Conversation thread identifier
  initialMessages?: Message[] // Initial messages
  initialState?: State // Initial state object
}

// Using the configuration
const agent = new HttpAgent({
  agentId: "my-agent-123",
  description: "A helpful assistant",
  threadId: "thread-456",
  initialMessages: [
    { id: "1", role: "system", content: "You are a helpful assistant." },
  ],
  initialState: { preferredLanguage: "English" },
})

Agent State Management

AG-UI agents maintain state across interactions:

// Access current state
console.log(agent.state)

// Access messages
console.log(agent.messages)

// Clone an agent with its state
const clonedAgent = agent.clone()

Conclusion

Agents are the foundation of the AG-UI protocol, providing a standardized way to connect front-end applications with AI services. By implementing the AbstractAgent class, you can create custom integrations with any AI service while maintaining a consistent interface for your applications.

The event-driven architecture enables real-time, streaming interactions that are essential for modern AI applications, and the standardized protocol ensures compatibility across different implementations.

Core architecture

Source: https://docs.ag-ui.com/concepts/architecture

Understand how AG-UI connects front-end applications to AI agents

Agent User Interaction Protocol (AG-UI) is built on a flexible, event-driven architecture that enables seamless, efficient communication between front-end applications and AI agents. This document covers the core architectural components and concepts.

Overview

AG-UI follows a client-server architecture that standardizes communication between agents and applications:

flowchart LR
    subgraph "Frontend"
        App["Application"]
        Client["AG-UI Client"]
    end

    subgraph "Backend"
        A1["AI Agent A"]
        P["Secure Proxy"]
        A2["AI Agent B"]
        A3["AI Agent C"]
    end

    App <--> Client
    Client <-->|"AG-UI Protocol"| A1
    Client <-->|"AG-UI Protocol"| P
    P <-->|"AG-UI Protocol"| A2
    P <-->|"AG-UI Protocol"| A3

    class P mintStyle;
    classDef mintStyle fill:#E0F7E9,stroke:#66BB6A,stroke-width:2px,color:#000000;

    style App rx:5, ry:5;
    style Client rx:5, ry:5;
    style A1 rx:5, ry:5;
    style P rx:5, ry:5;
    style A2 rx:5, ry:5;
    style A3 rx:5, ry:5;

• Application: User-facing apps (i.e. chat or any AI-enabled application).
• AG-UI Client: Generic communication clients like HttpAgent or specialized clients for connecting to existing protocols.
• Agents: Backend AI agents that process requests and generate streaming responses.
• Secure Proxy: Backend services that provide additional capabilities and act as a secure proxy.

Core components

Protocol layer

AG-UI's protocol layer provides a flexible foundation for agent communication.

• Universal compatibility: Connect to any protocol by implementing run(input: RunAgentInput) -> Observable<BaseEvent>

The protocol's primary abstraction enables applications to run agents and receive a stream of events:

{/* prettier-ignore */}

// Core agent execution interface
type RunAgent = () => Observable<BaseEvent>

class MyAgent extends AbstractAgent {
  run(input: RunAgentInput): RunAgent {
    const { threadId, runId } = input
    return () =>
      from([
        { type: EventType.RUN_STARTED, threadId, runId },
        {
          type: EventType.MESSAGES_SNAPSHOT,
          messages: [
            { id: "msg_1", role: "assistant", content: "Hello, world!" }
          ],
        },
        { type: EventType.RUN_FINISHED, threadId, runId },
      ])
  }
}

Standard HTTP client

AG-UI offers a standard HTTP client HttpAgent that can be used to connect to any endpoint that accepts POST requests with a body of type RunAgentInput and sends a stream of BaseEvent objects.

HttpAgent supports the following transports:

• HTTP SSE (Server-Sent Events)

  • Text-based streaming for wide compatibility
  • Easy to read and debug

• HTTP binary protocol

  • Highly performant and space-efficient custom transport
  • Robust binary serialization for production environments

Message types

AG-UI defines several event categories for different aspects of agent communication:

• Lifecycle events

  • RUN_STARTED, RUN_FINISHED, RUN_ERROR
  • STEP_STARTED, STEP_FINISHED

• Text message events

  • TEXT_MESSAGE_START, TEXT_MESSAGE_CONTENT, TEXT_MESSAGE_END

• Tool call events

  • TOOL_CALL_START, TOOL_CALL_ARGS, TOOL_CALL_END

• State management events

  • STATE_SNAPSHOT, STATE_DELTA, MESSAGES_SNAPSHOT

• Special events

  • RAW, CUSTOM

Running Agents

To run an agent, you create a client instance and execute it:

// Create an HTTP agent client
const agent = new HttpAgent({
  url: "https://your-agent-endpoint.com/agent",
  agentId: "unique-agent-id",
  threadId: "conversation-thread"
});

// Start the agent and handle events
agent.runAgent({
  tools: [...],
  context: [...]
}).subscribe({
  next: (event) => {
    // Handle different event types
    switch(event.type) {
      case EventType.TEXT_MESSAGE_CONTENT:
        // Update UI with new content
        break;
      // Handle other event types
    }
  },
  error: (error) => console.error("Agent error:", error),
  complete: () => console.log("Agent run complete")
});

State Management

AG-UI provides efficient state management through specialized events:

• STATE_SNAPSHOT: Complete state representation at a point in time
• STATE_DELTA: Incremental state changes using JSON Patch format (RFC 6902)
• MESSAGES_SNAPSHOT: Complete conversation history

These events enable efficient client-side state management with minimal data transfer.

Tools and Handoff

AG-UI supports agent-to-agent handoff and tool usage through standardized events:

• Tool definitions are passed in the runAgent parameters
• Tool calls are streamed as sequences of TOOL_CALL_START → TOOL_CALL_ARGS → TOOL_CALL_END events
• Agents can hand off to other agents, maintaining context continuity

Events

All communication in AG-UI is based on typed events. Every event inherits from BaseEvent:

interface BaseEvent {
  type: EventType
  timestamp?: number
  rawEvent?: any
}

Events are strictly typed and validated, ensuring reliable communication between components.

Events

Source: https://docs.ag-ui.com/concepts/events

Understanding events in the Agent User Interaction Protocol

Events

The Agent User Interaction Protocol uses a streaming event-based architecture. Events are the fundamental units of communication between agents and frontends, enabling real-time, structured interaction.

Event Types Overview

Events in the protocol are categorized by their purpose:

Base Event Properties

All events share a common set of base properties:

Lifecycle Events

These events represent the lifecycle of an agent run. A typical agent run follows a predictable pattern: it begins with a RunStarted event, may contain multiple optional StepStarted/StepFinished pairs, and concludes with either a RunFinished event (success) or a RunError event (failure).

Lifecycle events provide crucial structure to agent runs, enabling frontends to track progress, manage UI states appropriately, and handle errors gracefully. They create a consistent framework for understanding when operations begin and end, making it possible to implement features like loading indicators, progress tracking, and error recovery mechanisms.

sequenceDiagram
    participant Agent
    participant Client

    Note over Agent,Client: Run begins
    Agent->>Client: RunStarted

    opt Sending steps is optional
        Note over Agent,Client: Step execution
        Agent->>Client: StepStarted
        Agent->>Client: StepFinished
    end

    Note over Agent,Client: Run completes
    alt
        Agent->>Client: RunFinished
    else
        Agent->>Client: RunError
    end

The RunStarted and either RunFinished or RunError events are mandatory, forming the boundaries of an agent run. Step events are optional and may occur multiple times within a run, allowing for structured, observable progress tracking.

RunStarted

Signals the start of an agent run.

The RunStarted event is the first event emitted when an agent begins processing a request. It establishes a new execution context identified by a unique runId. This event serves as a marker for frontends to initialize UI elements such as progress indicators or loading states. It also provides crucial identifiers that can be used to associate subsequent events with this specific run.

RunFinished

Signals the successful completion of an agent run.

The RunFinished event indicates that an agent has successfully completed all its work for the current run. Upon receiving this event, frontends should finalize any UI states that were waiting on the agent's completion. This event marks a clean termination point and indicates that no further processing will occur in this run unless explicitly requested.

RunError

Signals an error during an agent run.

The RunError event indicates that the agent encountered an error it could not recover from, causing the run to terminate prematurely. This event provides information about what went wrong, allowing frontends to display appropriate error messages and potentially offer recovery options. After a RunError event, no further processing will occur in this run.

StepStarted

Signals the start of a step within an agent run.

The StepStarted event indicates that the agent is beginning a specific subtask or phase of its processing. Steps provide granular visibility into the agent's progress, enabling more precise tracking and feedback in the UI. Steps are optional but highly recommended for complex operations that benefit from being broken down into observable stages. The stepName could be the name of a node or function that is currently executing.

StepFinished

Signals the completion of a step within an agent run.

The StepFinished event indicates that the agent has completed a specific subtask or phase. When paired with a corresponding StepStarted event, it creates a bounded context for a discrete unit of work. Frontends can use these events to update progress indicators, show completion animations, or reveal results specific to that step. The stepName must match the corresponding StepStarted event to properly pair the beginning and end of the step.

Text Message Events

These events represent the lifecycle of text messages in a conversation. Text message events follow a streaming pattern, where content is delivered incrementally. A message begins with a TextMessageStart event, followed by one or more TextMessageContent events that deliver chunks of text as they become available, and concludes with a TextMessageEnd event.

This streaming approach enables real-time display of message content as it's generated, creating a more responsive user experience compared to waiting for the entire message to be complete before showing anything.

sequenceDiagram
    participant Agent
    participant Client

    Note over Agent,Client: Message begins
    Agent->>Client: TextMessageStart

    loop Content streaming
        Agent->>Client: TextMessageContent
    end

    Note over Agent,Client: Message completes
    Agent->>Client: TextMessageEnd

The TextMessageContent events each contain a delta field with a chunk of text. Frontends should concatenate these deltas in the order received to construct the complete message. The messageId property links all related events, allowing the frontend to associate content chunks with the correct message.

TextMessageStart

Signals the start of a text message.

The TextMessageStart event initializes a new text message in the conversation. It establishes a unique messageId that will be referenced by subsequent content chunks and the end event. This event allows frontends to prepare the UI for an incoming message, such as creating a new message bubble with a loading indicator. The role property identifies whether the message is coming from the assistant or potentially another participant in the conversation.

TextMessageContent

Represents a chunk of content in a streaming text message.

The TextMessageContent event delivers incremental parts of the message text as they become available. Each event contains a small chunk of text in the delta property that should be appended to previously received chunks. The streaming nature of these events enables real-time display of content, creating a more responsive and engaging user experience. Implementations should handle these events efficiently to ensure smooth text rendering without visible delays or flickering.

TextMessageEnd

Signals the end of a text message.

The TextMessageEnd event marks the completion of a streaming text message. After receiving this event, the frontend knows that the message is complete and no further content will be added. This allows the UI to finalize rendering, remove any loading indicators, and potentially trigger actions that should occur after message completion, such as enabling reply controls or performing automatic scrolling to ensure the full message is visible.

Tool Call Events

These events represent the lifecycle of tool calls made by agents. Tool calls follow a streaming pattern similar to text messages. When an agent needs to use a tool, it emits a ToolCallStart event, followed by one or more ToolCallArgs events that stream the arguments being passed to the tool, and concludes with a ToolCallEnd event.

This streaming approach allows frontends to show tool executions in real-time, making the agent's actions transparent and providing immediate feedback about what tools are being invoked and with what parameters.

sequenceDiagram
    participant Agent
    participant Client

    Note over Agent,Client: Tool call begins
    Agent->>Client: ToolCallStart

    loop Arguments streaming
        Agent->>Client: ToolCallArgs
    end

    Note over Agent,Client: Tool call completes
    Agent->>Client: ToolCallEnd

    Note over Agent,Client: Tool execution result
    Agent->>Client: ToolCallResult

The ToolCallArgs events each contain a delta field with a chunk of the arguments. Frontends should concatenate these deltas in the order received to construct the complete arguments object. The toolCallId property links all related events, allowing the frontend to associate argument chunks with the correct tool call.

ToolCallStart

Signals the start of a tool call.

The ToolCallStart event indicates that the agent is invoking a tool to perform a specific function. This event provides the name of the tool being called and establishes a unique toolCallId that will be referenced by subsequent events in this tool call. Frontends can use this event to display tool usage to users, such as showing a notification that a specific operation is in progress. The optional parentMessageId allows linking the tool call to a specific message in the conversation, providing context for why the tool is being used.

ToolCallArgs

Represents a chunk of argument data for a tool call.

The ToolCallArgs event delivers incremental parts of the tool's arguments as they become available. Each event contains a segment of the argument data in the delta property. These deltas are often JSON fragments that, when combined, form the complete arguments object for the tool. Streaming the arguments is particularly valuable for complex tool calls where constructing the full arguments may take time. Frontends can progressively reveal these arguments to users, providing insight into exactly what parameters are being passed to tools.

ToolCallEnd

Signals the end of a tool call.

The ToolCallEnd event marks the completion of a tool call. After receiving this event, the frontend knows that all arguments have been transmitted and the tool execution is underway or completed. This allows the UI to finalize the tool call display and prepare for potential results. In systems where tool execution results are returned separately, this event indicates that the agent has finished specifying the tool and its arguments, and is now waiting for or has received the results.

ToolCallResult

Provides the result of a tool call execution.

The ToolCallResult event delivers the output or result from a tool that was previously invoked by the agent. This event is sent after the tool has been executed by the system and contains the actual output generated by the tool. Unlike the streaming pattern of tool call specification (start, args, end), the result is delivered as a complete unit since tool execution typically produces a complete output. Frontends can use this event to display tool results to users, append them to the conversation history, or trigger follow-up actions based on the tool's output.

State Management Events

These events are used to manage and synchronize the agent's state with the frontend. State management in the protocol follows an efficient snapshot-delta pattern where complete state snapshots are sent initially or infrequently, while incremental updates (deltas) are used for ongoing changes.

This approach optimizes for both completeness and efficiency: snapshots ensure the frontend has the full state context, while deltas minimize data transfer for frequent updates. Together, they enable frontends to maintain an accurate representation of agent state without unnecessary data transmission.

sequenceDiagram
    participant Agent
    participant Client

    Note over Agent,Client: Initial state transfer
    Agent->>Client: StateSnapshot

    Note over Agent,Client: Incremental updates
    loop State changes over time
        Agent->>Client: StateDelta
        Agent->>Client: StateDelta
    end

    Note over Agent,Client: Occasional full refresh
    Agent->>Client: StateSnapshot

    loop More incremental updates
        Agent->>Client: StateDelta
    end

    Note over Agent,Client: Message history update
    Agent->>Client: MessagesSnapshot

The combination of snapshots and deltas allows frontends to efficiently track changes to agent state while ensuring consistency. Snapshots serve as synchronization points that reset the state to a known baseline, while deltas provide lightweight updates between snapshots.

StateSnapshot

Provides a complete snapshot of an agent's state.

The StateSnapshot event delivers a comprehensive representation of the agent's current state. This event is typically sent at the beginning of an interaction or when synchronization is needed. It contains all state variables relevant to the frontend, allowing it to completely rebuild its internal representation. Frontends should replace their existing state model with the contents of this snapshot rather than trying to merge it with previous state.

StateDelta

Provides a partial update to an agent's state using JSON Patch.

The StateDelta event contains incremental updates to the agent's state in the form of JSON Patch operations (as defined in RFC 6902). Each delta represents specific changes to apply to the current state model. This approach is bandwidth-efficient, sending only what has changed rather than the entire state. Frontends should apply these patches in sequence to maintain an accurate state representation. If a frontend detects inconsistencies after applying patches, it may request a fresh StateSnapshot.

MessagesSnapshot

Provides a snapshot of all messages in a conversation.

The MessagesSnapshot event delivers a complete history of messages in the current conversation. Unlike the general state snapshot, this focuses specifically on the conversation transcript. This event is useful for initializing the chat history, synchronizing after connection interruptions, or providing a comprehensive view when a user joins an ongoing conversation. Frontends should use this to establish or refresh the conversational context displayed to users.

Special Events

Special events provide flexibility in the protocol by allowing for system-specific functionality and integration with external systems. These events don't follow the standard lifecycle or streaming patterns of other event types but instead serve specialized purposes.

Raw

Used to pass through events from external systems.

The Raw event acts as a container for events originating from external systems or sources that don't natively follow the Agent UI Protocol. This event type enables interoperability with other event-based systems by wrapping their events in a standardized format. The enclosed event data is preserved in its original form inside the event property, while the optional source property identifies the system it came from. Frontends can use this information to handle external events appropriately, either by processing them directly or by delegating them to system-specific handlers.

Custom

Used for application-specific custom events.

The Custom event provides an extension mechanism for implementing features not covered by the standard event types. Unlike Raw events which act as passthrough containers, Custom events are explicitly part of the protocol but with application-defined semantics. The name property identifies the specific custom event type, while the value property contains the associated data. This mechanism allows for protocol extensions without requiring formal specification changes. Teams should document their custom events to ensure consistent implementation across frontends and agents.

Event Flow Patterns

Events in the protocol typically follow specific patterns:

1. Start-Content-End Pattern: Used for streaming content (text messages, tool calls)

   • Start event initiates the stream
   • Content events deliver data chunks
   • End event signals completion

2. Snapshot-Delta Pattern: Used for state synchronization

   • Snapshot provides complete state
   • Delta events provide incremental updates

3. Lifecycle Pattern: Used for monitoring agent runs

   • Started events signal beginnings
   • Finished/Error events signal endings

Implementation Considerations

When implementing event handlers:

• Events should be processed in the order they are received
• Events with the same ID (e.g., messageId, toolCallId) belong to the same logical stream
• Implementations should be resilient to out-of-order delivery
• Custom events should follow the established patterns for consistency

Messages

Source: https://docs.ag-ui.com/concepts/messages

Understanding message structure and communication in AG-UI

Messages

Messages form the backbone of communication in the AG-UI protocol. They represent the conversation history between users and AI agents, and provide a standardized way to exchange information regardless of the underlying AI service being used.

Message Structure

AG-UI messages follow a vendor-neutral format, ensuring compatibility across different AI providers while maintaining a consistent structure. This allows applications to switch between AI services (like OpenAI, Anthropic, or custom models) without changing the client-side implementation.

The basic message structure includes:

interface BaseMessage {
  id: string // Unique identifier for the message
  role: string // The role of the sender (user, assistant, system, tool)
  content?: string // Optional text content of the message
  name?: string // Optional name of the sender
}

Message Types

AG-UI supports several message types to accommodate different participants in a conversation:

User Messages

Messages from the end user to the agent:

interface UserMessage {
  id: string
  role: "user"
  content: string // Text input from the user
  name?: string // Optional user identifier
}

Assistant Messages

Messages from the AI assistant to the user:

interface AssistantMessage {
  id: string
  role: "assistant"
  content?: string // Text response from the assistant (optional if using tool calls)
  name?: string // Optional assistant identifier
  toolCalls?: ToolCall[] // Optional tool calls made by the assistant
}

System Messages

Instructions or context provided to the agent:

interface SystemMessage {
  id: string
  role: "system"
  content: string // Instructions or context for the agent
  name?: string // Optional identifier
}

Tool Messages

Results from tool executions:

interface ToolMessage {
  id: string
  role: "tool"
  content: string // Result from the tool execution
  toolCallId: string // ID of the tool call this message responds to
}

Developer Messages

Internal messages used for development or debugging:

interface DeveloperMessage {
  id: string
  role: "developer"
  content: string
  name?: string
}

Vendor Neutrality

AG-UI messages are designed to be vendor-neutral, meaning they can be easily mapped to and from proprietary formats used by various AI providers:

// Example: Converting AG-UI messages to OpenAI format
const openaiMessages = agUiMessages
  .filter((msg) => ["user", "system", "assistant"].includes(msg.role))
  .map((msg) => ({
    role: msg.role as "user" | "system" | "assistant",
    content: msg.content || "",
    // Map tool calls if present
    ...(msg.role === "assistant" && msg.toolCalls
      ? {
          tool_calls: msg.toolCalls.map((tc) => ({
            id: tc.id,
            type: tc.type,
            function: {
              name: tc.function.name,
              arguments: tc.function.arguments,
            },
          })),
        }
      : {}),
  }))

This abstraction allows AG-UI to serve as a common interface regardless of the underlying AI service.

Message Synchronization

Messages can be synchronized between client and server through two primary mechanisms:

Complete Snapshots

The MESSAGES_SNAPSHOT event provides a complete view of all messages in a conversation:

interface MessagesSnapshotEvent {
  type: EventType.MESSAGES_SNAPSHOT
  messages: Message[] // Complete array of all messages
}

This is typically used:

• When initializing a conversation
• After connection interruptions
• When major state changes occur
• To ensure client-server synchronization

Streaming Messages

For real-time interactions, new messages can be streamed as they're generated:

1. Start a message: Indicate a new message is being created

   interface TextMessageStartEvent {
     type: EventType.TEXT_MESSAGE_START
     messageId: string
     role: string
   }

2. Stream content: Send content chunks as they become available

   interface TextMessageContentEvent {
     type: EventType.TEXT_MESSAGE_CONTENT
     messageId: string
     delta: string // Text chunk to append
   }

3. End a message: Signal the message is complete

   interface TextMessageEndEvent {
     type: EventType.TEXT_MESSAGE_END
     messageId: string
   }

This streaming approach provides a responsive user experience with immediate feedback.

Tool Integration in Messages

AG-UI messages elegantly integrate tool usage, allowing agents to perform actions and process their results:

Tool Calls

Tool calls are embedded within assistant messages:

interface ToolCall {
  id: string // Unique ID for this tool call
  type: "function" // Type of tool call
  function: {
    name: string // Name of the function to call
    arguments: string // JSON-encoded string of arguments
  }
}

Example assistant message with tool calls:

{
  id: "msg_123",
  role: "assistant",
  content: "I'll help you with that calculation.",
  toolCalls: [
    {
      id: "call_456",
      type: "function",
      function: {
        name: "calculate",
        arguments: '{"expression": "24 * 7"}'
      }
    }
  ]
}

Tool Results

Results from tool executions are represented as tool messages:

{
  id: "result_789",
  role: "tool",
  content: "168",
  toolCallId: "call_456" // References the original tool call
}

This creates a clear chain of tool usage:

1. Assistant requests a tool call
2. Tool executes and returns a result
3. Assistant can reference and respond to the result

Streaming Tool Calls

Similar to text messages, tool calls can be streamed to provide real-time visibility into the agent's actions:

1. Start a tool call:

   interface ToolCallStartEvent {
     type: EventType.TOOL_CALL_START
     toolCallId: string
     toolCallName: string
     parentMessageId?: string // Optional link to parent message
   }

2. Stream arguments:

   interface ToolCallArgsEvent {
     type: EventType.TOOL_CALL_ARGS
     toolCallId: string
     delta: string // JSON fragment to append to arguments
   }

3. End a tool call:

   interface ToolCallEndEvent {
     type: EventType.TOOL_CALL_END
     toolCallId: string
   }

This allows frontends to show tools being invoked progressively as the agent constructs its reasoning.

Practical Example

Here's a complete example of a conversation with tool usage:

// Conversation history
;[
  // User query
  {
    id: "msg_1",
    role: "user",
    content: "What's the weather in New York?",
  },

  // Assistant response with tool call
  {
    id: "msg_2",
    role: "assistant",
    content: "Let me check the weather for you.",
    toolCalls: [
      {
        id: "call_1",
        type: "function",
        function: {
          name: "get_weather",
          arguments: '{"location": "New York", "unit": "celsius"}',
        },
      },
    ],
  },

  // Tool result
  {
    id: "result_1",
    role: "tool",
    content:
      '{"temperature": 22, "condition": "Partly Cloudy", "humidity": 65}',
    toolCallId: "call_1",
  },

  // Assistant's final response using tool results
  {
    id: "msg_3",
    role: "assistant",
    content:
      "The weather in New York is partly cloudy with a temperature of 22°C and 65% humidity.",
  },
]

Conclusion

The message structure in AG-UI enables sophisticated conversational AI experiences while maintaining vendor neutrality. By standardizing how messages are represented, synchronized, and streamed, AG-UI provides a consistent way to implement interactive human-agent communication regardless of the underlying AI service.

This system supports everything from simple text exchanges to complex tool-based workflows, all while optimizing for both real-time responsiveness and efficient data transfer.

State Management

Source: https://docs.ag-ui.com/concepts/state

Understanding state synchronization between agents and frontends in AG-UI

State Management

State management is a core feature of the AG-UI protocol that enables real-time synchronization between agents and frontend applications. By providing efficient mechanisms for sharing and updating state, AG-UI creates a foundation for collaborative experiences where both AI agents and human users can work together seamlessly.

Shared State Architecture

In AG-UI, state is a structured data object that:

1. Persists across interactions with an agent
2. Can be accessed by both the agent and the frontend
3. Updates in real-time as the interaction progresses
4. Provides context for decision-making on both sides

This shared state architecture creates a bidirectional communication channel where:

• Agents can access the application's current state to make informed decisions
• Frontends can observe and react to changes in the agent's internal state
• Both sides can modify the state, creating a collaborative workflow

State Synchronization Methods

AG-UI provides two complementary methods for state synchronization:

State Snapshots

The STATE_SNAPSHOT event delivers a complete representation of an agent's current state:

interface StateSnapshotEvent {
  type: EventType.STATE_SNAPSHOT
  snapshot: any // Complete state object
}

Snapshots are typically used:

• At the beginning of an interaction to establish the initial state
• After connection interruptions to ensure synchronization
• When major state changes occur that require a complete refresh
• To establish a new baseline for future delta updates

When a frontend receives a STATE_SNAPSHOT event, it should replace its existing state model entirely with the contents of the snapshot.

State Deltas

The STATE_DELTA event delivers incremental updates to the state using JSON Patch format (RFC 6902):

interface StateDeltaEvent {
  type: EventType.STATE_DELTA
  delta: JsonPatchOperation[] // Array of JSON Patch operations
}

Deltas are bandwidth-efficient, sending only what has changed rather than the entire state. This approach is particularly valuable for:

• Frequent small updates during streaming interactions
• Large state objects where most properties remain unchanged
• High-frequency updates that would be inefficient to send as full snapshots

JSON Patch Format

AG-UI uses the JSON Patch format (RFC 6902) for state deltas, which defines a standardized way to express changes to a JSON document:

interface JsonPatchOperation {
  op: "add" | "remove" | "replace" | "move" | "copy" | "test"
  path: string // JSON Pointer (RFC 6901) to the target location
  value?: any // The value to apply (for add, replace)
  from?: string // Source path (for move, copy)
}

Common operations include:

1. add: Adds a value to an object or array

   { "op": "add", "path": "/user/preferences", "value": { "theme": "dark" } }

2. replace: Replaces a value

   { "op": "replace", "path": "/conversation_state", "value": "paused" }

3. remove: Removes a value

   { "op": "remove", "path": "/temporary_data" }

4. move: Moves a value from one location to another

   { "op": "move", "path": "/completed_items", "from": "/pending_items/0" }

Frontends should apply these patches in sequence to maintain an accurate state representation. If inconsistencies are detected after applying patches, the frontend can request a fresh STATE_SNAPSHOT.

State Processing in AG-UI

In the AG-UI implementation, state deltas are applied using the fast-json-patch library:

case EventType.STATE_DELTA: {
  const { delta } = event as StateDeltaEvent;

  try {
    // Apply the JSON Patch operations to the current state without mutating the original
    const result = applyPatch(state, delta, true, false);
    state = result.newDocument;
    return emitUpdate({ state });
  } catch (error: unknown) {
    console.warn(
      `Failed to apply state patch:\n` +
      `Current state: ${JSON.stringify(state, null, 2)}\n` +
      `Patch operations: ${JSON.stringify(delta, null, 2)}\n` +
      `Error: {{errorMessage}}`
    );
    return emitNoUpdate();
  }
}

This implementation ensures that:

• Patches are applied atomically (all or none)
• The original state is not mutated during the application process
• Errors are caught and handled gracefully

Human-in-the-Loop Collaboration

The shared state system is fundamental to human-in-the-loop workflows in AG-UI. It enables:

1. Real-time visibility: Users can observe the agent's thought process and current status
2. Contextual awareness: The agent can access user actions, preferences, and application state
3. Collaborative decision-making: Both human and AI can contribute to the evolving state
4. Feedback loops: Humans can correct or guide the agent by modifying state properties

For example, an agent might update its state with a proposed action:

{
  "proposal": {
    "action": "send_email",
    "recipient": "client@example.com",
    "content": "Draft email content..."
  }
}

The frontend can display this proposal to the user, who can then approve, reject, or modify it before execution.

CopilotKit Implementation

CopilotKit, a popular framework for building AI assistants, leverages AG-UI's state management system through its "shared state" feature. This implementation enables bidirectional state synchronization between agents (particularly LangGraph agents) and frontend applications.

CopilotKit's shared state system is implemented through:

// In the frontend React application
const { state: agentState, setState: setAgentState } = useCoAgent({
  name: "agent",
  initialState: { someProperty: "initialValue" },
})

This hook creates a real-time connection to the agent's state, allowing:

1. Reading the agent's current state in the frontend
2. Updating the agent's state from the frontend
3. Rendering UI components based on the agent's state

On the backend, LangGraph agents can emit state updates using:

# In the LangGraph agent
async def tool_node(self, state: ResearchState, config: RunnableConfig):
    # Update state with new information
    tool_state = {
        "title": new_state.get("title", ""),
        "outline": new_state.get("outline", {}),
        "sections": new_state.get("sections", []),
        # Other state properties...
    }

    # Emit updated state to frontend
    await copilotkit_emit_state(config, tool_state)

    return tool_state

These state updates are transmitted using AG-UI's state snapshot and delta mechanisms, creating a seamless shared context between agent and frontend.

Best Practices

When implementing state management in AG-UI:

1. Use snapshots judiciously: Full snapshots should be sent only when necessary to establish a baseline.
2. Prefer deltas for incremental changes: Small state updates should use deltas to minimize data transfer.
3. Structure state thoughtfully: Design state objects to support partial updates and minimize patch complexity.
4. Handle state conflicts: Implement strategies for resolving conflicting updates from agent and frontend.
5. Include error recovery: Provide mechanisms to resynchronize state if inconsistencies are detected.
6. Consider security implications: Avoid storing sensitive information in shared state.

Conclusion

AG-UI's state management system provides a powerful foundation for building collaborative applications where humans and AI agents work together. By efficiently synchronizing state between frontend and backend through snapshots and JSON Patch deltas, AG-UI enables sophisticated human-in-the-loop workflows that combine the strengths of both human intuition and AI capabilities.

The implementation in frameworks like CopilotKit demonstrates how this shared state approach can create collaborative experiences that are more effective than either fully autonomous systems or traditional user interfaces.

Tools

Source: https://docs.ag-ui.com/concepts/tools

Understanding tools and how they enable human-in-the-loop AI workflows

Tools

Tools are a fundamental concept in the AG-UI protocol that enable AI agents to interact with external systems and incorporate human judgment into their workflows. By defining tools in the frontend and passing them to agents, developers can create sophisticated human-in-the-loop experiences that combine AI capabilities with human expertise.

What Are Tools?

In AG-UI, tools are functions that agents can call to:

1. Request specific information
2. Perform actions in external systems
3. Ask for human input or confirmation
4. Access specialized capabilities

Tools bridge the gap between AI reasoning and real-world actions, allowing agents to accomplish tasks that would be impossible through conversation alone.

Tool Structure

Tools follow a consistent structure that defines their name, purpose, and expected parameters:

interface Tool {
  name: string // Unique identifier for the tool
  description: string // Human-readable explanation of what the tool does
  parameters: {
    // JSON Schema defining the tool's parameters
    type: "object"
    properties: {
      // Tool-specific parameters
    }
    required: string[] // Array of required parameter names
  }
}

The parameters field uses JSON Schema to define the structure of arguments that the tool accepts. This schema is used by both the agent (to generate valid tool calls) and the frontend (to validate and parse tool arguments).

Frontend-Defined Tools

A key aspect of AG-UI's tool system is that tools are defined in the frontend and passed to the agent during execution:

// Define tools in the frontend
const userConfirmationTool = {
  name: "confirmAction",
  description: "Ask the user to confirm a specific action before proceeding",
  parameters: {
    type: "object",
    properties: {
      action: {
        type: "string",
        description: "The action that needs user confirmation",
      },
      importance: {
        type: "string",
        enum: ["low", "medium", "high", "critical"],
        description: "The importance level of the action",
      },
    },
    required: ["action"],
  },
}

// Pass tools to the agent during execution
agent.runAgent({
  tools: [userConfirmationTool],
  // Other parameters...
})

This approach has several advantages:

1. Frontend control: The frontend determines what capabilities are available to the agent
2. Dynamic capabilities: Tools can be added or removed based on user permissions, context, or application state
3. Separation of concerns: Agents focus on reasoning while frontends handle tool implementation
4. Security: Sensitive operations are controlled by the application, not the agent

Tool Call Lifecycle

When an agent needs to use a tool, it follows a standardized sequence of events:

1. ToolCallStart: Indicates the beginning of a tool call with a unique ID and tool name

   {
     type: EventType.TOOL_CALL_START,
     toolCallId: "tool-123",
     toolCallName: "confirmAction",
     parentMessageId: "msg-456" // Optional reference to a message
   }

2. ToolCallArgs: Streams the tool arguments as they're generated

   {
     type: EventType.TOOL_CALL_ARGS,
     toolCallId: "tool-123",
     delta: '{"act' // Partial JSON being streamed
   }

   {
     type: EventType.TOOL_CALL_ARGS,
     toolCallId: "tool-123",
     delta: 'ion":"Depl' // More JSON being streamed
   }

   {
     type: EventType.TOOL_CALL_ARGS,
     toolCallId: "tool-123",
     delta: 'oy the application to production"}' // Final JSON fragment
   }

3. ToolCallEnd: Marks the completion of the tool call

   {
     type: EventType.TOOL_CALL_END,
     toolCallId: "tool-123"
   }

The frontend accumulates these deltas to construct the complete tool call arguments. Once the tool call is complete, the frontend can execute the tool and provide results back to the agent.

Tool Results

After a tool has been executed, the result is sent back to the agent as a "tool message":

{
  id: "result-789",
  role: "tool",
  content: "true", // Tool result as a string
  toolCallId: "tool-123" // References the original tool call
}

This message becomes part of the conversation history, allowing the agent to reference and incorporate the tool's result in subsequent responses.

Human-in-the-Loop Workflows

The AG-UI tool system is especially powerful for implementing human-in-the-loop workflows. By defining tools that request human input or confirmation, developers can create AI experiences that seamlessly blend autonomous operation with human judgment.

For example:

1. Agent needs to make an important decision
2. Agent calls the confirmAction tool with details about the decision
3. Frontend displays a confirmation dialog to the user
4. User provides their input
5. Frontend sends the user's decision back to the agent
6. Agent continues processing with awareness of the user's choice

This pattern enables use cases like:

• Approval workflows: AI suggests actions that require human approval
• Data verification: Humans verify or correct AI-generated data
• Collaborative decision-making: AI and humans jointly solve complex problems
• Supervised learning: Human feedback improves future AI decisions

CopilotKit Integration

CopilotKit provides a simplified way to work with AG-UI tools in React applications through its [useCopilotAction]

Truncated - read the full file at https://github.com/GreenShadeZhang/ag-ui-sample/blob/04de96745b20b0f69e052b9d0738c3b6d456993f/.github/prompts/ag-ui.prompt.md.