Channels
Purpose and Scope
A channel is the application-facing communication surface that connects an outside client to a LlamaIndex agent, chat engine, or voice runtime. The term is useful even though the supplied source does not define one monolithic Channel class, because the repository exposes the pattern through typed response objects, streaming queues, session updates, audio input events, deltas, and tool-result events. When a frontend, gateway, worker, or voice client talks to LlamaIndex, the channel is where user input becomes runtime messages and where runtime output becomes visible text, audio, metadata, or control signals. Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx, llama-index-core/llama_index/core/chat_engine/types.py, llama-index-integrations/voice_agents/llama-index-voice-agents-openai/llama_index/voice_agents/openai/types.py
This page is intended for developers who are designing that boundary rather than only calling a local Python object. A web chat application may need server-sent events or WebSockets; a background worker may need a queue; a voice experience may need session configuration, audio frames, transcripts, and tool-call correlation. The same LlamaIndex runtime facts should survive all of those transports: the latest user message, conversation history or memory, tool schemas, tool calls, tool results, response deltas, source information, completion state, and errors. Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx, llama-index-core/llama_index/core/chat_engine/types.py
Relevant Source Files
docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx— defines an agent as an LLM, memory, and tools handling outside user input; shows a FunctionAgent run call; explains the tool-call loop; describes memory customization; and notes that streaming is enabled by default for compatible models.llama-index-core/llama_index/core/chat_engine/types.py— defines chat response modes, completed agent chat responses, streaming agent chat responses, synchronous and asynchronous generators, queues, completion flags, source propagation, and chat-engine instrumentation events.llama-index-integrations/voice_agents/llama-index-voice-agents-openai/llama_index/voice_agents/openai/types.py— defines OpenAI voice-agent models for voice activity detection, session configuration, audio input, output deltas, completed outputs, function-call completion, and function-result sending.
Core Primitives
The agent guide establishes the basic channel contract for agent applications. An agent receives outside user input, combines it with chat history, sends tool schemas and history to the model, and either returns a direct answer or requests tool calls. Each tool call is executed, the result is added back into history, and the agent is invoked again until the model can answer. For channel design, this means the client request is not merely a string; it begins a loop that can produce intermediate tool activity before producing a final response. Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx
The same guide also shows why channels must account for memory and model capability. LlamaIndex agents use memory by default, and a caller can provide a separate ChatMemoryBuffer when running an agent. That makes the channel responsible for deciding whether conversation state lives in a local process, a deployed service, or a client-controlled session. The guide also warns that some models may not support streaming output, even though streaming is enabled by default, so a robust channel should have a fallback path for waiting responses. Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx
For text chat, the core chat engine types provide the most direct source-level representation of channel behavior. ChatResponseMode distinguishes wait mode from stream mode. AgentChatResponse represents a completed answer with response text, tool sources, source nodes, optional metadata, and a string conversion that returns the response. StreamingAgentChatResponse represents an active stream with synchronous or asynchronous chat generators, standard and asynchronous queues, accumulated response text, source tracking, an is_function marker, and done signaling. Those fields are the pieces an adapter should preserve when translating to an external transport. Sources: llama-index-core/llama_index/core/chat_engine/types.py
Voice channels add a richer session layer. ConversationSession declares supported modalities, default instructions, voice selection, input and output audio formats, transcription settings, server voice activity detection, available function tools, tool-choice policy, temperature, output-token limits, playback speed, and tracing behavior. ConversationSessionUpdate wraps that configuration as an event. This makes a voice channel more than an audio pipe: it is also a negotiated runtime configuration describing how turns are detected, how audio is encoded, which tools are available, and how much output may be generated. Sources: llama-index-integrations/voice_agents/llama-index-voice-agents-openai/llama_index/voice_agents/openai/types.py
System-to-Code Mapping
| Channel concern | Source-level representation | Practical meaning |
|---|---|---|
| Agent request boundary | FunctionAgent run flow in the agent guide | The client starts an agent loop with user input and optional memory. |
| Tool-call loop | Tool schemas, tool calls, tool results, updated chat history | The channel may need to expose intermediate actions before the final answer. |
| Blocking text output | AgentChatResponse | The caller receives a complete answer plus sources, source nodes, and metadata. |
| Streaming text output | StreamingAgentChatResponse and ChatResponseMode STREAM | The caller consumes deltas through generators or queues and watches done state. |
| Voice runtime configuration | ConversationSession and ConversationSessionUpdate | The client and runtime agree on modalities, instructions, audio formats, transcription, tools, and limits. |
| Audio input | ConversationInputEvent | The channel accepts audio bytes or base64 text and normalizes byte payloads. |
| Voice deltas and completion | ConversationDeltaEvent and ConversationDoneEvent | The channel emits incremental text or audio and then a final text or transcript payload. |
| Function-call correlation | FunctionCallDoneEvent, FunctionResultItem, SendFunctionItemEvent | The channel carries call identifiers, parsed arguments, and tool outputs back to the runtime. |
Execution Flow
A request-response text channel is the simplest flow. The client sends a prompt or message, the application calls the agent or chat engine, and the channel returns the completed response object. This works well for command-line tools, batch jobs, or services that do not need token-by-token rendering. The important implementation detail is to retain more than the visible answer. AgentChatResponse can also carry tool outputs, source nodes derived from response objects, and metadata. If the adapter only returns plain text, it may discard information needed for citations, debugging, evaluation, or audit trails. Sources: llama-index-core/llama_index/core/chat_engine/types.py
A streaming text channel should treat the response as a producer-consumer relationship. StreamingAgentChatResponse can hold a chat stream, an async chat stream, a standard queue, an async queue, events for new items, and a completion flag. That shape maps naturally to server-sent events, WebSockets, terminal streaming, or worker queues. The same stream may include normal assistant text and function-call information, so the is_function marker is significant. A UI should not assume every chunk is display text; it should handle tool-call control data, accumulated text, source collection, completion, and errors. Sources: llama-index-core/llama_index/core/chat_engine/types.py
A voice channel normally starts by sending or applying a session update, then sends input audio, then consumes deltas and completion events. ConversationInputEvent accepts audio as bytes or a string, and its validator base64-encodes byte payloads when they are not already valid base64. ConversationDeltaEvent carries incremental text or bytes with an item identifier, while ConversationDoneEvent carries the final item identifier plus optional text and transcript. That structure lets a voice UI correlate partial output, final transcript state, and any downstream persistence or display behavior. Sources: llama-index-integrations/voice_agents/llama-index-voice-agents-openai/llama_index/voice_agents/openai/types.py
Tool execution is part of the same channel story in both text and voice experiences. The agent guide describes model-selected tool calls being executed and appended to chat history before another model invocation. The voice integration models the corresponding event exchange with ConversationTool schemas, FunctionCallDoneEvent for call id, name, arguments, and item id, and SendFunctionItemEvent for returning a FunctionResultItem. FunctionCallDoneEvent validates that arguments can be parsed as JSON and raises an error when they are not serializable, so adapters should surface malformed tool arguments as channel errors rather than silently continuing. Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx, llama-index-integrations/voice_agents/llama-index-voice-agents-openai/llama_index/voice_agents/openai/types.py
API Components
| Name | Kind | Key fields or behavior |
|---|---|---|
| ChatResponseMode | enum | WAIT and STREAM select waiting versus streaming behavior. |
| AgentChatResponse | dataclass | response, sources, source_nodes, is_dummy_stream, metadata, response_gen, and async_response_gen. |
| StreamingAgentChatResponse | dataclass | response accumulation, sources, chat_stream, achat_stream, queue, aqueue, is_function, is_done, and event signaling. |
| ConversationVAD | model | Server voice activity detection settings such as threshold, padding, silence duration, and response creation. |
| ConversationSession | model | Modalities, instructions, voice, audio formats, transcription, turn detection, tools, tool choice, temperature, token limit, speed, and tracing. |
| ConversationInputEvent | event | Accepts audio bytes or string input and normalizes byte payloads to base64 when needed. |
| FunctionCallDoneEvent | event | Carries call id, optional name, JSON arguments, and item id; validates arguments by JSON parsing. |
| ConversationDeltaEvent | event | Carries incremental text or audio bytes with an item id. |
| ConversationDoneEvent | event | Carries final item id, optional text, and optional transcript. |
| SendFunctionItemEvent | event | Sends a FunctionResultItem containing the tool call id and output. |
Implementation Guidance
When building a channel adapter, start with the client contract and then map it back to the typed LlamaIndex surface. A command-line client can often wait for a complete response and print it. A browser chat client should usually stream deltas, expose completion, and preserve source metadata for citations or inspection. A voice client should configure the session before sending audio and should keep item identifiers and call identifiers intact. In all cases, keep transport concerns at the edge; the agent loop, memory, tool execution, and response objects should remain recognizable. Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx, llama-index-core/llama_index/core/chat_engine/types.py, llama-index-integrations/voice_agents/llama-index-voice-agents-openai/llama_index/voice_agents/openai/types.py
Also design for capability differences and failure states. The deployment guide notes that streaming can be disabled for models that do not support it, so channel implementations should support both waiting and streaming outputs rather than assuming one mode. The chat-engine module imports stream start, delta, end, and error instrumentation events, which indicates that streaming should be observable as a runtime phase. The voice event models validate audio and function-call arguments, so a production adapter should report validation failures clearly, stop or recover the affected turn, and avoid committing incomplete state as a successful conversation result. Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx, llama-index-core/llama_index/core/chat_engine/types.py, llama-index-integrations/voice_agents/llama-index-voice-agents-openai/llama_index/voice_agents/openai/types.py
Next, read the agents overview and agent configuration pages to understand how workflow agents select tools, apply prompts, and use memory. Read the sessions page when the channel needs durable state across requests or deployed runtime instances. Read the streaming page when implementing incremental rendering, cancellation, observability, or backpressure. Channels sit at the intersection of those topics: they are the concrete application boundary where prompts, tool calls, response chunks, audio frames, transcripts, sources, and completion events become a client-facing protocol.