Chat Engines

Purpose and Scope

A chat engine is the LlamaIndex interface for a conversation over data rather than a single isolated query. The official deployment guide frames it as a stateful counterpart to a query engine: it keeps conversation history so later turns can be answered with previous user and assistant messages in mind. That distinction is the reader problem this page addresses. If you want standalone question answering over an index, start with a query engine. If you want an application that behaves like a knowledge-base-aware assistant across multiple back-and-forth turns, use a chat engine.

The repository documentation exposes chat engines in two complementary ways. The API reference index maps the shared typed interface through llama_index.core.chat_engine.types, while the mode-specific pages identify concrete chat engine classes under llama_index.core.chat_engine. The community FAQ reinforces the intended use case: retaining context while answering is explicitly a chat-engine responsibility, and data-agent usage is routed through chat mode configuration when initializing a chat engine. Sources: docs/api_reference/api_reference/chat_engines/index.md, docs/src/content/docs/framework/community/faq/chat_engines.md

Chat engines sit at the boundary between retrieval, generation, and application state. An index can be turned into a chat engine with index.as_chat_engine(), after which the application calls chat() for a complete response or stream_chat() for token-by-token streaming. The engine may retrieve relevant context from indexed data, apply the selected chat mode, include conversation history, and return a response object suitable for the user interface. The API pages in this repository do not document every runtime parameter inline, but they do name the principal public engine classes that readers will encounter in guides and reference pages. Sources: docs/api_reference/api_reference/chat_engines/condense_plus_context.md, docs/api_reference/api_reference/chat_engines/condense_question.md, docs/api_reference/api_reference/chat_engines/context.md, docs/api_reference/api_reference/chat_engines/simple.md

Relevant Source Files

• docs/api_reference/api_reference/chat_engines/index.md - API reference entry point for shared chat engine types from llama_index.core.chat_engine.types.
• docs/api_reference/api_reference/chat_engines/simple.md - API reference page for SimpleChatEngine, the simplest named chat engine implementation in the reference set.
• docs/api_reference/api_reference/chat_engines/context.md - API reference page for ContextChatEngine, the mode focused on conversational answering with contextual information.
• docs/api_reference/api_reference/chat_engines/condense_question.md - API reference page for CondenseQuestionChatEngine, the mode that condenses a conversational turn into a standalone question before querying.
• docs/api_reference/api_reference/chat_engines/condense_plus_context.md - API reference page for CondensePlusContextChatEngine, the mode that combines question condensation with contextual response generation.
• docs/src/content/docs/framework/community/faq/chat_engines.md - Community FAQ confirming that chat engines are the supported way to retain conversational context and that data agents use chat mode selection.

Core Primitives

The first primitive is the typed chat engine interface. The API reference entry point is generated from llama_index.core.chat_engine.types, which is the place readers should look for shared chat-engine contracts such as common request, response, streaming, and lifecycle types. Even when a concrete engine is created through an index helper rather than instantiated directly, the application-level mental model remains the same: a chat engine accepts a user message in the context of prior messages and returns an assistant response that may have been informed by retrieval. Sources: docs/api_reference/api_reference/chat_engines/index.md

The second primitive is chat mode. Mode names select different strategies for handling the relationship between the current user message, prior conversation, and indexed data. SimpleChatEngine is the most direct entry in the public reference. ContextChatEngine emphasizes adding context from data. CondenseQuestionChatEngine is for workflows where a follow-up question needs to be rewritten into a standalone query before retrieval. CondensePlusContextChatEngine combines question condensation with context-aware answering. These names are not just labels; they tell you where conversational state is transformed before retrieval and where retrieved context is inserted before synthesis. Sources: docs/api_reference/api_reference/chat_engines/simple.md, docs/api_reference/api_reference/chat_engines/context.md, docs/api_reference/api_reference/chat_engines/condense_question.md, docs/api_reference/api_reference/chat_engines/condense_plus_context.md

The third primitive is conversation state. The deployment documentation describes a chat engine as stateful because it tracks back-and-forth history. In practical applications, that state may live only for the current process, or it may be connected to a persistent chat store when you need conversations to survive restarts, user sessions, or distributed deployment. Treat the chat store as the persistence layer for messages, while the chat engine is the runtime interface that decides how messages, retrieved nodes, prompts, and model output are combined for each turn. Sources: docs/src/content/docs/framework/community/faq/chat_engines.md

The fourth primitive is streaming. A chat user interface often should not wait for an entire answer before rendering anything. The official usage pattern shows stream_chat() returning an object with a response_gen iterator, allowing the caller to print or display tokens as they arrive. Streaming does not change the conceptual role of the chat engine; it changes the delivery contract. The engine still uses its mode, history, and retrieval behavior, but the response is consumed incrementally instead of as one completed message. Sources: docs/api_reference/api_reference/chat_engines/index.md

Chat Engine Modes

Use SimpleChatEngine when you want the smallest named chat engine abstraction and do not need a specialized retrieval-conversation transformation. It is useful for learning the interface, for tests, and for applications where the model can answer directly with available chat history. The source reference page documents it as a public member of llama_index.core.chat_engine, which means it is part of the chat engine API family rather than an example-only helper. Start here when the main question is how your UI should call a chat engine, not how to optimize retrieval behavior. Sources: docs/api_reference/api_reference/chat_engines/simple.md

Use ContextChatEngine when the assistant should answer in conversation while grounding itself in contextual information from the LlamaIndex data layer. This mode is the natural bridge from an index-backed retrieval application to an interactive assistant. The key design idea is that the user sees a conversational experience, while the engine can still bring in relevant indexed context behind the scenes. That makes it a strong default for knowledge-base chat where users ask follow-up questions, clarify prior requests, or refine the same information need over several turns. Sources: docs/api_reference/api_reference/chat_engines/context.md

Use CondenseQuestionChatEngine when follow-up questions need to become explicit standalone retrieval queries. In a conversation, a user may ask, “What about the second option?” or “Can you compare it with last year?” Those phrases depend on prior turns and are often poor retrieval queries by themselves. The condensation step rewrites the conversational turn into a more complete question before the retrieval-oriented part of the pipeline runs. This mode is therefore valuable when retrieval quality depends on the current question being self-contained. Sources: docs/api_reference/api_reference/chat_engines/condense_question.md

Use CondensePlusContextChatEngine when both parts matter: the user’s message should be condensed with the help of chat history, and the final answer should be generated with retrieved or supplied context. The name exposes the intended composition. It is appropriate for production-style knowledge assistants where follow-up turns are common and answer grounding is important. Compared with a simple mode, this approach introduces more orchestration, but it gives the application better control over ambiguity introduced by conversation history. Sources: docs/api_reference/api_reference/chat_engines/condense_plus_context.md

Usage and Execution Flow

The shortest usage path is to build or load an index, turn it into a chat engine, and call chat(). The official guide uses index.as_chat_engine() followed by chat_engine.chat("Tell me a joke."). In an application, the same call usually receives the latest user input from a web request, CLI prompt, notebook cell, or background job. The important point is that the caller does not need to manually reassemble all prior turns for every request when the chosen chat engine manages state. That is the difference from a stateless query engine call.

chat_engine = index.as_chat_engine()
response = chat_engine.chat("Tell me a joke.")

For streaming user interfaces, call stream_chat() and iterate over the returned generator. The official guide shows streaming_response.response_gen, which lets a terminal, browser, or chat client render tokens as they are produced. This is the preferred shape for latency-sensitive interfaces: users see progress, cancellation can be handled at the UI layer, and long responses do not block rendering until completion. The engine choice still matters. A condense mode may do preparatory work before the first generated token, while a simpler mode may begin output sooner.

chat_engine = index.as_chat_engine()
streaming_response = chat_engine.stream_chat("Tell me a joke.")
for token in streaming_response.response_gen:
    print(token, end="")

When data agents are involved, the community FAQ says to set the chat mode while initializing the chat engine. That advice matters because agent behavior and chat behavior are related but not identical. A data agent may decide when to use tools or data-access capabilities, while the chat engine provides the conversational wrapper and history-aware interaction surface. In practice, choose the chat mode based on the kind of conversational transformation you need, then connect the resulting engine to the agent or application layer that handles user requests. Sources: docs/src/content/docs/framework/community/faq/chat_engines.md

API Components Reference

The API reference pages are generated through documentation directives rather than long hand-written pages. Each mode-specific page points at llama_index.core.chat_engine and lists exactly one public member. This is useful when navigating the generated reference because the page title tells you the mode, and the directive tells you the import family. If you are reading code or notebook examples and see one of these class names, use the matching reference page to confirm that it belongs to the supported chat engine surface instead of a private helper. Sources: docs/api_reference/api_reference/chat_engines/condense_plus_context.md, docs/api_reference/api_reference/chat_engines/condense_question.md, docs/api_reference/api_reference/chat_engines/context.md, docs/api_reference/api_reference/chat_engines/simple.md

Deployment, State, and Chat Stores

Deployment turns the stateful nature of chat engines into an architectural concern. In a notebook, keeping history in memory may be enough. In a web service, the application must know which conversation belongs to which user, how long messages should be retained, and how workers recover history after a restart. The official concept page defines the chat engine as stateful, and the FAQ confirms that context retention is the central reason to use chat engines. That means deployment design should treat conversation history as first-class application state, not as incidental prompt text. Sources: docs/src/content/docs/framework/community/faq/chat_engines.md

Chat stores are the persistence-facing companion to chat engines. A chat engine is the interface that receives messages and returns assistant responses; a chat store is where message history can be stored when conversations need durability or session isolation. This separation lets an application change storage choices without changing the high-level chat call pattern. For example, a development prototype can use transient state, while a deployed service can attach a persistent store keyed by user or session. When reading the broader OpenWiki set, pair this page with the sessions and storage pages for details about long-lived state.

The practical deployment checklist is straightforward. Decide whether the application needs one-off query behavior or stateful chat behavior. Choose the chat mode that matches the expected conversation pattern. Decide how user identity maps to chat history. Decide whether streaming is required for the client experience. Finally, decide how chat history is stored, cleared, and audited. These decisions are more important than the first index.as_chat_engine() call because they determine whether the assistant behaves consistently across multiple turns, browser refreshes, background workers, and production restarts.

Testing Signals and Next Steps

A useful chat-engine test should cover more than one turn. Ask an initial question that establishes a subject, then ask a follow-up question that depends on that subject. A successful stateful engine should preserve enough conversational context to answer correctly, and a condensation-oriented mode should produce behavior consistent with a standalone rewritten question. For streaming, test that the client consumes response_gen incrementally and still handles completion, errors, and cancellation. For deployment, test that separate users or sessions do not share history unless that is explicitly intended.

Next, read the query engine page if you are deciding between stateless and stateful interfaces, the sessions page if you need durable conversation state, and the streaming page if your application renders partial model output. If you are building a data-agent experience, review the agents and tools pages after choosing a chat mode. The most common path is to prototype with index.as_chat_engine(), validate conversation behavior with a small indexed corpus, then harden session storage and streaming delivery before exposing the assistant to users.