Sessions

Purpose and Scope

A session is the stateful boundary around a conversation or agent run: it is the place where prior user messages, assistant responses, tool results, and sometimes streaming output are accumulated so the next step can reason with context instead of starting from scratch. In LlamaIndex agent docs, an agent is defined as a system that uses an LLM, memory, and tools to handle outside user input. That definition makes memory a first-class part of deployed agent behavior rather than a convenience wrapper around prompts.

Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx, llama-index-core/llama_index/core/memory/types.py

The practical reader problem is deciding where conversational state lives and how it is passed through runtime APIs. The deploying guide shows a FunctionAgent created with tools, an LLM, and a system prompt, then run asynchronously with await agent.run(...). It also explains the loop that makes sessions necessary: the agent reads the latest message plus chat history, sends tool schemas and history to the model, records tool call results into chat history, and invokes the model again with updated state until it can respond directly.

Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx

Relevant Source Files

docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx - Defines the deployed-agent mental model, the agent action loop, default memory behavior, and the supported pattern for passing custom memory to agent.run.
llama-index-core/llama_index/core/memory/types.py - Defines the abstract memory contract, async wrappers, chat-store-backed memory base class, default chat store key, and serialization behavior for chat stores.
docs/api_reference/api_reference/storage/chat_store/index.md - Exposes the storage API reference page for BaseChatStore, the backing store family used by chat-store memory implementations.
llama-index-core/llama_index/core/chat_engine/types.py - Defines chat response types, streaming response state, response/source metadata, and imports BaseMemory for chat engine integration points.

Core Primitives

The central runtime primitive is BaseMemory. It is an abstract BaseComponent with synchronous methods for reading and mutating chat history: get, get_all, put, put_messages, set, and reset. The same class also provides asynchronous equivalents such as aget, aget_all, aput, aput_messages, aset, and areset, implemented by delegating the synchronous work through asyncio.to_thread. This means a memory implementation can define the core storage behavior once while still fitting into async agent and chat workflows.

Sources: llama-index-core/llama_index/core/memory/types.py

BaseChatStoreMemory specializes memory for multi-tenant chat history by adding a chat_store and a chat_store_key. The default key is chat_history, and the default store is SimpleChatStore. Rather than storing messages directly on the memory object, this base class delegates reads to chat_store.get_messages(chat_store_key) and async reads to chat_store.aget_messages(chat_store_key). That key is the session selector: different keys can represent different conversations, users, threads, or deployments when a concrete memory implementation exposes them.

Sources: llama-index-core/llama_index/core/memory/types.py, docs/api_reference/api_reference/storage/chat_store/index.md

Chat engines participate in the same stateful model. The chat_engine/types.py module imports BaseMemory and defines response containers that carry not only text but also tool outputs, source nodes, metadata, and streaming state. AgentChatResponse is the non-streaming container, while StreamingAgentChatResponse is documented as a streaming chat response to the user and writing to chat history. That phrase captures the important session rule for streaming interfaces: output delivery and history mutation are coordinated parts of one runtime interaction.

Sources: llama-index-core/llama_index/core/chat_engine/types.py

Agent Session Flow

A typical stateful agent run begins with an input message and an existing memory object. The agent obtains chat history, packages that history alongside the latest user message and tool schemas, and asks the LLM to decide the next action. If the model returns a direct answer, the session can end with the final assistant message. If the model returns tool calls, each tool call is executed, the results are appended to chat history, and the agent repeats the model invocation with the updated context.

Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx

The deploying guide states that all LlamaIndex agents use ChatMemoryBuffer for memory by default, and it shows the customization path: create memory outside the agent and pass it to agent.run. That design is important for session ownership. If an application wants a fresh session, it can allow default memory or reset the memory. If it wants a durable or shared session, it should keep a memory object connected to an appropriate chat store and use a stable key for that conversation boundary.

Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx, llama-index-core/llama_index/core/memory/types.py

from llama_index.core.memory import ChatMemoryBuffer
 
memory = ChatMemoryBuffer.from_defaults(token_limit=40000)
 
response = await agent.run(
    "Continue from the previous discussion about quarterly results.",
    memory=memory,
 )

The same principle applies when the session is exposed through a chat engine instead of a workflow-style agent. AgentChatResponse can collect tool outputs in sources and derive source_nodes from raw Response or StreamingResponse objects. A session therefore carries more than visible chat text: it may also preserve provenance, tool-level artifacts, and metadata needed for debugging, observability, or follow-up answer generation. The user sees a conversation, but the runtime often maintains a richer record.

Sources: llama-index-core/llama_index/core/chat_engine/types.py

API Components Reference

Component	Source-level contract	Session role
`BaseMemory.class_name()`	Returns `BaseMemory`	Identifies the abstract memory component type
`BaseMemory.from_defaults(**kwargs)`	Abstract constructor	Standard factory hook for concrete memory implementations
`BaseMemory.get(input=None, **kwargs)` / `aget(...)`	Returns `List[ChatMessage]`	Reads the current context window for a session
`BaseMemory.get_all()` / `aget_all()`	Returns `List[ChatMessage]`	Reads the complete stored history
`BaseMemory.put(message)` / `aput(message)`	Accepts one `ChatMessage`	Appends a message to the session
`BaseMemory.put_messages(messages)` / `aput_messages(messages)`	Accepts `List[ChatMessage]`	Appends several messages in order
`BaseMemory.set(messages)` / `aset(messages)`	Accepts `List[ChatMessage]`	Replaces stored session history
`BaseMemory.reset()` / `areset()`	No arguments	Clears the session history
`BaseChatStoreMemory.chat_store`	Defaults to `SimpleChatStore`	Stores messages behind the memory interface
`BaseChatStoreMemory.chat_store_key`	Defaults to `chat_history`	Selects the conversation record within the store
`AgentChatResponse.response`	String response field	Final non-streaming user-visible answer
`StreamingAgentChatResponse.chat_stream` / `achat_stream`	Sync or async chat response generators	Streams model output while coordinating history writes

The chat store reference page exposes BaseChatStore as the documented storage API entry point. BaseChatStoreMemory depends on that contract instead of requiring every memory implementation to own persistence details. In code, it serializes the chat store by dumping the store model and adding class_name, which is a useful detail for configurations that need to persist or reconstruct stateful components. The separation between memory and chat store lets application code reason in terms of conversation history while storage backends handle message persistence.

Sources: docs/api_reference/api_reference/storage/chat_store/index.md, llama-index-core/llama_index/core/memory/types.py

Implementation Details and Constraints

The async memory methods are convenience adapters, not separate abstract storage protocols. Because they call the synchronous methods through asyncio.to_thread, a custom memory can start with blocking implementations and still be usable from async agent code. For higher-throughput deployments, concrete chat stores may still provide native async methods; BaseChatStoreMemory.aget and aget_all call chat_store.aget_messages directly, so the chat-store layer can optimize asynchronous reads when it supports them.

Sources: llama-index-core/llama_index/core/memory/types.py

Streaming introduces a second session concern: partial output must be delivered while final state remains coherent. StreamingAgentChatResponse includes fields such as queue, optional aqueue, is_done, new_item_event, is_function, unformatted_response, and stream handles for sync and async generation. These fields show that streaming sessions are not just generators of text tokens; they coordinate concurrency, function-call detection, completion signaling, accumulated response text, source nodes, and eventual chat history writes.

Sources: llama-index-core/llama_index/core/chat_engine/types.py

When designing a deployed service, treat memory lifetime as an application decision. A request-scoped memory gives every call a clean slate. A user-scoped memory creates a continuing assistant experience. A thread-scoped memory supports multiple independent conversations for one user. The LlamaIndex contracts give you the operations needed for each policy: set for restoring a known transcript, put for appending new turns, get for context selection, get_all for full audit or export, and reset for explicit session clearing.

Sources: llama-index-core/llama_index/core/memory/types.py

Next Steps

Start with the deploying agents guide when you need to understand the runtime loop and how to pass memory into agent.run. Then read the memory and chat store API references when you need durable, multi-tenant, or externally managed session state. If your application exposes incremental output, pair this page with the streaming documentation so you can decide how streamed deltas, final responses, tool outputs, and memory writes should be synchronized in your own service boundary.

Sources: docs/src/content/docs/framework/module_guides/deploying/agents/index.mdx, docs/api_reference/api_reference/storage/chat_store/index.md, llama-index-core/llama_index/core/chat_engine/types.py

Ask a question...