Response Synthesis

Purpose and Scope

Response synthesis is the final stage of a typical LlamaIndex retrieval-augmented generation flow. After an index-backed retriever finds candidate nodes and optional postprocessors rerank, transform, or filter those nodes, the response synthesizer combines the user query, retrieved text chunks, prompt configuration, and model call strategy to produce the answer returned to the application. The official querying guide describes this as the stage where the query, most-relevant data, and prompt are combined and sent to an LLM, while the module guide names the component responsible for that work a Response Synthesizer.

The practical reader problem is deciding where answer construction belongs. Retrieval should focus on finding evidence; node postprocessing should focus on improving or constraining that evidence; response synthesis should focus on turning the evidence into a final response object. Keeping those responsibilities separate makes it easier to tune each step independently. For example, increasing top_k changes retrieval breadth, adding a score threshold changes postprocessing, and switching response mode changes how the synthesizer spends LLM calls across retrieved chunks.

Sources: docs/api_reference/api_reference/response_synthesizers/index.md, llama-index-core/llama_index/core/base/response/schema.py

Relevant Source Files

• docs/api_reference/api_reference/response_synthesizers/index.md - Defines the generated API reference surface for response synthesizers, including BaseSynthesizer, get_response_synthesizer, and ResponseMode.
• llama-index-core/llama_index/core/base/response/schema.py - Defines the concrete response container types returned by query and synthesis paths, including non-streaming, structured, and streaming response shapes.
• docs/api_reference/api_reference/schema/index.md - Points the generated schema reference at llama_index.core.schema, the broader schema module that includes node-related types used by responses, such as NodeWithScore.

These files split the topic into two layers. The response synthesizer API reference tells readers which public names to look for when constructing or configuring synthesis behavior. The response schema file shows what applications receive after synthesis completes or streams. The schema index matters because response objects carry source_nodes, and those source nodes are typed with core schema objects rather than plain strings. Together, these sources show that synthesis is not just a text-generation helper; it is a boundary between retrieval artifacts and application-facing response values.

Sources: docs/api_reference/api_reference/response_synthesizers/index.md, llama-index-core/llama_index/core/base/response/schema.py, docs/api_reference/api_reference/schema/index.md

System-to-Code Mapping

In code-facing terms, the response synthesizer family is exposed through the API reference module entries for llama_index.core.response_synthesizers.base, llama_index.core.response_synthesizers.factory, and llama_index.core.response_synthesizers.type. The documented members are BaseSynthesizer, get_response_synthesizer, and ResponseMode. BaseSynthesizer is the conceptual base contract for synthesizers, get_response_synthesizer is the factory-style entry point used in examples, and ResponseMode names the selectable strategies. The official module guide demonstrates passing response_mode=ResponseMode.COMPACT or response_mode="compact" when creating a synthesizer.

The response schema maps the output side. Response is the standard non-streaming response container. It stores response: Optional[str], source_nodes: List[NodeWithScore], and optional metadata. Its __str__ returns the response text or None, which is why simple examples can print the query result directly. get_formatted_sources(length=100) iterates over source_nodes, truncates each node content, reads the node id, and returns readable source excerpts. That source formatting behavior is important for user interfaces and debugging because it preserves a link from the generated answer back to the retrieved context.

Sources: docs/api_reference/api_reference/response_synthesizers/index.md, llama-index-core/llama_index/core/base/response/schema.py

Execution Flow

A normal query flow begins with a query engine, often created from an index with index.as_query_engine(). The query engine receives the user question and coordinates retrieval, optional node postprocessing, and response synthesis. In the official docs, a response synthesizer can also be created separately and passed into index.as_query_engine(response_synthesizer=response_synthesizer). That makes synthesis a swappable component rather than a hidden implementation detail of the index. Developers can start with defaults and later replace only the synthesis step when answer style, latency, token budget, or structured-output requirements change.

When used directly, the response synthesizer receives query text plus nodes. The official examples show response_synthesizer.synthesize("query text", nodes=[...]), with nodes represented as NodeWithScore values wrapping node content and score metadata. That shape mirrors the response schema: the final response keeps source_nodes with the answer. This continuity is useful because downstream code can render an answer, inspect metadata, and display source excerpts without rerunning retrieval. It also means that synthesis strategy should avoid discarding provenance unless the application intentionally transforms it.

Sources: llama-index-core/llama_index/core/base/response/schema.py, docs/api_reference/api_reference/schema/index.md

from llama_index.core import get_response_synthesizer
 
response_synthesizer = get_response_synthesizer(response_mode="compact")
query_engine = index.as_query_engine(response_synthesizer=response_synthesizer)
 
response = query_engine.query("What does this corpus say about renewal terms?")
print(str(response))
print(response.get_formatted_sources(length=160))

API Components

The compact public reference for this page is intentionally small. docs/api_reference/api_reference/response_synthesizers/index.md registers only three members: BaseSynthesizer, get_response_synthesizer, and ResponseMode. That tells developers to treat synthesis configuration as a focused API surface. Use the factory when selecting a built-in mode, use the mode enumeration or string values when configuring strategy, and look at the base class when implementing a custom synthesizer. The official guide frames the available strategies broadly: synthesis may be as simple as iterating over chunks or as complex as building a tree of intermediate answers.

ResponseMode.COMPACT is the mode shown in the official guide snippets, and the same guide also shows the string value "compact". The important implementation-independent idea is that response mode changes the way retrieved chunks are combined and sent through LLM prompts. A compact mode is usually chosen to fit chunks together efficiently, while other modes may trade latency and cost for better summarization structure or stepwise reasoning. Because the response object shape remains stable, applications can often experiment with synthesis modes without changing display, logging, or evaluation code.

Sources: docs/api_reference/api_reference/response_synthesizers/index.md, llama-index-core/llama_index/core/base/response/schema.py

Response Object Behavior

Response is optimized for ordinary text answers. Its response field may be absent, so string conversion returns "None" when there is no text. Its source_nodes default to an empty list, making it safe for flows that synthesize without retrieved evidence or for tests that construct minimal responses. Its metadata field is optional and can carry application-specific or synthesizer-specific information. The get_formatted_sources helper is deliberately presentation-oriented: it truncates node content to a caller-provided length and emits a per-source line with the node id.

PydanticResponse supports structured outputs while preserving the same source and metadata pattern. Its payload is Optional[BaseModel], and string conversion serializes the model as JSON when present. It also implements attribute access that prioritizes fields on the underlying Pydantic response object, returning None when the requested field is not present. That behavior lets callers treat structured answers ergonomically while still retaining a conversion path through get_response(), which turns the structured payload into a standard Response containing JSON text.

Sources: llama-index-core/llama_index/core/base/response/schema.py

The response schema file also imports TokenGen and TokenAsyncGen, and begins defining StreamingResponse for streaming=True flows. Even from the visible constructor fields, the pattern is clear: streaming responses carry a generator, source nodes, optional metadata, and accumulated response text. This keeps streaming compatible with the same provenance model as non-streaming synthesis. A UI can consume generated tokens incrementally while still preserving the source nodes that explain where the answer came from once retrieval has completed.

Implementation Guidance

Choose the simplest integration point that matches the amount of control you need. If you are building a basic RAG application, call index.as_query_engine() and rely on defaults until you have a reason to tune synthesis. If your answers are too verbose, too terse, too slow, or too expensive, create a synthesizer with get_response_synthesizer(response_mode=...) and pass it into the query engine. If your application needs typed fields rather than prose, design around PydanticResponse and convert to Response only when a plain text representation is required.

When displaying responses, avoid treating the generated text as the only output. The schema makes source_nodes a first-class part of the response containers, so developer tools, chat UIs, evaluation scripts, and audit views should keep those nodes available. get_formatted_sources() is a convenient built-in formatter for quick inspection, but production applications may prefer to render node ids, snippets, scores, and metadata in a custom layout. The key design constraint is to preserve the connection between synthesized answer and retrieved evidence for debugging and user trust.

Sources: llama-index-core/llama_index/core/base/response/schema.py, docs/api_reference/api_reference/response_synthesizers/index.md

Next Steps

After this page, read the query engine and retriever documentation to understand the stages that feed the synthesizer. Then review documents and nodes, because source attribution depends on node content and node ids. For application behavior, experiment with get_response_synthesizer(response_mode="compact") in an existing query engine and compare answer quality, latency, and formatted sources. For custom output contracts, inspect the broader schema reference and design whether your application should consume Response, PydanticResponse, or a streaming response shape.

Related pages: query-engines, retrievers, documents-and-nodes, streaming, core-api