Callbacks
Purpose and Scope
Callbacks are the LlamaIndex observability surface for understanding what happens inside an application while it loads data, parses nodes, embeds text, retrieves context, calls language models, synthesizes responses, and runs higher-level query or agent operations. The callback system is useful when a developer wants to debug a surprising answer, measure token or latency patterns, send traces to an external product, or compare runs during experimentation. In this repository’s API documentation, callbacks are split between core callback abstractions and several integration-specific handlers that connect callback events to external observability or feedback systems.
The most important idea is that application code does not need to own every trace destination directly. Instead, callback handlers can be registered with a manager, and the manager coordinates event notifications as LlamaIndex components execute. The official guide describes callbacks as a way to debug, track, and trace the inner workings of the library, including event duration, occurrence counts, and trace maps. The generated API reference anchors that guide in concrete public classes such as the manager, the base handler contract, event objects, event types, and event payload names.
Sources: docs/api_reference/api_reference/callbacks/index.md
Core Callback Primitives
The core callback reference exposes the public classes and schema names that developers should recognize before choosing any particular integration. CallbackManager is the coordination object. BaseCallbackHandler is the extension point for handlers that react to events. CBEvent, CBEventType, and EventPayload are the schema-level names used to represent events and associated data. Those names matter because they form the vocabulary shared by local debugging handlers, token accounting handlers described in the official docs, and integration packages that forward LlamaIndex execution data to third-party systems.
A useful mental model is to treat callbacks as a structured event bus inside the framework rather than as a logging side effect. A query, retrieval, embedding batch, synthesis call, or text-splitting step can emit start and end information, and registered handlers can decide what to do with it. One handler may print a trace for a developer, another may count tokens, and another may create an external run record. Because the public API separates the manager from individual handlers, teams can add or remove destinations without rewriting retriever, index, or agent code.
Sources: docs/api_reference/api_reference/callbacks/index.md
Relevant Source Files
- docs/api_reference/api_reference/callbacks/index.md — defines the core callback API reference page for CallbackManager, BaseCallbackHandler, CBEvent, CBEventType, and EventPayload.
- docs/api_reference/api_reference/callbacks/agentops.md — defines the AgentOps callback integration reference entry for AgentOpsEventHandler.
- docs/api_reference/api_reference/callbacks/aim.md — defines the Aim callback integration reference entry for AimCallback.
- docs/api_reference/api_reference/callbacks/argilla.md — defines the Argilla callback integration reference entry for argilla_callback_handler.
- docs/api_reference/api_reference/callbacks/arize_phoenix.md — defines the Arize Phoenix callback integration reference entry for arize_phoenix_callback_handler.
- docs/api_reference/api_reference/callbacks/honeyhive.md — defines the HoneyHive callback integration reference entry for honeyhive_callback_handler.
Callback Event Model
The official documentation lists callback event types for chunking, node parsing, embedding, language model calls, queries, retrieval, synthesis, tree summarization, and sub-question handling. These categories map directly to common RAG workflow boundaries. For example, a bad answer may originate in chunking, retrieval, prompt construction, model output, or synthesis. Callback events let a handler observe those phases separately, which is more actionable than a single application-level log line. The API reference does not expand every event category in prose, but it exposes the schema names that handlers use to receive and interpret these events.
Developers should choose event handling depth based on the question they are trying to answer. During local debugging, a trace map and printed event tree may be enough to reveal that no nodes were retrieved or that a prompt contained unexpected context. During production evaluation, the same event boundaries can support latency measurement, count aggregation, and correlation with user feedback. During model or retriever experiments, handlers can preserve inputs and outputs so teams can compare runs. The callback layer therefore sits between application workflow code and observability destinations, translating internal phases into inspectable records.
Sources: docs/api_reference/api_reference/callbacks/index.md
System-to-Code Mapping
The callback API family has a small core and several optional integration entry points. The core page is the right reference when implementing a custom handler or when reading framework code that emits callback events. The integration pages are the right reference when wiring LlamaIndex to an external tool. Each integration page follows the same generated documentation pattern: it points at an importable callback module and lists the public member that the API reference intends to expose. That consistency is important because callback integrations should feel interchangeable from the application author’s perspective.
AgentOps is represented by AgentOpsEventHandler, which indicates an event-handler style integration for tracing or observing agent-oriented runs. Aim is represented by AimCallback, matching the official guide’s description of Aim support for tracking language model inputs and outputs. Argilla, Arize Phoenix, and HoneyHive are represented by callback handler entry points named for their integrations. The exact setup requirements for each external product live with the integration package and product documentation, but this API index tells readers which public LlamaIndex symbol to look for when adding the callback to an application.
Sources: docs/api_reference/api_reference/callbacks/agentops.md, docs/api_reference/api_reference/callbacks/aim.md, docs/api_reference/api_reference/callbacks/argilla.md, docs/api_reference/api_reference/callbacks/arize_phoenix.md, docs/api_reference/api_reference/callbacks/honeyhive.md
Integration Reference
| Integration page | Public member | Typical role |
|---|---|---|
| docs/api_reference/api_reference/callbacks/agentops.md | AgentOpsEventHandler | AgentOps event tracing and run observability |
| docs/api_reference/api_reference/callbacks/aim.md | AimCallback | Aim tracking for language model inputs and outputs |
| docs/api_reference/api_reference/callbacks/argilla.md | argilla_callback_handler | Argilla feedback or annotation-oriented callback handling |
| docs/api_reference/api_reference/callbacks/arize_phoenix.md | arize_phoenix_callback_handler | Arize Phoenix tracing and evaluation-oriented observability |
| docs/api_reference/api_reference/callbacks/honeyhive.md | honeyhive_callback_handler | HoneyHive experiment and feedback tracking |
Execution Flow
In a typical application, callbacks are configured before expensive or important framework operations run. The developer selects one or more handlers, registers them through the callback manager or project settings pattern used by the surrounding LlamaIndex code, and then executes ingestion, indexing, querying, chat, or agent workflows as usual. When those components reach instrumented boundaries, the manager notifies the handlers. Handlers can record the event payload, calculate timing, count occurrences, or forward data to a service. The application receives the same query response or workflow result, but it now has an audit trail for the path taken.
This flow becomes especially valuable when diagnosing RAG quality. If a user reports a poor answer, callback traces can help separate retrieval failure from synthesis failure. A retrieval event can show which nodes were selected, while a language model event can show the prompt and response shape, depending on handler behavior and configuration. A synthesis event can then show how retrieved context was combined into the final answer. Even when a callback integration sends the data elsewhere, the framework-level event boundaries give teams a consistent way to reason about root causes.
Sources: docs/api_reference/api_reference/callbacks/index.md
Implementation Guidance
When adding callbacks, start with the smallest handler set that answers the operational question. For a notebook or local reproduction, a debugging or token-counting handler described in the official docs may be enough. For a shared development environment, choose an integration that preserves traces across runs so team members can inspect the same execution. For production, consider what payloads may include prompts, retrieved context, or user data before forwarding events to any external service. The callback mechanism is powerful precisely because it can observe sensitive intermediate data, so privacy and retention choices belong in the integration plan.
Custom handlers should be designed around the BaseCallbackHandler contract rather than around a single workflow. The callback event vocabulary spans ingestion, retrieval, synthesis, model invocation, and query orchestration, so a handler may receive events that are not relevant to its destination. A robust handler should ignore unsupported event types gracefully, record enough context to reconstruct a trace, and avoid blocking the main application longer than necessary. If multiple handlers are registered, each should remain focused: one for human-readable debugging, one for accounting, one for external experiment tracking, and one for production observability if needed.
Sources: docs/api_reference/api_reference/callbacks/index.md
Choosing the Next Page
Read this page before the broader instrumentation page if the immediate task is callback-manager based observability inside LlamaIndex workflows. Move to instrumentation when you need lower-level spans, dispatchers, event handlers, or span handlers outside the callback API family. Move to evaluation pages when the goal is scoring answer quality rather than tracing execution. Move to agents, query engines, retrievers, or ingestion pipelines when the question is which application phase is producing the events. The callback reference is the bridge: it explains where to attach observers without changing the core behavior of those higher-level components.