Instrumentation API

Purpose and Scope

The Instrumentation API page is the focused entrypoint for observing what LlamaIndex is doing at runtime. In this API family, instrumentation means a framework-level mechanism for reporting events and spans while application code executes. The official instrumentation index describes it as a simple framework that lets users observe events and spans happening in LlamaIndex, and then routes readers into four reference groups: event handlers, event types, span handlers, and span types. That split is important because it separates the things being observed from the code that receives or records those observations.

Sources: docs/api_reference/api_reference/instrumentation/index.md

Use this page when you are wiring observability into an application, building an integration that consumes framework telemetry, or trying to understand which typed events are emitted by agents, chat engines, LLM calls, embeddings, retrieval, query execution, and response synthesis. The API reference is intentionally narrow: it does not replace the higher-level observability guide, but it does identify the public reference surfaces that generated documentation exposes. In practice, a developer starts with event and span concepts, picks the handler extension point that matches the desired sink, and then maps runtime behavior to the typed event classes listed here.

Sources: docs/api_reference/api_reference/instrumentation/index.md, docs/api_reference/api_reference/instrumentation/event_types.md

Relevant Source Files

docs/api_reference/api_reference/instrumentation/index.md — Defines the Instrumentation API landing page and links the four subfamilies: event handlers, event types, span handlers, and span types.
docs/api_reference/api_reference/instrumentation/event_handlers.md — Generates the API reference for the event handler base class, the extension point for receiving instrumentation events.
docs/api_reference/api_reference/instrumentation/event_types.md — Generates the API reference for the concrete event classes emitted across agent, chat engine, embedding, LLM, query, retrieval, and synthesis workflows.
docs/api_reference/api_reference/instrumentation/span_handlers.md — Generates the API reference for span handler classes, including the base handler and the simple handler implementation.
docs/api_reference/api_reference/instrumentation/span_types.md — Generates the API reference for the base span type used by span-oriented instrumentation.

Core Primitives

There are two observation primitives in this API surface: events and spans. An event is a typed record that marks a meaningful point in execution, such as the start or end of a query, a retrieval operation, an LLM completion, or a streaming chat chunk. A span represents a duration-oriented unit of work, useful when the important question is not only that something happened but how an operation is bounded. The documentation index groups both concepts together because most observability systems need both point-in-time signals and structured timing boundaries.

Sources: docs/api_reference/api_reference/instrumentation/index.md, docs/api_reference/api_reference/instrumentation/event_types.md, docs/api_reference/api_reference/instrumentation/span_types.md

Handlers are the receiving side of this design. The event handler reference exposes BaseEventHandler, while the span handler reference exposes BaseSpanHandler and SimpleSpanHandler. Even without expanding implementation details, the names define the public contract shape: custom event handling begins by conforming to the base event handler, custom span handling begins from the base span handler, and the simple span handler is the reference implementation surfaced by the generated API docs. This is the API layer to inspect before forwarding instrumentation data to logs, traces, metrics, dashboards, or test assertions.

Sources: docs/api_reference/api_reference/instrumentation/event_handlers.md, docs/api_reference/api_reference/instrumentation/span_handlers.md

The generated reference pages use mkdocstrings directives, which means these Markdown files are not hand-written class manuals; they are documentation entrypoints that point the docs build at importable Python objects. That matters for contributors because changing the public class path or object name affects whether the reference page resolves correctly. It also matters for readers because the authoritative details for methods, fields, and inheritance are presented through the generated API output for each directive rather than through prose embedded directly in these Markdown files.

Sources: docs/api_reference/api_reference/instrumentation/event_handlers.md, docs/api_reference/api_reference/instrumentation/event_types.md, docs/api_reference/api_reference/instrumentation/span_handlers.md, docs/api_reference/api_reference/instrumentation/span_types.md

API Components

The event type reference starts with BaseEvent, the common event abstraction, and then enumerates domain-specific subclasses. Agent instrumentation includes events for chat-with-step boundaries, run-step boundaries, and tool calls. The concrete names are AgentChatWithStepStartEvent, AgentChatWithStepEndEvent, AgentRunStepStartEvent, AgentRunStepEndEvent, and AgentToolCallEvent. Together, these names show that the agent runtime can be observed at both conversational and tool-execution levels, which is especially useful when debugging multi-step plans, tool selection, or intermediate agent state transitions.

Sources: docs/api_reference/api_reference/instrumentation/event_types.md

Chat engine streaming has its own event family: StreamChatStartEvent, StreamChatDeltaReceivedEvent, StreamChatEndEvent, and StreamChatErrorEvent. The separation between start, delta, end, and error events gives consumers enough structure to model a streamed response as a lifecycle rather than a single opaque message. A client can correlate the beginning of a streamed chat request, each received delta, successful completion, and exceptional termination. This is the reference family to inspect when building a UI, gateway, or logger that must preserve incremental chat output.

Sources: docs/api_reference/api_reference/instrumentation/event_types.md

LLM and embedding operations are represented by paired start and end events. Embeddings expose EmbeddingStartEvent and EmbeddingEndEvent. LLM chat exposes LLMChatStartEvent and LLMChatEndEvent; completions expose LLMCompletionStartEvent and LLMCompletionEndEvent; prediction exposes LLMPredictStartEvent and LLMPredictEndEvent. The paired naming convention is a practical signal for observability implementers: these events are designed to bracket calls that often have latency, token usage, model selection, or provider behavior that developers want to inspect.

Sources: docs/api_reference/api_reference/instrumentation/event_types.md

Retrieval and query events mark the RAG execution path. QueryStartEvent and QueryEndEvent identify the outer query lifecycle, while RetrievalStartEvent and RetrievalEndEvent identify the retrieval phase inside that lifecycle. Response synthesis is represented by SynthesizeStartEvent, SynthesizeEndEvent, GetResponseStartEvent, and GetResponseEndEvent. This gives the API reference a useful operational map: a RAG request can be observed from user query entry, through retrieval, into synthesis, and finally to response construction.

Sources: docs/api_reference/api_reference/instrumentation/event_types.md

Compact Reference

Reference area	Public objects surfaced by the docs	Reader task
Event handlers	`llama_index_instrumentation.event_handlers.base.BaseEventHandler`	Implement or inspect handlers that receive event objects.
Event types	`llama_index.core.instrumentation.events.base.BaseEvent` and concrete event classes	Identify which runtime signals are emitted for agents, chat, embeddings, LLMs, queries, retrieval, and synthesis.
Span handlers	`llama_index_instrumentation.span_handlers.base.BaseSpanHandler`, `llama_index_instrumentation.span_handlers.simple.SimpleSpanHandler`	Handle duration-oriented instrumentation spans.
Span types	`llama_index_instrumentation.span.base.BaseSpan`	Understand the base representation for span data.

The compact reference shows a deliberate package boundary. Event classes in the generated docs are listed under llama_index.core.instrumentation.events, while handler and span base objects are surfaced from llama_index_instrumentation. For application developers, that means the event vocabulary is part of the core framework’s instrumentation namespace, while handler-oriented extension points live in the instrumentation package namespace. The docs do not require the reader to infer this layout from source files; the import paths are visible directly in the generated reference directives.

System-to-Code Mapping

The index file is the navigation contract for this API family. It promises that instrumentation is about observing events and spans, then exposes the four child pages as the supported reference structure. The event handler page maps to the event consumer extension point. The event types page maps to the event vocabulary emitted by framework subsystems. The span handler page maps to span consumers, including a simple implementation. The span types page maps to the base representation for spans. This organization keeps the API reference discoverable even as individual event families grow.

Sources: docs/api_reference/api_reference/instrumentation/index.md, docs/api_reference/api_reference/instrumentation/event_handlers.md, docs/api_reference/api_reference/instrumentation/event_types.md, docs/api_reference/api_reference/instrumentation/span_handlers.md, docs/api_reference/api_reference/instrumentation/span_types.md

A useful way to read the event catalog is by runtime layer. At the top are user-facing orchestration layers such as agents, chat engines, and query engines. Beneath them are model and retrieval layers, represented by LLM, embedding, and retrieval events. Finally, synthesis events describe the final answer-building phase. This ordering is not just taxonomy; it helps when instrumenting a full RAG or agent application. You can decide whether to observe only high-level request boundaries or collect lower-level events for latency analysis and troubleshooting.

Sources: docs/api_reference/api_reference/instrumentation/event_types.md

The span API completes that picture by covering operations that are naturally interval based. While event names often come in start and end pairs, spans provide a first-class type for representing bounded work. Span handlers therefore sit beside event handlers rather than replacing them. A production integration might use events for semantically rich milestones and spans for trace structure. A lightweight debug tool might start with the simple span handler, then add a custom event handler only for the event families it needs to inspect.

Sources: docs/api_reference/api_reference/instrumentation/span_handlers.md, docs/api_reference/api_reference/instrumentation/span_types.md

Execution and Integration Flow

A typical instrumentation integration begins by choosing the signal level. If you need to react to specific framework milestones, start with event types and the event handler base class. If you need duration or trace-like structure, start with span types and span handlers. Next, map the application subsystem you care about to the concrete event family: agents for tool calls and agent steps, chat streaming for incremental output, LLM and embedding for model operations, retrieval and query for RAG execution, and synthesis for final response construction.

For debugging, begin narrowly. A query latency investigation can follow QueryStartEvent, RetrievalStartEvent, RetrievalEndEvent, SynthesizeStartEvent, and QueryEndEvent before expanding into LLM or embedding events. A streaming UI problem can focus on the stream chat start, delta, end, and error events. An agent tool issue can start with AgentToolCallEvent and the agent step boundary events. The API reference gives you the vocabulary for this selective approach, which is usually easier to operate than capturing every possible signal immediately.

Sources: docs/api_reference/api_reference/instrumentation/event_types.md

Testing Signals and Next Steps

Instrumentation APIs are also useful in tests because they expose observable behavior without requiring a test to depend only on final text output. For example, a test can assert that retrieval was attempted, that synthesis completed, or that a streaming chat path emitted deltas before ending. The supplied reference pages do not define testing helpers directly, but the typed event names make test intent clearer than generic log matching. For contributors, adding a new runtime feature should usually include thinking about whether it needs a new event, a span boundary, or both.

Sources: docs/api_reference/api_reference/instrumentation/event_types.md, docs/api_reference/api_reference/instrumentation/span_types.md

After this page, read the broader instrumentation and callbacks documentation if you need setup guidance, tracing behavior, or integrations with observability systems. Read the agent, chat engine, query engine, retrieval, and response synthesis pages when you need to interpret a specific event in its runtime context. If you are extending the API reference itself, keep the same pattern used here: expose importable objects through focused mkdocstrings directives, group them by developer task, and preserve the distinction between event vocabulary, event handlers, span vocabulary, and span handlers.

Ask a question...