Metadata Extraction

Purpose and Scope

Metadata extraction in LlamaIndex is the process of turning source content into predictable fields that downstream code can validate, store, retrieve, or display. The framework documentation places this under the broader label “Structured Data Extraction,” which is the conversion of unstructured human language into structured data while ignoring irrelevant text. That framing is important: title, summary, keyword, entity, question, Pydantic, and document-context extraction are all variations of the same application problem. You are asking an LLM or parser to read source material and return a constrained shape instead of an open-ended answer.

Sources: docs/src/content/docs/framework/understanding/extraction/_meta.yml, docs/src/content/docs/framework/understanding/extraction/index.md

The documented center of this workflow is Pydantic. You define a BaseModel that describes the fields your application expects, and LlamaIndex uses that model to guide the LLM and coerce the result into a validated Python object. This applies naturally to invoice extraction examples in the docs, but the same pattern can represent metadata attached to documents and nodes: a generated title, a short summary, extracted keywords, recognized entities, synthetic questions, or a richer document-context object. The practical goal is to make generated metadata usable by code, not merely readable by humans.

Sources: docs/src/content/docs/framework/understanding/extraction/index.md

Relevant Source Files

docs/src/content/docs/framework/understanding/extraction/_meta.yml — Places these pages in the documentation navigation as the “Structured Data Extraction” section and records the ordering/collapsed state for the guide group.
docs/src/content/docs/framework/understanding/extraction/index.md — Introduces extraction, unstructured versus structured data, Pydantic models, JSON schema generation, and annotation practices with Field descriptions and docstrings.
docs/src/content/docs/framework/understanding/extraction/structured_llms.md — Shows the highest-level extraction path with llm.as_structured_llm(...), complete, chat, streaming methods, and query-engine usage with a structured LLM.
docs/src/content/docs/framework/understanding/extraction/structured_prediction.md — Documents structured_predict, its async and streaming variants, and the internal split between FunctionCallingProgram and LLMTextCompletionProgram.
docs/src/content/docs/framework/understanding/extraction/lower_level.md — Shows lower-level extraction with get_function_tool, chat_with_tools, tool-call parsing, multiple extracted objects, and manual JSON parsing with model_validate_json.
docs/src/content/docs/framework/understanding/extraction/structured_input.md — Covers structured inputs using RichPromptTemplate, Jinja, the to_xml filter, and combining structured input with structured output.

Core Extraction Model

The first design decision is what shape the extracted metadata should have. In the documentation, simple Pydantic classes such as User demonstrate required and default fields, while nested examples such as Foo, Bar, and Spam show that extraction schemas can contain nested objects and lists. For metadata extraction, that means you can start with a single field such as title: str, then evolve toward richer models with summary, keywords, entities, or questions. Pydantic type declarations provide the contract that both the LLM prompt and your application code can share.

Sources: docs/src/content/docs/framework/understanding/extraction/index.md

Pydantic’s JSON schema support is the bridge between Python types and LLM instructions. The docs show that a model such as User serializes into an object schema with properties, required fields, defaults, titles, and primitive types. LlamaIndex uses this kind of schema because LLMs can treat it as a precise description of how output should be formatted. When you design metadata extractors, field names and types are not cosmetic; they become operational guidance for the model and validation criteria for the returned object.

Sources: docs/src/content/docs/framework/understanding/extraction/index.md

Annotations make extraction more reliable. The documentation recommends docstrings and Field(description=...) values because they expand the generated schema with natural-language explanations. For example, the invoice model describes a line item, an invoice identifier, a creation date, and a list of items. In metadata extraction, similar descriptions should explain whether a title should be concise, whether a summary should be abstractive or extractive, what qualifies as a keyword, or which entity categories matter. These descriptions reduce ambiguity without requiring a separate, fragile prompt for every field.

Sources: docs/src/content/docs/framework/understanding/extraction/index.md

High-Level APIs

The highest-level API is a structured LLM. The docs instantiate an OpenAI LLM, call llm.as_structured_llm(Invoice), and then call complete(text) on the resulting object. The response is still a LlamaIndex response object, but it has two useful representations: text, containing JSON-serialized output, and raw, containing the Pydantic object itself. This pattern is a good fit when the extraction schema is stable and you want to reuse an LLM-like object throughout an application, including in RAG query engines.

Sources: docs/src/content/docs/framework/understanding/extraction/structured_llms.md

A structured LLM is intentionally presented as behaving like a regular LLM class. The guide says it supports calls such as chat, stream, achat, and astream, and it can be passed as llm=sllm to VectorStoreIndex.as_query_engine(...) so RAG answers are returned as structured objects. For metadata extraction, that means the same schema can be used in batch document processing, interactive chat, streaming interfaces, or query-time extraction. The abstraction lets application code focus on the schema and handling of validated results rather than duplicating prompting mechanics.

Sources: docs/src/content/docs/framework/understanding/extraction/structured_llms.md

Structured Prediction and Control

When you need tighter prompt control, the docs move from structured LLMs to structured_predict. This method is available on every LLM class and takes a Pydantic class, a PromptTemplate, and keyword arguments for template variables. The invoice example adds a fallback rule for constructing an invoice identifier when no ID is found. The equivalent metadata use case is common: a title extractor might require a maximum length, a keyword extractor might prefer domain vocabulary, or a document-context extractor might include source-specific business rules that cannot be inferred from field names alone.

Sources: docs/src/content/docs/framework/understanding/extraction/structured_prediction.md

The structured_predict response is the Pydantic object itself, and the docs show converting it with model_dump_json() when JSON output is needed. The same page names async and streaming variants: astructured_predict, stream_structured_predict, and astream_structured_predict. These variants matter when extraction is part of a larger ingestion or user-facing workflow. Batch metadata enrichment may use async calls for throughput, while an interactive review UI may stream intermediate structured output. In all cases, your schema remains the stable contract between LlamaIndex and the rest of the system.

Sources: docs/src/content/docs/framework/understanding/extraction/structured_prediction.md

Under the hood, LlamaIndex chooses between two program styles. If the LLM supports function calling, FunctionCallingProgram converts the Pydantic object into a tool, prompts the LLM while forcing tool use, and returns the generated Pydantic object. If the LLM is text-only, LLMTextCompletionProgram emits the JSON schema, asks the model to respond according to that schema, and validates the raw text with model_validate_json(). The docs explicitly describe the function-calling path as generally more reliable, while the text-completion path is more broadly supported.

Sources: docs/src/content/docs/framework/understanding/extraction/structured_prediction.md

Lower-Level and Structured Input Flows

For direct control, the lower-level guide shows using get_function_tool(Invoice) and calling llm.chat_with_tools(...) with tool_required=True. The application then reads tool calls with llm.get_tool_calls_from_response(...), checks the tool name, and constructs Pydantic objects from tool_kwargs. The same page demonstrates allow_parallel_tool_calls=True with a LineItem schema to extract multiple objects from one input. That is the pattern to use when a document may contain many entities, questions, action items, or repeated metadata records.

Sources: docs/src/content/docs/framework/understanding/extraction/lower_level.md

The lowest-level fallback is direct prompting. The docs build a prompt from Invoice.model_json_schema(), instruct the model to output only a JSON object, call llm.complete(prompt), and parse with Invoice.model_validate_json(response.text). This is useful when you need complete prompt ownership or when a model/provider combination does not fit the higher-level abstractions. It is also the least protected path: the application owns prompt wording, markdown suppression, malformed JSON handling, and any repair strategy before Pydantic validation succeeds.

Sources: docs/src/content/docs/framework/understanding/extraction/lower_level.md

Structured input complements structured output. The structured input guide uses RichPromptTemplate, Jinja syntax, and the to_xml filter to format a Pydantic User object as XML inside the prompt. It then combines that input formatting with llm.as_structured_llm(ContactDetails) and awaits sllm.achat(...) to produce typed contact details. For metadata extraction, structured input is useful when the source material already has fields, sections, or records and you want the model to preserve that organization while producing a validated output object.

Sources: docs/src/content/docs/framework/understanding/extraction/structured_input.md

Compact API Reference

Task	Public surface shown in docs	What it returns or enables
Define an extraction schema	`class X(BaseModel): ...` with `Field(description=...)`	A typed Pydantic contract and JSON schema for LLM guidance
Inspect schema	`model_json_schema()`	JSON schema used in prompts or direct extraction flows
High-level extraction	`llm.as_structured_llm(OutputModel)`	LLM-like object that returns structured responses
Structured completion/chat	`sllm.complete`, `sllm.chat`, `sllm.stream`, `sllm.achat`, `sllm.astream`	Response objects with JSON `text` and Pydantic `raw` where applicable
Prompt-controlled extraction	`llm.structured_predict(OutputModel, prompt, **kwargs)`	A Pydantic object
Async and streaming prediction	`astructured_predict`, `stream_structured_predict`, `astream_structured_predict`	Async or streamed structured prediction variants
Function-calling internals	`FunctionCallingProgram`	Converts the Pydantic schema into a tool and returns validated output
Text-only internals	`LLMTextCompletionProgram`	Prompts with schema text and validates with Pydantic
Custom parsing	`PydanticOutputParser` subclass	Custom control over turning model text into a Pydantic object
Direct tool use	`get_function_tool`, `chat_with_tools`, `get_tool_calls_from_response`	Lower-level access to tool calls and multiple extracted objects
Structured input	`RichPromptTemplate` with `{{ value	to_xml }}`

Sources: docs/src/content/docs/framework/understanding/extraction/structured_llms.md, docs/src/content/docs/framework/understanding/extraction/structured_prediction.md, docs/src/content/docs/framework/understanding/extraction/lower_level.md, docs/src/content/docs/framework/understanding/extraction/structured_input.md

Implementation Guidance

Treat metadata extraction as schema design first and prompting second. Start with the smallest Pydantic model your index or application needs, annotate each field with the interpretation you expect, and test whether the generated object is valid and semantically useful. If the default structured LLM behavior is sufficient, use as_structured_llm because it keeps extraction calls close to normal LLM usage. If field-level interpretation requires business rules, use structured_predict with a PromptTemplate so the rules live beside the schema.

Sources: docs/src/content/docs/framework/understanding/extraction/structured_llms.md, docs/src/content/docs/framework/understanding/extraction/structured_prediction.md

Choose the lower-level APIs only when they solve a specific problem. Tool calling is appropriate when the model can call one or more extraction tools, especially for repeated records such as entities or line items. Direct prompting is appropriate when you must control every instruction or integrate with a text-only model path, but it shifts parsing risk to your code. Structured input is a separate lever: use XML-formatted inputs when the source object already has structure and you want the model to understand that structure before producing typed output.

Sources: docs/src/content/docs/framework/understanding/extraction/lower_level.md, docs/src/content/docs/framework/understanding/extraction/structured_input.md

A practical next step is to implement one extractor schema for the metadata that most improves retrieval quality, such as a document summary or list of keywords, and run it against a small representative document set. Inspect both the JSON text and the Pydantic object, refine Field descriptions, then decide whether the extraction belongs in ingestion, query-time response formatting, or an agent/tool workflow. From here, read the adjacent pages on documents and nodes, ingestion pipelines, node parsers, vector store indexing, and response synthesis to see where the extracted fields should be attached and consumed.

Ask a question...