Language Models

Purpose and Scope

Language models are the runtime boundary between LangChain applications and provider-hosted AI systems. In LangChain Core, that boundary is deliberately split into two related abstractions: chat models, which consume and produce structured messages, and legacy LLMs, which are fundamentally string-in and string-out. The package documentation in langchain_core.language_models presents chat models as the primary modern abstraction while still preserving a common interface for older text completion models. This lets application code treat model calls as runnables, compose them with prompts and tools, and swap provider integrations without rewriting the surrounding chain or agent logic.

Sources: libs/core/langchain_core/language_models/init.py, libs/core/langchain_core/language_models/base.py

The most important practical distinction is the input and output shape. A chat model receives a sequence of role-bearing messages and returns a chat message, so system instructions, human turns, AI turns, and tool results remain distinct. A legacy LLM receives a string and returns a string, but LangChain wrappers can still accept messages by formatting them into text before calling the underlying provider. That compatibility layer is why the shared LanguageModelInput, LanguageModelOutput, and LanguageModelLike aliases matter: they describe the common model-facing contract that other LangChain components can depend on.

Sources: libs/core/langchain_core/language_models/init.py, libs/core/langchain_core/language_models/base.py

Relevant Source Files

libs/core/langchain_core/language_models/__init__.py — defines the public package surface, documents the chat-model versus LLM distinction, and dynamically exports model classes, fake models, model profiles, tokenizer utilities, and content-block helpers.
libs/core/langchain_core/language_models/_compat_bridge.py — converts AIMessageChunk and finalized AIMessage objects into protocol events for streaming and replay scenarios.
libs/core/langchain_core/language_models/_utils.py — contains tracing and multimodal utility helpers, including OpenAI-format data block detection and invocation-parameter filtering.
libs/core/langchain_core/language_models/base.py — defines shared base-model typing, LangSmith trace metadata, tokenizer fallback behavior, and the common language-model input/output aliases.
libs/core/langchain_core/language_models/chat_model_stream.py — implements per-message stream objects for v3 event streams with accumulated projections such as text, reasoning, tool calls, usage, and output.
libs/core/langchain_core/language_models/chat_models.py — provides the BaseChatModel and SimpleChatModel chat-model contracts used by provider integrations.

Core Abstractions

BaseLanguageModel is the shared conceptual base for model implementations, while BaseChatModel is the central abstraction for modern chat providers. The package entry point exposes both, along with BaseLLM and LLM for legacy completion models, so downstream integrations can import stable names from langchain_core.language_models instead of reaching into implementation modules. The public surface also includes fake language models and fake chat models, which are useful for tests and examples because they satisfy the same interfaces without making provider API calls.

Sources: libs/core/langchain_core/language_models/init.py, libs/core/langchain_core/language_models/chat_models.py

The shared input type is intentionally broad: a language model can be invoked with a PromptValue, a raw string, or a sequence of message-like representations. The output type is correspondingly either a BaseMessage for chat-like responses or a string for LLM-like responses. Treating LanguageModelLike as a Runnable[LanguageModelInput, LanguageModelOutput] is the key architectural choice: models participate in the same runnable lifecycle as prompt templates, parsers, retrievers, and composed chains. That gives applications consistent entry points for invocation, batching, streaming, callbacks, and configuration.

Sources: libs/core/langchain_core/language_models/base.py

LangSmith tracing metadata is modeled explicitly through LangSmithParams. The fields describe provider, model name, model type, temperature, maximum tokens, stop words, and integration name. Provider implementations can use those fields to make traces searchable and comparable without stuffing the trace payload with oversized request data. The utility helper _filter_invocation_params_for_tracing reinforces this boundary by removing fields such as tools, functions, messages, and response_format, which can be large or inappropriate for compact invocation metadata.

Sources: libs/core/langchain_core/language_models/base.py, libs/core/langchain_core/language_models/_utils.py

Provider Implementation Contract

A provider integration should implement the standard LangChain model interface for its backend while preserving provider-specific capabilities behind the common abstraction. Official LangChain docs describe model selection in provider-prefixed form, such as openai:gpt-5.5 or anthropic:claude-opus-4-8, and the Python repository mirrors that goal at the core layer by defining stable base classes and common runnable-compatible types. Provider packages can expose extra constructor options, but their chat model classes still need to behave like LangChain chat models when invoked, streamed, traced, or composed.

Sources: libs/core/langchain_core/language_models/init.py, libs/core/langchain_core/language_models/chat_models.py

The package also exposes ModelProfile and ModelProfileRegistry from its public namespace. Those names indicate that model capability data is part of the language-model developer surface, even though provider integrations own the concrete capability records. In practice, a profile is how higher-level code can reason about model behavior such as tool calling, structured output, context size, or multimodal support without hard-coding every provider’s catalog. For implementers, that means the base interface is only one part of the contract; accurate capability metadata is also important for agent and application behavior.

Sources: libs/core/langchain_core/language_models/init.py

Token counting is another area where implementations should prefer provider- or model-specific behavior when available. The shared fallback in base.py uses a cached GPT-2 tokenizer and warns that counts may be inaccurate for non-GPT-2 models. That fallback is valuable because it gives the core package a default get_token_ids path, but the warning is a design signal: production integrations should supply model-aware tokenization when exact budgeting matters for context windows, truncation, pricing, or prompt assembly.

Sources: libs/core/langchain_core/language_models/base.py

Streaming and Event Flow

LangChain chat streaming has two layers. At the lower layer, providers emit AIMessageChunk values, often in server-sent-event style where each chunk represents a delta rather than accumulated state. The compatibility bridge converts those chunks into a protocol lifecycle: message-start, content-block-start, repeated content-block-delta events, content-block-finish, and finally message-finish. The bridge trusts each chunk’s content_blocks property as the single protocol view, then accumulates per-index state only when it needs to finalize blocks such as tool-call chunks.

Sources: libs/core/langchain_core/language_models/_compat_bridge.py

At the user-facing layer, ChatModelStream and AsyncChatModelStream provide typed projections over that event stream. The stream object can be iterated directly for raw events, but it also exposes accumulated views such as .text, .reasoning, .tool_calls, .usage, and .output. This design solves a common application problem: different consumers may need different levels of detail from the same model call. A UI might render text deltas immediately, an observability sink might record usage, and an agent runtime might wait for finalized tool calls.

Sources: libs/core/langchain_core/language_models/chat_model_stream.py, libs/core/langchain_core/language_models/_compat_bridge.py

Tool-call streaming has special merge behavior because providers may split the call identifier, function name, and argument JSON across multiple chunks. The stream helpers keep sticky values for identifiers and names while concatenating argument fragments, then sweep indexed stores into finalized tool-call blocks. The bridge also supports replaying a finalized AIMessage as a synthetic lifecycle, which is useful for cache hits, checkpoint restores, or graph nodes that return a completed message but still need to feed a client expecting event semantics.

Sources: libs/core/langchain_core/language_models/chat_model_stream.py, libs/core/langchain_core/language_models/_compat_bridge.py

Multimodal Content and Compatibility Utilities

Modern chat providers do not only exchange plain text. The utility function is_openai_data_block recognizes OpenAI Chat Completions-style multimodal data blocks for images, audio, and files. It accepts both data-backed and ID-backed file blocks, checks the required nested fields, and supports an optional filter for matching only one modality. This helper is exported from the package namespace, so integration and message-conversion code can reuse the same validation behavior when translating content blocks between provider formats and LangChain’s common message representation.

Sources: libs/core/langchain_core/language_models/init.py, libs/core/langchain_core/language_models/_utils.py

The compatibility bridge exists because LangChain Core message content blocks and wire-protocol content blocks are structurally similar but represented by different type unions. Internally, the bridge launders blocks through dictionaries at a narrow seam, then casts them to protocol content or finalized content when emitting events. That detail matters for contributors: compatibility code should keep provider translation, event production, and type-system workarounds localized instead of spreading casts through model implementations or stream consumers.

Sources: libs/core/langchain_core/language_models/_compat_bridge.py

Compact API Reference

Name	Kind	Contract
`BaseChatModel`	class	Primary base class for chat model integrations that accept messages and return chat messages.
`SimpleChatModel`	class	Convenience chat-model base exported from the chat models module.
`BaseLanguageModel`	class	Shared language-model base used by the common model surface.
`BaseLLM` / `LLM`	classes	Legacy string-in/string-out model abstractions exposed alongside chat models.
`LanguageModelInput`	type alias	`PromptValue`, `str`, or a sequence of message-like representations.
`LanguageModelOutput`	type alias	`BaseMessage` or `str`.
`LanguageModelLike`	type alias	Runnable-compatible model interface from language-model input to language-model output.
`LangSmithParams`	typed dict	Trace metadata for provider, model name, model type, temperature, max tokens, stop words, and integration.
`get_tokenizer`	function	Cached GPT-2 tokenizer fallback used when transformers is installed.
`is_openai_data_block`	function	Validates OpenAI-format image, audio, or file content blocks.
`ChatModelStream` / `AsyncChatModelStream`	stream objects	Event-stream wrappers with typed projections for text, reasoning, tool calls, usage, and output.

Next Steps

When implementing or selecting a provider model, start from the chat-model contract rather than a provider-specific API shape. Confirm that invocation accepts LangChain message inputs, that tracing metadata is compact and meaningful, that streaming produces coherent content-block events, and that capability information is exposed through profiles when available. Application developers should then compose models through the runnable interface and rely on higher-level primitives such as tools, structured output, and agents only after the model’s basic invoke and stream behavior is verified.

Sources: libs/core/langchain_core/language_models/base.py, libs/core/langchain_core/language_models/chat_models.py, libs/core/langchain_core/language_models/chat_model_stream.py

Ask a question...