Configurations and Model Outputs

Purpose and Scope

This page explains how Transformers connects three concepts that appear in nearly every workflow: configuration objects, pretrained checkpoints, and model outputs. A configuration object records architecture-level choices and default runtime flags; a checkpoint stores learned weights and usually includes or points to a compatible configuration; a model class implements the computation that consumes tensors and returns structured outputs. Understanding this relationship helps you decide when to load with from_pretrained, when to construct from a config, and how to read returned values without accidentally depending on tuple positions. Sources: docs/source/en/main_classes/model.md, docs/source/en/main_classes/text_generation.md

The public model documentation centers this relationship on PreTrainedModel, the base class that implements common loading and saving behavior for models from local files, local directories, or pretrained model artifacts. The same documentation also exposes shared helpers through ModuleUtilsMixin, Hub publishing through PushToHubMixin, and text-generation behavior through GenerationMixin. In practice, the model class owns the neural network, the config object describes how that model should be built and what it should return by default, and task-specific heads determine the shape and semantic meaning of fields such as logits, loss, and generated token scores. Sources: docs/source/en/main_classes/model.md, docs/source/ko/main_classes/model.md

Relevant Source Files

• docs/source/en/main_classes/model.md — English model API page for PreTrainedModel, shared model utilities, embedding resize behavior, Hub publishing, and the current PyTorch-oriented model reference surface.
• docs/source/ja/main_classes/model.md — Japanese model API page that preserves broader multilingual context for model loading, saving, large-model loading, device_map, dtype, and generation mixins.
• docs/source/ko/main_classes/model.md — Korean model API page covering PreTrainedModel, multilingual framework context, embedding resizing, attention-head pruning, and custom-model initialization guidance.
• docs/source/zh/main_classes/model.md — Chinese model API page with detailed large-model loading examples, explicit dtype loading patterns, device_map structure, and from_config usage.
• docs/source/en/main_classes/text_generation.md — English generation API page documenting GenerationConfig, GenerationMixin.generate, transition scores, and continuous batching-related generation classes.
• docs/source/ja/main_classes/text_generation.md — Japanese generation API page documenting generation mixins across frameworks and the core GenerationConfig and GenerationMixin autodoc surface.

System-to-Code Mapping

At the code and documentation boundary, PreTrainedModel.from_pretrained is the standard entry point for turning a model identifier, directory, or checkpoint into an initialized model instance. The model docs describe the base class as the common implementation for loading and saving, which is why higher-level APIs such as AutoModelForSequenceClassification.from_pretrained can share the same checkpoint mechanics across architectures. If you only have architecture parameters and do not want pretrained weights, the documented pattern is to load or create a config first and instantiate a model from that config, as shown in the Chinese model page with T5Config.from_pretrained("t5") followed by AutoModel.from_config(config). Sources: docs/source/en/main_classes/model.md, docs/source/zh/main_classes/model.md

Configurations are also where cross-cutting output defaults live. The official configuration reference names common flags such as output_hidden_states, output_attentions, and return_dict, and the model-output reference explains that model calls return subclasses of ModelOutput when dictionary-style returns are enabled. That means an individual forward pass can be controlled at two levels: persistent defaults stored on model.config, and ad hoc keyword arguments passed to the call. This is especially important when debugging internals, because hidden states and attentions are often omitted unless explicitly requested, and omitted fields appear as None rather than as empty tensors.

Generation adds a second configuration layer. The text generation API page documents GenerationConfig as the class used to parameterize generate, with methods such as from_pretrained, from_model_config, save_pretrained, update, validate, and get_generation_mode. The model configuration describes the architecture and default model behavior; the generation configuration describes decoding behavior such as how generation should proceed. Keeping those two roles separate avoids a common confusion: changing a model config affects model construction and forward defaults, while changing a generation config affects the policy used by GenerationMixin.generate. Sources: docs/source/en/main_classes/text_generation.md, docs/source/ja/main_classes/text_generation.md

Execution Flow

A typical inference flow starts by resolving a checkpoint and config, then instantiating a concrete model class, preparing inputs, running the model, and interpreting the returned ModelOutput. For a classifier, passing labels can cause the output object to include a loss in addition to logits; without labels, the same model usually returns logits and any optional diagnostics requested by flags. When treated like a tuple or dictionary, a ModelOutput only includes non-None fields, so code that indexes outputs should be written with care. Attribute access is usually clearer because it names the intended value, such as outputs.logits or outputs.hidden_states.

Large-model loading changes the resource plan but not the conceptual model/config/output contract. The Japanese and Chinese model pages document device_map="auto", where Accelerate chooses where layers should live, and expose the resulting placement through hf_device_map. They also document explicit layer-to-device dictionaries and lower-precision loading through dtype=torch.float16 or dtype="auto". These options influence how weights are materialized and placed, but the loaded object is still a pretrained model with a config, shared PreTrainedModel behavior, and normal forward or generation methods. Sources: docs/source/ja/main_classes/model.md, docs/source/zh/main_classes/model.md

The output contract also matters during generation. Calling generate uses GenerationMixin, and its behavior can be parameterized with a GenerationConfig. The English generation page points readers to the text generation strategies guide for inspecting defaults, changing parameters ad hoc, saving customized generation configurations, and using related features such as token streaming. The API page itself exposes compute_transition_scores, which is useful when you need to inspect token-level generation scores after decoding rather than only consume final generated sequences. Sources: docs/source/en/main_classes/text_generation.md

API Components

Component	Role	Source-backed entry points
PreTrainedModel	Base class for model loading, saving, and shared model behavior	from_pretrained, push_to_hub, model-specific forward implementations
ModuleUtilsMixin	Shared utilities common to model classes	Exposed through the model main class documentation
PushToHubMixin	Publishes model artifacts to the Hugging Face Hub	push_to_hub through the model docs
GenerationMixin	Adds text generation behavior to compatible models	generate, compute_transition_scores
GenerationConfig	Stores decoding and generation parameters	from_pretrained, from_model_config, save_pretrained, update, validate, get_generation_mode
ModelOutput subclasses	Structured return values from model calls	Attribute access, tuple-like access, dictionary-like access for non-None fields

Use PreTrainedModel and its auto-class wrappers when you need weights and architecture together from a checkpoint. Use a config-driven construction path when you need an architecture shell, custom initialization, or a controlled experiment that should not load pretrained weights. Use GenerationConfig when the question is about decoding policy rather than model architecture. Use returned ModelOutput attributes instead of positional indexes whenever you are writing reusable code, because optional fields can appear or disappear depending on labels, return_dict, output_hidden_states, and output_attentions.

Implementation Details and Edge Cases

Custom model authors should pay attention to the model documentation note about _supports_assign_param_buffer. The English and Korean pages state that custom models should include this flag to indicate whether superfast initialization can apply, and they connect failures in save/load tests to setting it to False. That detail is part of the model/config/checkpoint contract: a model class must be instantiable, saveable, and reloadable in the same framework-level path that regular pretrained models use. Sources: docs/source/en/main_classes/model.md, docs/source/ko/main_classes/model.md

One subtle output edge case is that the final hidden state exposed as last_hidden_state may not always be byte-for-byte identical to the final entry in hidden_states. The official model-output guidance notes that some architectures apply normalization or additional processing before returning the last hidden state. Treat hidden-state fields as semantic API fields, not as guaranteed aliases. If you are writing analysis code, request the fields you need explicitly and check whether each attribute is present before assuming it was computed.

Next Steps

If your immediate task is loading checkpoints, read the Auto Classes and Model Loading page next. If your task is decoding, continue to Text Generation and the Generation API, where GenerationConfig and GenerationMixin.generate are covered in more detail. If you are debugging memory use while loading large checkpoints, connect this page with the LLM Speed and Memory Optimization and Accelerate Integration pages, because device_map, dtype, and offloading affect resource behavior without changing the public model-output contract.