Trainer Features and Callbacks

Purpose and Scope

Trainer features are the extension points around the PyTorch Trainer training loop: callbacks for observing and steering loop events, custom loss hooks for changing loss calculation, reporting integrations for experiment tracking, and lower-level subclassing when a recipe needs to alter the loop itself. This page focuses on the callback and feature surface documented under main_classes, not on the complete Trainer constructor reference. The key design boundary is that callbacks can inspect TrainingArguments, TrainerState, and keyword objects passed by the loop, but they are intentionally read-only except for returning changes through TrainerControl. Sources: docs/source/en/main_classes/callback.md

Use this page when you already have a Trainer run and need to add behavior without rewriting the training script. Typical tasks include printing custom progress messages, logging to platforms such as TensorBoard or Weights & Biases, triggering evaluation, using early stopping, or coordinating checkpoint behavior. If the behavior requires changing the forward pass, optimizer construction, batch preparation, or how loss is computed before model outputs exist, use the dedicated customization or subclassing path instead of a callback. Sources: docs/source/en/main_classes/callback.md

Relevant Source Files

• docs/source/en/main_classes/callback.md - English callback reference page; defines callback purpose, read-only constraint, available callback classes, and registration examples.
• docs/source/ja/main_classes/callback.md - Japanese localized callback page; preserves the same callback model and lists reporting integrations and report_to behavior.
• docs/source/ko/main_classes/callback.md - Korean localized callback page; describes callback state inspection, control decisions, default flow behavior, and installed integration callbacks.
• docs/source/zh/main_classes/callback.md - Chinese localized callback page; documents callback registration, available integrations, and TrainerState and TrainerControl sections.
• docs/source/en/main_classes/backbones.md - Main-classes documentation page for vision backbones; useful context for feature extraction components that may appear in Trainer-based vision workflows.
• docs/source/en/main_classes/configuration.md - Main-classes configuration reference; explains PreTrainedConfig, common model attributes, and saving/loading configuration used by trained checkpoints.

Callback Model and Control Flow

A TrainerCallback is an object whose methods are called at specific training events. The official callback guide frames these events around training begin and end, epoch boundaries, step boundaries, optimizer steps, logging, evaluation, saving, prediction, and Hub push events. In practical terms, callbacks are best for reacting to facts the loop already knows: the current global step, the current epoch, the latest metrics, the learning rate scheduler state, or whether training should stop early. This makes them a good fit for orchestration and observability rather than model math. Sources: docs/source/en/main_classes/callback.md

The callback API has three central data objects. TrainingArguments represents static or mostly static run configuration created before training starts. TrainerState exposes live loop state such as progress, best metric information, and other values maintained by the trainer. TrainerControl is the sanctioned way for a callback to request actions, such as changing whether logging, evaluation, saving, or stopping should happen. The source documentation emphasizes that callbacks cannot otherwise mutate the training loop, which keeps stacked callbacks predictable when multiple integrations are enabled together. Sources: docs/source/en/main_classes/callback.md

The localized callback pages are useful because they confirm that this callback contract is part of the public documentation surface across languages, not just an implementation detail in English. They also preserve older explanatory notes about default reporting integrations when packages are installed, and about selecting integrations explicitly with TrainingArguments.report_to. When maintaining docs or examples, treat the English page as canonical for current wording, but check localized pages before removing concepts that are still important for international readers. Sources: docs/source/ja/main_classes/callback.md, docs/source/ko/main_classes/callback.md, docs/source/zh/main_classes/callback.md

Built-in Callbacks and Reporting Integrations

Transformers documents a set of built-in callbacks that cover the default flow and common tracking systems. DefaultFlowCallback manages the ordinary logging, saving, and evaluation cadence. PrinterCallback and ProgressCallback display progress, with progress bar behavior depending on training arguments. EarlyStoppingCallback implements a common training-control policy. Integration callbacks include TensorBoardCallback, TrackioCallback, WandbCallback, MLflowCallback, CometCallback, AzureMLCallback, CodeCarbonCallback, ClearMLCallback, DagsHubCallback, FlyteCallback, KubeflowCallback, DVCLiveCallback, and SwanLabCallback. Sources: docs/source/en/main_classes/callback.md

TrainingArguments.report_to is the switch that determines which reporting integrations the trainer uses. The English callback page states that the default is "none", while localized pages still describe a historical or localized default of "all" and explain how installed packages can activate integrations. For current examples, prefer the English source’s default and make the desired reporting target explicit in scripts. For example, pass only the integrations you want when a machine has many optional packages installed, so training logs do not unexpectedly go to a platform you did not intend to use. Sources: docs/source/en/main_classes/callback.md, docs/source/ja/main_classes/callback.md, docs/source/ko/main_classes/callback.md, docs/source/zh/main_classes/callback.md

Custom Callback Pattern

A custom callback subclasses TrainerCallback and overrides one or more event methods. The repository documentation shows the smallest useful form: implement on_train_begin, receive args, state, control, and **kwargs, then register the callback either in the Trainer(..., callbacks=[...]) constructor argument or later with trainer.add_callback(...). Passing a class or an already-created instance is supported in the documented examples, which is convenient when a callback has constructor arguments such as thresholds, file paths, or tracker handles. Sources: docs/source/en/main_classes/callback.md

from transformers import Trainer, TrainerCallback
 
class MyCallback(TrainerCallback):
    "A callback that prints a message at the beginning of training"
 
    def on_train_begin(self, args, state, control, **kwargs):
        print("Starting training")
 
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    callbacks=[MyCallback],
 )
 
trainer.add_callback(MyCallback())

When writing a callback, decide first whether it observes, reports, or controls. Observing callbacks read state and print or store information. Reporting callbacks bridge trainer metrics to an external service. Control callbacks set fields on TrainerControl, such as requesting evaluation after a scheduler threshold or stopping when a metric fails to improve. Avoid hiding training-loop mutations inside callbacks; if your desired feature changes batch inputs, labels, model outputs, or optimizer stepping, move that logic into a custom Trainer subclass or the documented trainer feature hook that owns that behavior. Sources: docs/source/en/main_classes/callback.md

Recipes, Loss Hooks, and Scheduler-Aware Behavior

The Trainer feature recipes distinguish callback-level behavior from computation-level behavior. A custom loss function supplied with compute_loss_func runs after the model forward pass and receives the model outputs, labels, and the number of prediction targets across the accumulated batch. That hook is appropriate when the forward pass remains the same but the loss formula or normalization changes. If the forward pass itself must change, the recipe guidance is to subclass Trainer and override the relevant method rather than trying to accomplish it with a callback. Callbacks are still useful in optimizer and scheduler recipes because training-loop objects are passed through event keyword arguments at the relevant moments. For example, a scheduler-aware callback can inspect the current learning rate at step end and request evaluation when a threshold is crossed. The event model also includes on_pre_optimizer_step, which runs after gradient clipping and before optimizer.step(), and on_optimizer_step, which runs after the optimizer update. Those hooks are designed for monitoring or control decisions around optimizer timing, not for replacing the optimizer implementation itself. Hyperparameter search and checkpointing recipes follow the same separation of concerns. The search system decides which configurations to run and how trials are evaluated, while callbacks can report metrics, react to trial progress, or request early stopping. Checkpointing policies are usually configured through training arguments and default flow behavior, with callbacks used to observe or adjust when save/evaluate/log actions should occur. This lets a training script combine reusable components: one callback for tracking, one for early stopping, and one for a project-specific control rule without making them own the entire loop.

Configuration, Backbones, and Feature Extraction Context

Although this page centers on callbacks, the requested source set also includes configuration.md and backbones.md because Trainer features often operate around model objects whose configuration and feature outputs matter. PreTrainedConfig is the common configuration base for loading and saving model settings from local paths, directories, or pretrained model configurations, and derived configs expose model-specific attributes such as hidden_size, num_attention_heads, num_hidden_layers, and, for text models, vocab_size. A training recipe that saves checkpoints depends on these configurations being serializable with the model. Sources: docs/source/en/main_classes/configuration.md

Backbones are feature-extraction models used for higher-level computer vision tasks such as object detection and image classification. The backbone documentation introduces AutoBackbone, BackboneMixin, BackboneConfigMixin, TimmBackbone, and TimmBackboneConfig, and lists supported model families such as BEiT, ConvNeXT, DINOV2, ResNet, Swin, and ViTDet. In Trainer workflows, these components are not callbacks, but they shape what the model emits, which features are selected, and what configuration is saved with a trained vision checkpoint. Sources: docs/source/en/main_classes/backbones.md

Compact API Reference

Practical Next Steps

Start with configuration before custom code: set logging, evaluation, save cadence, and report_to explicitly in TrainingArguments. Add built-in callbacks next, especially progress, reporting, and early stopping callbacks. Write a custom TrainerCallback only when the desired behavior can be expressed by reading state and returning TrainerControl. If the feature changes loss computation, prefer compute_loss_func; if it changes the forward pass or training-loop mechanics, subclass Trainer. For adjacent APIs, read the Trainer API, Fine-tuning, Configuration and Model Outputs, and Vision Tasks pages next.